1-Dimensional polynomial neural networks for audio signal related problems

by   Habib Ben Abdallah, et al.

In addition to being extremely non-linear, modern problems require millions if not billions of parameters to solve or at least to get a good approximation of the solution, and neural networks are known to assimilate that complexity by deepening and widening their topology in order to increase the level of non-linearity needed for a better approximation. However, compact topologies are always preferred to deeper ones as they offer the advantage of using less computational units and less parameters. This compacity comes at the price of reduced non-linearity and thus, of limited solution search space. We propose the 1-Dimensional Polynomial Neural Network (1DPNN) model that uses automatic polynomial kernel estimation for 1-Dimensional Convolutional Neural Networks (1DCNNs) and that introduces a high degree of non-linearity from the first layer which can compensate the need for deep and/or wide topologies. We show that this non-linearity introduces more computational complexity but enables the model to yield better results than a regular 1DCNN that has the same number of training parameters on various classification and regression problems related to audio signals. The experiments were conducted on three publicly available datasets and demonstrate that the proposed model can achieve a much faster convergence than a 1DCNN on the tackled regression problems.


page 1

page 2

page 3

page 4


Rational Neural Networks for Approximating Jump Discontinuities of Graph Convolution Operator

For node level graph encoding, a recent important state-of-art method is...

Analysis of Deep Neural Networks with Quasi-optimal polynomial approximation rates

We show the existence of a deep neural network capable of approximating ...

Network of Recurrent Neural Networks

We describe a class of systems theory based neural networks called "Netw...

Cluster-Based Partitioning of Convolutional Neural Networks, A Solution for Computational Energy and Complexity Reduction

In this paper, we propose a novel solution to reduce the computational c...

Collaborative Filtering via High-Dimensional Regression

While the SLIM approach obtained high ranking-accuracy in many experimen...

Random Feature Approximation for Online Nonlinear Graph Topology Identification

Online topology estimation of graph-connected time series is challenging...

Please sign up or login with your details

Forgot password? Click here to reset