  In addition to being extremely non-linear, modern problems require millions
if not billions of parameters to solve or at least to get a good approximation
of the solution, and neural networks are known to assimilate that complexity by
deepening and widening their topology in order to increase the level of
non-linearity needed for a better approximation. However, compact topologies
are always preferred to deeper ones as they offer the advantage of using less
computational units and less parameters. This compacity comes at the price of
reduced non-linearity and thus, of limited solution search space. We propose
the 1-Dimensional Polynomial Neural Network (1DPNN) model that uses automatic
polynomial kernel estimation for 1-Dimensional Convolutional Neural Networks
(1DCNNs) and that introduces a high degree of non-linearity from the first
layer which can compensate the need for deep and/or wide topologies. We show
that this non-linearity enables the model to yield better results with less
computational and spatial complexity than a regular 1DCNN on various
classification and regression problems related to audio signals, even though it
introduces more computational and spatial complexity on a neuronal level. The
experiments were conducted on three publicly available datasets and demonstrate
that, on the problems that were tackled, the proposed model can extract more
relevant information from the data than a 1DCNN in less time and with less
memory.
