  Deep Neural Networks (DNNs) became the standard tool for function
approximation with most of the introduced architectures being developed for
high-dimensional input data. However, many real-world problems have
low-dimensional inputs for which standard Multi-Layer Perceptrons (MLPs) are
the default choice. An investigation into specialized architectures is missing.
We propose a novel DNN layer called Univariate Radial Basis Function (U-RBF)
layer as an alternative. Similar to sensory neurons in the brain, the U-RBF
layer processes each individual input dimension with a population of neurons
whose activations depend on different preferred input values. We verify its
effectiveness compared to MLPs in low-dimensional function regressions and
reinforcement learning tasks. The results show that the U-RBF is especially
advantageous when the target function becomes complex and difficult to
approximate.
