  Neural networks are popular and useful in many fields, but they have the
problem of giving high confidence responses for examples that are away from the
training data. This makes the neural networks very confident in their
prediction while making gross mistakes, thus limiting their reliability for
safety-critical applications such as autonomous driving, space exploration,
etc. This paper introduces a novel neuron generalization that has the standard
dot-product-based neuron and the {\color{black} radial basis function (RBF)}
neuron as two extreme cases of a shape parameter. Using a rectified linear unit
(ReLU) as the activation function results in a novel neuron that has compact
support, which means its output is zero outside a bounded domain. To address
the difficulties in training the proposed neural network, it introduces a novel
training method that takes a pretrained standard neural network that is
fine-tuned while gradually increasing the shape parameter to the desired value.
The theoretical findings of the paper are a bound on the gradient of the
proposed neuron and a proof that a neural network with such neurons has the
universal approximation property. This means that the network can approximate
any continuous and integrable function with an arbitrary degree of accuracy.
The experimental findings on standard benchmark datasets show that the proposed
approach has smaller test errors than state-of-the-art competing methods and
outperforms the competing methods in detecting out-of-distribution samples on
two out of three datasets.
