  One of the main drawbacks of deep neural networks, like many other
classifiers, is their vulnerability to adversarial attacks. An important reason
for their vulnerability is assigning high confidence to regions with few or
even no feature points. By feature points, we mean a nonlinear transformation
of the input space extracting a meaningful representation of the input data. On
the other hand, deep-RBF networks assign high confidence only to the regions
containing enough feature points, but they have been discounted due to the
widely-held belief that they have the vanishing gradient problem. In this
paper, we revisit the deep-RBF networks by first giving a general formulation
for them, and then proposing a family of cost functions thereof inspired by
metric learning. In the proposed deep-RBF learning algorithm, the vanishing
gradient problem does not occur. We make these networks robust to adversarial
attack by adding the reject option to their output layer. Through several
experiments on the MNIST dataset, we demonstrate that our proposed method not
only achieves significant classification accuracy but is also very resistant to
various adversarial attacks.
