  Deep neural network models are used today in various applications of
artificial intelligence, the strengthening of which, in the face of adversarial
attacks is of particular importance. An appropriate solution to adversarial
attacks is adversarial training, which reaches a trade-off between robustness
and generalization. This paper introduces a novel framework (Layer
Sustainability Analysis (LSA)) for the analysis of layer vulnerability in an
arbitrary neural network in the scenario of adversarial attacks. LSA can be a
helpful toolkit to assess deep neural networks and to extend the adversarial
training approaches towards improving the sustainability of model layers via
layer monitoring and analysis. The LSA framework identifies a list of Most
Vulnerable Layers (MVL list) of the given network. The relative error, as a
comparison measure, is used to evaluate representation sustainability of each
layer against adversarial inputs. The proposed approach for obtaining robust
neural networks to fend off adversarial attacks is based on a layer-wise
regularization (LR) over LSA proposal(s) for adversarial training (AT); i.e.
the AT-LR procedure. AT-LR could be used with any benchmark adversarial attack
to reduce the vulnerability of network layers and to improve conventional
adversarial training approaches. The proposed idea performs well theoretically
and experimentally for state-of-the-art multilayer perceptron and convolutional
neural network architectures. Compared with the AT-LR and its corresponding
base adversarial training, the classification accuracy of more significant
perturbations increased by 16.35%, 21.79%, and 10.730% on Moon, MNIST, and
CIFAR-10 benchmark datasets, respectively. The LSA framework is available and
published at https://github.com/khalooei/LSA.
