  Deep convolutional neural network (CNN) based models are vulnerable to the
adversarial attacks. One of the possible reasons is that the embedding space of
CNN based model is sparse, resulting in a large space for the generation of
adversarial samples. In this study, we propose a method, denoted as Dynamic
Feature Aggregation, to compress the embedding space with a novel
regularization. Particularly, the convex combination between two samples are
regarded as the pivot for aggregation. In the embedding space, the selected
samples are guided to be similar to the representation of the pivot. On the
other side, to mitigate the trivial solution of such regularization, the last
fully-connected layer of the model is replaced by an orthogonal classifier, in
which the embedding codes for different classes are processed orthogonally and
separately. With the regularization and orthogonal classifier, a more compact
embedding space can be obtained, which accordingly improves the model
robustness against adversarial attacks. An averaging accuracy of 56.91% is
achieved by our method on CIFAR-10 against various attack methods, which
significantly surpasses a solid baseline (Mixup) by a margin of 37.31%. More
surprisingly, empirical results show that, the proposed method can also achieve
the state-of-the-art performance for out-of-distribution (OOD) detection, due
to the learned compact feature space. An F1 score of 0.937 is achieved by the
proposed method, when adopting CIFAR-10 as in-distribution (ID) dataset and
LSUN as OOD dataset. Code is available at
https://github.com/HaozheLiu-ST/DynamicFeatureAggregation.
