  Self-ensemble adversarial training methods improve model robustness by
ensembling models at different training epochs, such as model weight averaging
(WA). However, previous research has shown that self-ensemble defense methods
in adversarial training (AT) still suffer from robust overfitting, which
severely affects the generalization performance. Empirically, in the late
phases of training, the AT becomes more overfitting to the extent that the
individuals for weight averaging also suffer from overfitting and produce
anomalous weight values, which causes the self-ensemble model to continue to
undergo robust overfitting due to the failure in removing the weight anomalies.
To solve this problem, we aim to tackle the influence of outliers in the weight
space in this work and propose an easy-to-operate and effective Median-Ensemble
Adversarial Training (MEAT) method to solve the robust overfitting phenomenon
existing in self-ensemble defense from the source by searching for the median
of the historical model weights. Experimental results show that MEAT achieves
the best robustness against the powerful AutoAttack and can effectively
allievate the robust overfitting. We further demonstrate that most defense
methods can improve robust generalization and robustness by combining with
MEAT.
