  Instances-reweighted adversarial training (IRAT) can significantly boost the
robustness of trained models, where data being less/more vulnerable to the
given attack are assigned smaller/larger weights during training. However, when
tested on attacks different from the given attack simulated in training, the
robustness may drop significantly (e.g., even worse than no reweighting). In
this paper, we study this problem and propose our solution--locally reweighted
adversarial training (LRAT). The rationale behind IRAT is that we do not need
to pay much attention to an instance that is already safe under the attack. We
argue that the safeness should be attack-dependent, so that for the same
instance, its weight can change given different attacks based on the same
model. Thus, if the attack simulated in training is mis-specified, the weights
of IRAT are misleading. To this end, LRAT pairs each instance with its
adversarial variants and performs local reweighting inside each pair, while
performing no global reweighting--the rationale is to fit the instance itself
if it is immune to the attack, but not to skip the pair, in order to passively
defend different attacks in future. Experiments show that LRAT works better
than both IRAT (i.e., global reweighting) and the standard AT (i.e., no
reweighting) when trained with an attack and tested on different attacks.
