  The robustness of a deep classifier can be characterized by its margins: the
decision boundary's distances to natural data points. However, it is unclear
whether existing robust training methods effectively increase the margin for
each vulnerable point during training. To understand this, we propose a
continuous-time framework for quantifying the relative speed of the decision
boundary with respect to each individual point. Through visualizing the moving
speed of the decision boundary under Adversarial Training, one of the most
effective robust training algorithms, a surprising moving-behavior is revealed:
the decision boundary moves away from some vulnerable points but simultaneously
moves closer to others, decreasing their margins. To alleviate these
conflicting dynamics of the decision boundary, we propose Dynamics-aware Robust
Training (DyART), which encourages the decision boundary to engage in movement
that prioritizes increasing smaller margins. In contrast to prior works, DyART
directly operates on the margins rather than their indirect approximations,
allowing for more targeted and effective robustness improvement. Experiments on
the CIFAR-10 and Tiny-ImageNet datasets verify that DyART alleviates the
conflicting dynamics of the decision boundary and obtains improved robustness
under various perturbation sizes compared to the state-of-the-art defenses. Our
code is available at
https://github.com/Yuancheng-Xu/Dynamics-Aware-Robust-Training.
