  Adversarial robustness, which primarily comprises sensitivity-based
robustness and spatial robustness, plays an integral part in achieving robust
generalization. In this paper, we endeavor to design strategies to achieve
universal adversarial robustness. To achieve this, we first investigate the
relatively less-explored realm of spatial robustness. Then, we integrate the
existing spatial robustness methods by incorporating both local and global
spatial vulnerability into a unified spatial attack and adversarial training
approach. Furthermore, we present a comprehensive relationship between natural
accuracy, sensitivity-based robustness, and spatial robustness, supported by
strong evidence from the perspective of robust representation. Crucially, to
reconcile the interplay between the mutual impacts of various robustness
components into one unified framework, we incorporate the \textit{Pareto
criterion} into the adversarial robustness analysis, yielding a novel strategy
called Pareto Adversarial Training for achieving universal robustness. The
resulting Pareto front, which delineates the set of optimal solutions, provides
an optimal balance between natural accuracy and various adversarial robustness.
This sheds light on solutions for achieving universal robustness in the future.
To the best of our knowledge, we are the first to consider universal
adversarial robustness via multi-objective optimization.
