  To advance the understanding of robust deep learning, we delve into the
effects of adversarial training on self-supervised and supervised contrastive
learning alongside supervised learning. Our analysis uncovers significant
disparities between adversarial and clean representations in standard-trained
networks across various learning algorithms. Remarkably, adversarial training
mitigates these disparities and fosters the convergence of representations
toward a universal set, regardless of the learning scheme used. Additionally,
increasing the similarity between adversarial and clean representations,
particularly near the end of the network, enhances network robustness. These
findings offer valuable insights for designing and training effective and
robust deep learning networks. Our code is released at
\textcolor{magenta}{\url{https://github.com/softsys4ai/CL-Robustness}}.
