  Based on the tremendous success of pre-trained language models (PrLMs) for
source code comprehension tasks, current literature studies either ways to
further improve the performance (generalization) of PrLMs, or their robustness
against adversarial attacks. However, they have to compromise on the trade-off
between the two aspects and none of them consider improving both sides in an
effective and practical way. To fill this gap, we propose Semantic-Preserving
Adversarial Code Embeddings (SPACE) to find the worst-case semantic-preserving
attacks while forcing the model to predict the correct labels under these worst
cases. Experiments and analysis demonstrate that SPACE can stay robust against
state-of-the-art attacks while boosting the performance of PrLMs for code.
