  Context: Pre-trained models (PTMs) have demonstrated significant potential in
automatic code translation. However, the vulnerability of these models in
translation tasks, particularly in terms of syntax, has not been extensively
investigated. Objective: To fill this gap, our study aims to propose a novel
approach CoTR to assess and improve the syntactic adversarial robustness of
PTMs in code translation. Method: CoTR consists of two components: CoTR-A and
CoTR-D. CoTR-A generates adversarial examples by transforming programs, while
CoTR-D proposes a semantic distance-based sampling data augmentation method and
adversarial training method to improve the model's robustness and
generalization capabilities. The Pass@1 metric is used by CoTR to assess the
performance of PTMs, which is more suitable for code translation tasks and
offers a more precise evaluation in real world scenarios. Results: The
effectiveness of CoTR is evaluated through experiments on real world Java to
Python datasets. The results demonstrate that CoTR-A can significantly reduce
the performance of existing PTMs, while CoTR-D effectively improves the
robustness of PTMs. Conclusion: Our study identifies the limitations of current
PTMs, including large language models, in code translation tasks. It highlights
the potential of CoTR as an effective solution to enhance the robustness of
PTMs for code translation tasks.
