Assessing and improving syntactic adversarial robustness of pre-trained models for code translation

Yang, G. and Zhang, X. and Chen, X. and Han, Tingting and Chen, Taolue (2025) Assessing and improving syntactic adversarial robustness of pre-trained models for code translation. Information & Software Technology 181 , ISSN 0950-5849.

Abstract

Context: Pre-trained models (PTMs) have demonstrated significant potential in automatic code translation. However, the vulnerability of these models in translation tasks, particularly in terms of syntax, has not been extensively investigated. Objective: To fill this gap, our study aims to propose a novel approach CoTR to assess and improve the syntactic adversarial robustness of PTMs in code translation. Methods: CoTR consists of two components: CoTR-A and CoTR-D. CoTR-A generates adversarial examples by transforming programs, while CoTR-D proposes a semantic distance-based sampling data augmentation method and adversarial training method to improve the model’s robustness and generalization capabilities. The Pass@1 metric is used by CoTR to assess the performance of PTMs, which is more suitable for code translation tasks and offers a more precise evaluation in real-world scenarios. Results: The effectiveness of CoTR is evaluated through experiments on real-world Java Python datasets. The results demonstrate that CoTR-A can significantly reduce the performance of existing PTMs, while CoTR-D effectively improves the robustness of PTMs. Conclusion: Our study identifies the limitations of current PTMs, including large language models, in code translation tasks. It highlights the potential of CoTR as an effective solution to enhance the robustness of PTMs for code translation tasks.

Metadata

Available Versions of this Item

• Assessing and improving syntactic adversarial robustness of pre-trained models for code translation. (deposited 03 Apr 2025 13:36) [Currently Displayed]

Statistics

DownloadsShow export options

Export as CSVJSONXML

Activity Overview

6 month trend

1Download

6 month trend

12Hits

Additional statistics are available via IRStats2.

See more details

Posted by 1 X users

7 readers on Mendeley

Archive Staff Only (login required)

Edit/View Item