Context-aware code generation with synchronous bidirectional decoder

Zhang, X. and Zhou, Y. and Yang, G. and Han, Tingting and Chen, Taolue (2024) Context-aware code generation with synchronous bidirectional decoder. Journal of Systems and Software 214 (112066), ISSN 0164-1212.

Abstract

Code generation aims to map natural language descriptions to code snippets. Recent approaches using sequence-to-tree models have shown promising results. However, they generally adopt an autoregressive way to predict the next token based on previous ones and do not consider potential future tokens. To address this issue, we propose Contextor, a novel context-sensitive model employing a bidirectional decoder to generate tokens in two different orders synchronously and interactively. Specifically, we employ two decoders to generate two sequences of different traversals and share their context knowledge via the attention mechanism. As a result, our model can synthesize both previous and future information simultaneously. To alleviate the information leakage problem caused by the teacher-forcing training strategy and bidirectional decoding, we propose an adapted scheduled sampling technique to prevent the decoders from contacting the actual label. Furthermore, Contextor also features a bidirectional beam search algorithm to better interact with both decoders. Experimental results demonstrate that our approach outperforms the state-of-the-art baselines.

Metadata

Available Versions of this Item

• Context-aware code generation with synchronous bidirectional decoder. (deposited 07 Apr 2025 14:23) [Currently Displayed]

Statistics

DownloadsShow export options

Export as JSONXMLCSV

Activity Overview

6 month trend

1Download

6 month trend

77Hits

Additional statistics are available via IRStats2.

Archive Staff Only (login required)

Edit/View Item