betaFeedback

WorksComputer Science

Enhancing Repository-Level Code Completion with Reinforcement Learning in Real-World Scenarios

Abstract

The rapid development of large language models (LLMs) for code, which closely aligns with the practical needs of real-world software development, has sparked growing attention in repository-level code completion. However, existing Code LLMs present challenges in focusing on suitable contexts from the repository and in deeply reasoning about cross-file dependencies. To address these challenges, we propose a novel reinforcement learning framework for repository-level code completion. To better understand both in-file and cross-file contexts, we employ identifier-driven intent recognition to capture completion intent, thereby improving the model's performance in real-world scenarios. To enhance cross-file reasoning, we propose a reward-driven completion learning to effectively reward in complex repository completion scenarios. To guide LLMs in completing code that aligns with intended functionality and repository context, we introduce a selective-exploration strategy that directs the model to focus on low-confidence, high-reward tokens, promoting the exploration of valuable, underexplored completion patterns. Experimental results show that our approach significantly improves the performance of Code LLMs. The code and the dataset are available at https://github.com/zhenlongDai/RepoCompletion.

Introduction

With the rapid development of large language models (LLMs) for code ([21] (Zheng et al. 2023b)), these LLMs have demonstrated advanced capabilities in code understanding and completion. Many LLMs are integrated into IDEs to support developers’ daily development, one of the most common usage scenarios is code completion ((Zhou et al., 2022; Wang et al., 2023a)). Previous studies have explored code completion tasks

on different granularities, such as line-level code completion ([30, 20]), function-level code completion ([27] (Quan et al. 2025)). More recently, the repository-level code completion ((Zhang et al., 2024)) has become increasingly important because it better aligns with the practical needs of developers in real-world software development. However, repo-level code completion is a non-trivial task by considering the following situations:

As shown in Figure 1 (s1), this example shows the LLM incorrectly focusing on method signatures in the local (i.e., in-file) context, while ignoring the broader global (i.e., cross-file) context. The LLM incorrectly invokes processData(product, quantity), which fails to reflect the intended action of placing an order. The proper call is processPurchase(product, quantity) (e.g., colored in green), which aligns with the intended purchase logic defined in the PurchaseProcessor class, imported from the cross-file context. This situation highlights that the LLM can’t correctly focus on local context or global context. As shown in Figure 1 (s2), this example shows an LLM’s failure to deeply reason about the target cross-file object. The LLM incorrectly calls dbConnection.connect() directly after instantiating newUser neglecting the initialization logic newUser.save() encapsulated within the UserManager class, which internally manages the database interaction.

Situations of repo-level code completion

Figure 1. Situations of repo-level code completion.

Therefore, a key research question regarding repo-level code completion stands out, i.e., how to improve the model’s capability to utilize both cross-file and in-file context effectively? It is a challenging task: (i) Guiding LLMs to focus on the suitable context (e.g., in-file context or cross-file context or both) remains a non-trivial problem. Different parts of the codebase depend on vastly different types of contextual information. Some completions can be resolved using local (in-file) scope, while others require a broader, cross-file perspective. LLMs still lack the capability to decide which type of context is relevant for a given completion task [12]. (ii) LLMs still struggle to reason deeply about the cross-file information due to complicated project dependencies and code structures. From the repository perspective, code completion requires the LLM to have a global view that spans multiple files and modules. This means a deep understanding of how different components are architected and interact.

To tackle the above challenges, we propose a novel reinforcement learning (RL) approach for repo-level code completion, which is designed to improve the model’s completion ability in different completion scenarios. To address the challenge of focusing on a suitable context, we leverage identifier-driven intent recognition to capture the completion intent. Specifically, we adopt the GRPO (Shao et al., 2024) method for autonomous learning and design an intent reward based on the matching degree between identifiers in the predicted and real code, guiding the LLM to focus on the correct context. To address the challenges of dependency resolution and cross-file reasoning, we propose a reward-driven completion learning utilizing a reward function for repo-level code completion, providing continuous and granular reward values based on the distance of the code editing as feedback for cross-file reasoning. To guide LLMs to effectively complete code that meets the intended functionality and adheres to the repository-level context, we design a selective-exploration strategy optimization that focuses on low-confidence, high-reward tokens in the model’s multiple responses, reflecting divergent thought paths and motivating the model to explore underexplored yet more valuable code completion patterns.

In summary, our paper makes the following contributions: (1) A reinforcement learning method for repo-level code completion, which effectively addresses the issue of rewards in complex repository completion scenarios and improves cross-file reasoning, enabling the model to learn autonomously on real-world repository data. (2) The first deep research into LLMs’ multi-scenario code completion ability, enhancing the model’s ability to focus on a suitable repository context through intent recognition and selective exploration. (3) IntentCoder achieves state-of-the-art (SOTA) performance on repo-level code completion tasks with code LLM (7B parameters) on RepoBench, demonstrating strong and stable improvements.

Related Work

Large Language Models for Code. A wide range of LLMs [15, 7, 8] (Wang et al. 2023b; Zheng et al. 2023a) tailored for code-related tasks, such as code comprehension and generation [10, 9], have been developed and widely adopted in software engineering to enhance development efficiency [5, 31]. Some open-source models, Deepseek-Coder [16] and Qwen2.5-Coder [19, 11], show strong code generation abilities. CodeLLMs have shown strong performance on benchmarks [24, 2] (Lu et al. 2021; Raychev et al. 2016; Yu et al. 2024) for single-file code completion, where they can predict whole lines of code effectively. However, their performance in repo-level code completion remains limited, as LLMs struggle to capture the full repository context due to token size restrictions (Phan et al., 2024), focusing mainly on local context within individual files and neglecting cross-file dependencies [28].

Repository-level Code Completion. Repository-level code completion aims to provide code completion by leveraging the context of a repository. Given the large size of repositories and model context limitations, many approaches adopt the retrieve-then-generate paradigm [1, 6]. Methods like A3-CodGen [29], CodePlan [4], and RepoFuse [28] use static code analysis for contextual integration to improve code completion. RepoCoder (Zhang et al., 2023) refines retrieval using feedback from generated code, while RLCoder (Wang et al., 2024) improves it with a stop signal mechanism. Despite advancements in context retrieval, effectively utilizing it remains challenging. FT2Ra [17] and CoCoMIC [13] integrate retrieved content, but require architectural changes, making them unsuitable for pre-trained LLMs. In contrast, aixcoder-v2 [26] uses the CoLT method to assimilate long contexts for improved code completion. Unlike previous methods, we use online reinforcement learning to enhance the LLM’s performance in various scenarios.

Methodology

Task Definition and Motivation

Given an incomplete code snippet xx (i.e., in-file context Cin-fileC_{\mathrm{in\text{-}file}}) and its cross-file contexts CcrossC_{\mathrm{cross}} in a specific repository, the task of repository-level code completion is to generate the completion yy. More formally, the task’s objective is to generate the code completion yy by a model πθ\pi_{\theta} such that the conditional probability Pθ(y∣Cin-file,Ccross)P_{\theta}(y\mid C_{\mathrm{in\text{-}file}}, C_{\mathrm{cross}}) is maximized. The definitions of in-file and cross-file contexts are in Appendix § 8.§ 1, and the task motivation is in Appendix § 8.§ 2.

Preliminary

Group Relative Policy Optimization (GRPO) is a method of reinforcement learning that explores the potential of LLMs to develop reasoning capabilities, focusing on their self-evolution through a pure RL process [16] (Shao et al. 2024). GRPO is applied to estimate the advantages of language model generations by comparing responses within a query-specific group.

Specifically, in repo-level completion task, given an input q=(Cin-file,Ccross)q=(C_{\mathrm{in\text{-}file}}, C_{\mathrm{cross}}), GRPO samples a group of outputs {y1,y2,…,yG}\{y_1,y_2,\ldots,y_G\} from the old policy model πθold\pi_{\theta_{\mathrm{old}}}, and then optimizes the policy model πθ\pi_{\theta} by maximizing the following objective:

JGRPO(θ)=Eq∼P(Q),{yi}i=1G∼πθold(y∣q)[1G∑i=1Gmin⁡(πθ(yi∣q)πθold(yi∣q)Ai,clip⁡(πθ(yi∣q)πθold(yi∣q),1−ϵ,1+ϵ)Ai)−βDKL(πθ∥πref)].(1)J_{\mathrm{GRPO}}(\theta)=\mathbb{E}_{q\sim P(Q),\{y_i\}_{i=1}^{G}\sim\pi_{\theta_{\mathrm{old}}}(y\mid q)} \left[\frac{1}{G}\sum_{i=1}^{G}\min\left(\frac{\pi_{\theta}(y_i\mid q)}{\pi_{\theta_{\mathrm{old}}}(y_i\mid q)}A_i,\operatorname{clip}\left(\frac{\pi_{\theta}(y_i\mid q)}{\pi_{\theta_{\mathrm{old}}}(y_i\mid q)},1-\epsilon,1+\epsilon\right)A_i\right)-\beta D_{\mathrm{KL}}(\pi_{\theta}\Vert\pi_{\mathrm{ref}})\right]. \tag*{(1)}

where ϵ\epsilon and β\beta are hyper-parameters, DKLD_{\mathrm{KL}} is the penalty function of Kullback-Leibler divergence. πref\pi_{\mathrm{ref}} is the reference model, which is initial by the policy model πθ\pi_{\theta}. AiA_i is the advantage for the ii-th response, computed using a group of rewards {r1,r2,…,rG}\{r_1,r_2,\ldots,r_G\} corresponding to the outputs within each group:

Ai=ri−mean⁡(r1,r2,…,rG)std⁡(r1,r2,…,rG)(2)A_i=\frac{r_i-\operatorname{mean}(r_1,r_2,\ldots,r_G)}{\operatorname{std}(r_1,r_2,\ldots,r_G)} \tag*{(2)}

Completion-Focused Reinforcement Learning

Even though LLM receives sufficient repository context information, it often fails to effectively leverage and understand it for accurate completion. Inspired by reinforcement learning, which helps mitigate the reliance on empirical memory for code completion, we propose completion-focused reinforcement learning based on GRPO (§ 3.2) to enable the model to autonomously learn code completion in a repository.

Reward-Driven Completion Learning. Employing reward feedback based on result matching presents challenges in complex and diverse cross-file completion scenarios, as the model often fails to receive rewards due to the sparsity of signals when directly matching the ground truth code. To address this challenge, we propose reward-driven completion learning, which provides more precise and granular reward feedback in the complex repo-level code completion environment, enabling the model to receive effective reward signals that enhance its cross-file reasoning ability.

Specifically, given in-file context Cin-fileC_{\mathrm{in\text{-}file}} and the cross-file context Ccross={Ci}i=1nC_{\mathrm{cross}}=\{C_i\}_{i=1}^{n} in repository, the ii-th output yiy_i in a group is sampled by the old policy model πθold\pi_{\theta_{\mathrm{old}}}, as described by the following equations:

yi=πθold(Cin-file,Ccross).(3)y_i=\pi_{\theta_{\mathrm{old}}}(C_{\mathrm{in\text{-}file}},C_{\mathrm{cross}}). \tag*{(3)}

Following the prompt of repository-level completion in Figure 4 (provided in Appendix § 8.§ 4), we obtained a group of generated code {yi}i=1G\{y_i\}_{i=1}^{G} through multiple samplings by Formula 3. Completion reward functions that evaluate the generated code are defined from the following perspectives: (1) completion reward provides a relative quality ranking of the generated code completions. The completion reward RcR_c for the ii-th output yiy_i is defined as follows:

The prompt of repository-level completion

Figure 4. the prompt of repository-level completion.

Rc=fEdit(yi,y^),(4)R_c = f_{\mathrm{Edit}}(y_i,\hat{y}), \tag*{(4)}

where y^\hat{y} is the ground truth code, fEditf_{\mathrm{Edit}} is the Levenshtein distance algorithm Yujian and Bo, 2007 to compare the character differences (i.e., insertions, deletions, substitutions) between generated code and ground truth code. This reward provides fine-grained, continuous feedback (ranging from 0 to 1), ensuring the model receives quality feedback even without exact matches. (2) Format reward RfR_f ensures the model’s output is evaluable by requiring the response to follow a specific format for reasoning and answers. A regular expression checks that these markers are present and correctly ordered. (3) Length-based reward: Extra code lines may receive spurious rewards, hindering accurate feedback and reducing training efficiency. To address this, we propose a Length-based reward RLR_L to regulate the number of generated lines for better alignment with code completion needs:

RL(yi)={1,if len(yi)=10,otherwise,(5)R_L(y_i) = \begin{cases} 1, & \text{if } len(y_i) = 1 \\ 0, & \text{otherwise,} \end{cases} \tag*{(5)}

where lenlen denotes the function that calculates the number of code lines. The justification for the reward designs is provided in Appendix § 8.§ 3.1. Based on these reward functions, we define the reward function RR as follows:

R=αc∗Rc+αf∗Rf+αL∗RL,(6)R = \alpha_c * R_c + \alpha_f * R_f + \alpha_L * R_L, \tag*{(6)}

where αk\alpha_k is hyper-parameters. As shown in Algorithm 1, we use the reward function RR to train the model πθ\pi_\theta, enabling the model to autonomously learn from repository data.

Algorithm 1 Completion-Focused Reinforcement Learning

1: Input: initial policy model πθinit\pi_\theta^{init}, reward function RR, dataset D={Cin−file,Ccross}\mathcal{D} = \{C_{\mathrm{in-file}}, C_{\mathrm{cross}}\}; hyperparameters α\alpha 2: policy model πθ←πθinit\pi_\theta\gets\pi_\theta^{init} 3: reference model πref←πθ\pi_{\mathrm{ref}} \gets\pi_\theta 4: for step =1,…,M= 1, \ldots, M do 5:     Sample a batch Db\mathcal{D}_b from D\mathcal{D} 6:     Update the old policy model πθold←πθ\pi_{\theta_{\mathrm{old}}} \gets\pi_\theta 7:     q←{Cin−file,Ccross}∈Dbq \gets\{C_{\mathrm{in-file}}, C_{\mathrm{cross}}\} \in\mathcal{D}_b 8:     Sample {yi}i=1G∼πθold(⋅∣Cin−file,Ccross)\{y_i\}_{i=1}^{G} \sim\pi_{\theta_{\mathrm{old}}}(\cdot|C_{\mathrm{in-file}}, C_{\mathrm{cross}}) for q∈Dbq \in\mathcal{D}_b 9:     {Ri}i=1G←R(yi,y^)\{R_i\}_{i=1}^{G} \gets R(y_i,\hat{y}) for yi∈{yi}i=1Gy_i \in\{y_i\}_{i=1}^{G} (Equation (6)) 10:     Compute AiA_i for yi∈{yi}i=1Gy_i \in\{y_i\}_{i=1}^{G} (Equation (2)) 11:     for iteration =1,…,μ= 1, \ldots, \mu do 12:         Update πθ\pi_\theta by maximizing JGRPO(θ)\mathcal{J}_{\mathrm{GRPO}}(\theta) (Equation (1)) 13:     end for 14: end for 15: Output: πθ\pi_\theta

Identifier-Driven Intent Recognition. In real-world code completion scenarios, the performance of LLMs is often enhanced by utilizing cross-file context from the repository [14] (Zhang et al. 2024). However, the model struggles to focus on suitable contexts from in-file and cross-file contexts [12], resulting in incorrect code completions. To address this challenge, we propose Identifier-Driven Intent Recognition, which is designed to guide the LLM in identifying the developer’s completion intent and understanding code contexts, thereby correctly focusing on suitable contexts. Specifically, as shown in Figure 4, we instruct the model’s reasoning process in the prompt to focus on the user’s intent of code completion (e.g., colored in blue), directing its reasoning towards the intent of completion in the reasoning process. Given that identifiers contain semantic information from the code, we utilize an identifier-based reward function RIR_I to evaluate the degree of correctness in the model’s intent recognition. As shown in Figure 2, we parse the code and extract the identifiers (e.g., variable, function, and class names) from the model’s prediction yy and ground truth code y^\hat{y}. The identifier-based reward function RIR_I is calculated as follows:

Figure 2. Overview of IntentCoder. (a) Illustration of the Completion-Focused RL process. (b) Illustration of the Selective-Exploration Strategy Optimization.

RI=F1(fid(πθold(Cin−file,Ccross)),fid(y^)),(7)R_I = F_1\left(f_{\mathrm{id}}\left(\pi_{\theta_{\mathrm{old}}}(C_{\mathrm{in-file}}, C_{\mathrm{cross}})\right), f_{\mathrm{id}}(\hat{y})\right), \tag*{(7)}

where fidf_{\mathrm{id}} extracts the sets of identifiers from code. F1F_1 is the harmonic mean of the precision and recall metric, which is used to evaluate the degree of intent consistency between the predicted and ground truth code. This approach enables the model to properly consider contextual information and adjust its focus based on the situation, thereby effectively leveraging the advantages of both in-file and cross-file contexts to improve performance.

To effectively coordinate the intent and completion rewards, we join them as follows:

Ric=(1+RI)∗Rc,(8)R_{ic} = (1 + R_I) * R_c, \tag*{(8)}
R=αic∗Ric+αf∗Rf+αL∗RL,(9)R = \alpha_{ic} * R_{ic} + \alpha_f * R_f + \alpha_L * R_L, \tag*{(9)}

where RR is the overall reward function, α∗\alpha_* is hyper-parameter. By combining these two reward signals, the model is encouraged to prioritize completing the code in a way that both fulfills the user’s intended functionality and adheres to the repo-level context. When the edit distance between two generated codes is similar in a group of outputs {yi}i=1G\{y_i\}_{i=1}^{G}, the intent reward acts as a coefficient, allowing for a more accurate reward assignment and reducing the lack of distinction in reward values. The justification and further analysis of reward design are provided in Appendix § 8.§ 3.2.

Selective-Exploration Strategy Optimization. The model’s multiple responses to the same completion task reflect different thought paths, with some responses aligning in part as a consensus, while others show divergence on key points. Rewarding the key information where the model shows uncertainty can help cross-file reasoning and contextual attention in the correct contexts. As shown in Figure 2, both candidates correctly identify ListCellViewAdapter (a class in cross-file context). The difference is the generic type (ProgramData vs. MySchedule), which is defined in different files and requires cross-file reasoning to resolve. This type of token (e.g., colored in green/red) significantly influences the reasoning strategy. However, result-level reward assigns credit to the entire line with the same reward, even when the correct decision depends on specific tokens. During the RL process, some key tokens (e.g., colored in green) in the group may have a lower generation probability but receive rewards higher than the group’s average. Based on the above observations, we propose a selective-exploration strategy optimization that emphasizes the tokens that correspond to competing cross-file hypotheses (e.g., ProgramData, MySchedule) rather than the shared tokens that are predictable (e.g., ListCellViewAdapter). The key idea is to encourage the model to focus on the competing tokens that represent the high-value and critical aspects of cross-file reasoning and the solution.

Specifically, for the tt-th token of the ii-th output yi,ty_{i,t} within the group, the confidence function PP and value function VV are as follows:

P(yi,t)=log⁡πθ(yi,t∣q,y<t)−1N∑j=1G∑t=1Lilog⁡πθ(yi,t∣q,y<t),(10)P(y_{i,t}) = \log\pi_{\theta}(y_{i,t}|q,y_{<t}) - \frac{1}{N}\sum_{j=1}^{G}\sum_{t=1}^{L_i}\log\pi_{\theta}(y_{i,t}|q,y_{<t}), \tag*{(10)}
V(yi,t)=A(yi,t)−1G∑j=1GA(yj),V(y_{i,t}) = A(y_{i,t}) - \frac{1}{G}\sum_{j=1}^{G} A(y_j),

where NN is the total number of tokens in a group of outputs, LiL_i is the number of tokens in the ii-th output. Then the tokens that have high value and low confidence are selected as follows:

Ts={yi,t∣P(yi,t)<0,V(yi,t)>0},(11)T_s = \{y_{i,t} \mid P(y_{i,t}) < 0, V(y_{i,t}) > 0\}, \tag*{(11)}

where TselectT_{\mathrm{select}} is the set of selected tokens. To simplify the expression and enhance readability, we define the function Adv(yi,t)Adv(y_{i,t}) as follows:

Adv(⋅)=min⁡(πθ(yi∣q)πθold(yi∣q)Ai, clip⁡(πθ(yi∣q)πθold(yi∣q),1−ϵ,1+ϵ)Ai).(12)Adv(\cdot) = \min\left(\frac{\pi_{\theta}(y_i|q)}{\pi_{\theta_{\mathrm{old}}}(y_i|q)}A_i,\, \operatorname{clip}\left(\frac{\pi_{\theta}(y_i|q)}{\pi_{\theta_{\mathrm{old}}}(y_i|q)},1-\epsilon,1+\epsilon\right)A_i\right). \tag*{(12)}

Based on the function AdvAdv, the training of the model πθ\pi_{\theta} for the position of token yi,ty_{i,t} is defined as maximizing the following objective:

J(yi,t)={Adv(yi,t)−βDKL(πθ∥πref),yi,t∉TsAdv(yi,t),yi,t∈Ts,(13)\mathcal{J}(y_{i,t}) = \begin{cases} Adv(y_{i,t})-\beta D_{KL}(\pi_{\theta}\|\pi_{\mathrm{ref}}), & y_{i,t}\notin T_s \\ Adv(y_{i,t}), & y_{i,t}\in T_s, \end{cases} \tag*{(13)}

where token yi,t∈Tselectedy_{i,t} \in T_{\mathrm{selected}} removes the KL penalty compared to other tokens. By applying fine-grained gradient updates to tokens in different areas, we adjust the model’s attention away from tokens where it already has high confidence and shift it towards those tokens that have the potential to improve the overall output. This approach encourages the model to gain a deeper understanding of code completion by focusing on divergent contextual areas and exploring valuable code completion patterns.

Experiments

Dataset

To study repository-level code completion in the context of in-file and cross-file scenarios, we construct the ICRepo dataset (In-file & Cross-file Repository-level Completion) for model training. We collect 32K repo-level code completion examples from the RepoBench-C and RepoBench-P datasets [31], with an equal number of Python and Java examples. We contain three completion scenarios as follows: Cross-File-First (CF-F), Cross-File-Random (CF-R), In-File (IF). Overall statistics of the dataset are given in Table 1. The training dataset encompasses a variety of real-world code completion scenarios, ensuring the model is exposed to diverse challenges. By incorporating both in-file and cross-file contexts, the dataset covers practical code completion situations that developers frequently encounter in actual projects. For the test dataset, we use Repobench-Java and Repobench-Python [31]. Further details are provided in Appendix § 8.§ 5.

LanguageSetsNum.Completion position
Java&Pythontrain32,00010,10011,30010,600
Java&Pythonval1,000300400300
Java&Pythontest6,0002,0002,0002,000

Table 1. Dataset Statistics of ICRepo.

Baselines

We consider the following open-source Code LLMs for comparison, which are widely used in code completion tasks. To simplify the model names, we have removed the “instruct” suffix in the following introductions: StarCoder2 (Lozhkov et al., 2024): StarCoder2-15B and StarCoder2-7B. DeepSeek-Coder [16]: Deepseek-Coder-6.7B and DeepSeek-Coder-33B. CodeLlama (Roziere et al., 2023): CodeLlama-7B and CodeLlama-34B. Qwen2.5-Coder [19]: Qwen2.5-Coder-7B and Qwen2.5-Coder-33B. aixcoder-7B-v2 [26] is currently the SOTA 7B model in repo-level code completion. Then we compare our method with the following methods in the training model for repo-level code completion: fine-tuning [18], GRPO [16], CoLT [26]. Further Details are provided in Appendix § 8.§ 6.

Implementation Details

We train IntentCoder based on Deepseek-Coder-6.7B and Qwen2.5-Coder-7B, respectively. For each prompt of training data, the old policy model πθold\pi_{\theta_{\mathrm{old}}} generates 6 outputs in a group using a temperature of 0.7 and top-p sampling with a value of 1.0. To improve training efficiency, we use LoRA [18], vllm [23], and deep-speed (Rasley et al., 2020) for train the model πθ\pi_{\theta}. Further details are provided in the Appendix § 8.§ 7.

Evaluation Metrics

Following the previous work [31, 14, 22] (Zhang et al. 2023), we report performance in two main categories to evaluate the performance of codeLLMs: Code Match: To evaluate the accuracy of the predicted code lines, we use Exact Match (EM) and Edit Similarity (ES) [25]. Identifier Match [13, 12]: To evaluate the model’s ability to understand the completion intent and predict the correct APIs, we use the following metrics about Identifier: Exact Match (EM), Precision, Recall and F1. Further details can be found in Appendix § 8.8.

Experimental Results

Repo-level Code Completion Evaluation and Method Analysis. To evaluate the effectiveness of IntentCoder, we apply our training method to Qwen2.5-Coder-7B and DeepSeek-Coder-6.7B, and then compare it with nine Code LLMs and two strong training methods for LLMs. Table 2 shows the experimental results of our model and other baselines. It is obvious that: (1) Regarding the comparison with other models, IntentCoder outperforms all strong baselines by a large margin in both code match and identifier match, surpassing even larger models. For example, IntentCoderQwen_{\mathrm{Qwen}} achieves 42.47% and 35.93% in Code EM for Java and Python, surpassing the next best model (i.e., Qwen2.5-Coder-32B) by a significant margin of 6.53% and 5.87%. Similarly, in terms of F1 in Identifier Match, IntentCoderQwen_{\mathrm{Qwen}} outperforms the next best model (i.e., Qwen2.5-Coder-32B) by a significant margin of 5.97% and 4.97%. Compared to models with the same parameters in Code EM, IntentCoder{sub}Qwen outperforms the current SOTA model (i.e., aixcoder-7B-v2) by a significant margin of 8.53% in Java and 12.07% in Python, which clearly demonstrates our model’s code completion capability. (2) Regarding the comparison with other methods, our approach demonstrates substantial improvements across all evaluation metrics over other training methods on two main strong baselines (e.g., Qwen2.5-Coder-7B, DeepSeek-Coder-6.7B). Specifically, IntentCoder{sub}DeepSeek outperforms DeepSeek-Coder-6.7B by up to 11.40% and 20.30% in Code EM for Java and Python. Similarly, IntentCoder{sub}Qwen outperforms Qwen2.5-Coder-7B by up to 15.67% and 15.96%. Overall, **IntentCoder shows substantial and stable improvements compared to other models and methods.** Further experiments and method analysis are discussed in {xref}Appendix A.9`.

ModelRepoBench-Java Code Match EMRepoBench-Java Code Match ESRepoBench-Java Identifier Match EMRepoBench-Java Identifier Match PrecisionRepoBench-Java Identifier Match RecallRepoBench-Java Identifier Match F1RepoBench-Python Code Match EMRepoBench-Python Code Match ESRepoBench-Python Identifier Match EMRepoBench-Python Identifier Match PrecisionRepoBench-Python Identifier Match RecallRepoBench-Python Identifier Match F1
StarCoder2-7B12.2029.7815.3724.6428.1925.091.8014.473.837.7213.258.40
CodeLlama-7B12.4048.5015.9035.6041.6036.464.4725.566.3718.2626.6818.84
aixcoder-7B-v233.0768.9738.7060.5765.9761.0723.2059.9429.6354.7062.8455.41
Starcoder2-15B12.2028.5915.6724.5127.7724.921.8313.704.077.6613.058.34
CodeLlama-34B12.1345.3915.4333.5939.1134.014.4026.026.7018.9727.5519.07
DeepSeek-Coder-33B26.7062.0331.5353.1157.9953.3914.0345.0018.5737.8145.4938.30
Qwen2.5-Coder-32B35.0771.1341.1063.1265.2862.5629.4063.8235.4057.5261.6957.43
DeepSeek-Coder-6.7B30.2066.2835.9757.6762.8157.9414.9743.8719.2038.1347.2038.94
DeepSeek-Coder-6.7B{sub}fine-tuning`38.9374.3744.7766.2366.1765.2231.2766.3637.5359.4458.7657.66
DeepSeek-Coder-6.7B{sub}GRPO`35.4771.2941.5763.9568.1363.8334.7068.8240.9362.8760.6460.39
DeepSeek-Coder-6.7B{sub}CoLT`35.2371.5241.0761.0961.9461.5130.2067.0340.2053.6659.2656.32
IntentCoder{sub}DeepSeek`41.6077.6447.7369.7469.1868.5335.2771.1242.1064.4162.9862.39
Qwen2.5-Coder-7B26.8062.7732.7054.7457.0653.8119.9752.3925.2046.5552.4546.51
Qwen2.5-Coder-7B{sub}fine-tuning`37.4374.7743.5065.9365.7864.8831.5366.6137.5359.5958.6957.81
Qwen2.5-Coder-7B{sub}GRPO`37.5073.4943.6065.8767.5865.0235.1368.7141.2062.2160.1059.86
Qwen2.5-Coder-7B{sub}CoLT`38.8075.3342.4061.4062.4361.9133.0069.8840.2061.2959.8160.54
IntentCoder{sub}Qwen`42.4777.5248.5769.9569.6068.7535.9371.5842.7065.5763.8063.37

Table 2. Evaluation results on the RepoBench-Java/Python dataset. All results in the table are reported in percentage (%). The best method is in boldface, and the second best method is underlined for each metric.

intent recognition (IDIR), reward-driven completion learning (RDCL), completion-focused RL (CFRL), selective-exploration strategy optimization (SESO)) of our approach separately. For IDIR and RDCL components, we separately remove their design of the reward function and retain the basic RL process (i.e., GRPO). Similarly, for CFCL, we remove all reward design (i.e., IDIR, RDCL), retain the intention prompt and adopt Code EM as a substitute for reward. As illustrated in Table 3, The experimental results show that: (1) No matter which component we drop, it consistently hurts the overall performance of IntentCoder across all metrics, signaling the importance and effectiveness of all four components. When all the components were removed, the performance of the model significantly declined. (2) When CFCL is removed, the code match and identifier match metrics, except for the EM metric, show a more significant drop than when removing IDIR and RDCL separately. This justifies the importance and necessity of CFCL in effectively coordinating the intent and completion rewards, which both fulfill the user’s intended functionality and adhere to the repository-level context. Notably, EM is slightly higher than removing any one of them (e.g., colored in gray), since we use the exact code match as an alternative to the reward function. Similarly, the ablation study on Java is provided in Appendix Table 7.

ModelCode Match EMCode Match ESIdentifier Match EMIdentifier Match PrecisionIdentifier Match RecallIdentifier Match F1
IntentCoder35.9371.5842.7065.5763.8063.37
w/o IDIR34.6069.9841.2763.7463.1062.05
w/o RDCL34.3068.9740.2062.2460.8760.25
w/o CFCL34.4767.7340.7062.0659.6159.45
w/o SESO34.4770.4340.9764.5862.9962.39
w/o All19.9752.3925.2046.5552.4546.51

Table 3. Results of Ablation study on the RepoBench-Python dataset in Qwen2.5-Coder-7b.

Contributions of Each Component. In this RQ, we conduct an ablation study on IntentCoder{sub}Qwen` to assess the contribution of different techniques by removing key components (i.e., identifier-driven

Robustness Evaluation. To evaluate the robustness of our training method, we assess the influence of different training methods on reducing cross-file context-induced failures (i.e., fail) and improving the accuracy of completion (i.e., pass). As shown in Table 4, compared to the baseline models, our training method significantly reduces cross-file context-induced failures and improves completion accuracy, thereby improving overall performance. For instance, our approach has reduced the number of failures of DeepSeek-Coder-6.7b on the Python dataset from -130 to -41 and increased the number of pass from 189 to 709, representing an overall improvement of 668. Compared with other methods, our approach reduces the failure rate comparable to GRPO and significantly improves the pass rate compared to the other methods, thereby achieving significant overall improvement. This justifies the importance and robustness of our method in real-world code completion scenarios, effectively handling and understanding in-file and cross-file contexts.

MethodJavaPython
failpassoverallfailpassoverall
DeepSeek-Coder-6.7B-131+351+220-130+189+59
fine-tuning-117+599+482-61+609+548
GRPO-78+456+378-40+691+651
CoLT-91+462+371-156+672+516
IntentCoderdeepseek_{\mathrm{deepseek}}-83+645+562-41+709+668
Qwen2.5-Coder-7B-128+301+173-138+225+87
fine-tuning-105+597+492-113+547+434
GRPO-54+548+494-62+604+542
CoLT-102+635+533-125+603+478
IntentCoderQwen_{\mathrm{Qwen}}-58+701+643-60+626+566

Table 4. Result of Exact Match (EM) count.

Various Scenarios Evaluation. To further evaluate the effectiveness of IntentCoder in various completion scenarios, we compare our approach with other methods (i.e., fine-tuning, GRPO) using the Code EM metric. As shown in Figure 3, in all completion scenarios (CF-F, CF-R, IF) that are illustrated in Appendix § 8.5.1, our method generally outperforms other methods, although there is one instance where it performs slightly worse. The experimental results show that: (1) In the IF setting, IntentCoder performs significantly better than other methods, indicating that IntentCoder is not reliant on guessing based on prior cross-file information and effectively handles in-file contexts. (2) Similarly, in the CF-F setting, IntentCoder outperforms other methods consistently, demonstrating that it does not incorrectly depend on in-file context or previous cross-file information, and accurately handles cross-file dependencies, even when encountered for the first time. Overall, compared to the baseline, IntentCoder achieves stable improvement and effectively handles different scenarios.

Figure 3. Performance of IntentCoder in Various Completion Scenarios (CF-F, CF-R, IF).

Runtime latency and Case Study. To evaluate the runtime latency, We conducted experiments on a test set of 3,000 samples on IntentCoder using the vLLM inference library. RepoBench’s max input tokens is 2048, the context fed into IntentCoder averages 1121 tokens, keeping max input latency under 238 ms. The maximum output token length of the <think> block is 119, with an average length of 20 tokens. The additional latency is small, with an average delay of less than 25 ms and a maximum delay not exceeding 145 ms. We believe the small increase in latency is acceptable, as the difference is imperceptible to humans and accuracy is improved. To illustrate the performance of IntentCoder compared to other models, we have included a case study in the Appendix § 8.10.

Conclusion

This research aims to improve the Code LLM’s capability of repo-level code completion by enhancing their capabilities in handling both in-file and cross-file contexts. To perform this task, we propose a novel training approach designed to improve LLMs’ understanding of complex repo-level code by incorporating reward-driven learning for better cross-file reasoning and intent recognition to focus on suitable contexts from in-file and cross-file contexts. Selective-exploration strategy optimization guides the model in exploring more valuable code completion patterns, resulting in more accurate and context-aware code suggestions. Experimental results demonstrate that the effectiveness of IntentCoder. We hope our work advances LLMs to handle real-world repo-level challenges and provides valuable insights into improving code LLMs for software engineering development.

Limitations.

Several limitations are concerned with our work. Firstly, we do not utilize execution-based rewards due to the complexities involved in constructing executable environments, which are difficult to scale for large datasets. Additionally, the time-intensive nature of software building and testing renders this approach impractical for model training. Although execution-based rewards represent a valuable research avenue, our focus is on rewards derived from edit similarity and identifier matching. This design is more flexible, scalable, and efficient, enabling practical application across large datasets, even when execution is not feasible, and supporting sparse rewards.

Secondly, in this work, we intentionally focused only on strengthening the generation side. i.e., improving code completion results by effectively using cross-file and in-file context. IntentCoder can naturally be integrated into any retrieval-augmented pipeline as the generator component. However, we believe that research on how to enable LLMs to search and select contexts in a repository within an acceptable time cost and context input limitation is an interesting topic for our future work.

Thirdly, while larger and more capable LLMs may yield better reasoning results during RL training, they come with higher computational costs, such as increased GPU memory usage and longer training times. Due to resource constraints and our goal of supporting community research, we only trained on 7B. We have released our replication package to enable others to replicate our work and verify their ideas.

Acknowledgements

This research was supported by the National Natural Science Foundation of China (No.62507040), the “Pioneer” and “Leading Goose” R&D Program of Zhejiang (2025C02022), the Ningbo “Yongjiang Talent Program” Youth Innovation Project (2024A156-G), and Ant Group.

Appendix

Definition

In repository-level code completion, in-file contexts and cross-file contexts refer to different types of contextual information that help a model to predict and complete code snippets:

  1. In-file Context: Following the previous work ([2, 5, 3]), in-file contexts refer to the information contained within the prefix

of the current file itself, specifically the code that precedes the ground truth code in the incomplete code snippet. In-file context provides the immediate surrounding code before the incomplete snippet, which typically helps the model understand the structure, syntax, and logic of the code. This context is crucial for making predictions when code completion can be inferred solely from the current file’s contents, without the need for additional external context.

(2) Cross-file Context: Cross-file Context refers to the information contained in external files within the repository upon which the current file depends. Such information can include functions, classes, or libraries imported into the current file. This type of context is extracted by analyzing dependencies or references to other files within the repository ([13]), providing a broader scope of information for code completion. Leveraging the cross-file context is essential when the in-file context is insufficient for completing the code.

Both types of In-file and Cross-file contexts are essential for repository-level code completion, allowing the model to make more informed predictions and generate accurate completions.

Motivation

In this study, we assume that if LLMs can effectively select between global context and local context, the repository-level code completion performance can be significantly boosted. To further verify our assumption, we conduct a preliminary experiment. In detail, we evaluate the performance of utilizing cross-context versus not using it for code completion in the RepoBench datasets (see details about RepoBench in § 4.1) As shown in Table 5, “In-file” column refers to correct completion using only the context from the current file. While “Both” column denotes correct completions obtained using both in-file and cross-file context. During the process of incorporating cross-files, it introduces context-induced failures (i.e., the red numbers). For example, CodeLlama-7B shows that the number of correct completions increases from 290 to 374 after using cross-file context, but 143 predictions that were originally correct under in-file context turn incorrect when additional cross-file context is included. This reveals a critical challenge: while cross-file context improves overall correctness, it simultaneously destabilizes previously correct predictions. It indicates that LLMs still lack reasoning capabilities for cross-file context and motivates us to explore how to guide LLMs to maximize cross-file benefits while mitigating context-induced failures. The experimental results indicate that while introducing cross-file context improves overall performance, it also harms the model’s original performance in in-file scenarios.

modelJava In-fileJava →\rightarrowJava BothPython In-filePython →\rightarrowPython Both
CodeLlama-7B290-143|+227374134-63|+100174
StarCoder2-7B31-143|+1928055-30|+2954
DeepSeek-Coder-6.7B686-131|+351906390-130|+189449
Qwen2.5-Coder-7B631-128|+301804489-138|+225576
DeepSeek-Coder-33B541-150|+410801373-166|+214421
Qwen2.5-Coder-32B844-99|+3071,052771-126|+237882

Table 5. Results of Exact Match (EM) Count Code Completion.

Justification of Reward Design

Format and length-based rewards.

The format and length-based rewards are auxiliary mechanisms required to make RL training well-defined for line-level code completion in the repository. (1) Format reward: The format constraint is a principled modeling choice in GRPO-style RL training. It defines an evaluation protocol so the completion span can be extracted and scored from the model output (e.g., text and code). Without a fixed output format, the reward cannot be computed consistently during training. (2) Length-based reward: The length-based reward addresses the reward-assignment problem. Our task aims to provide line-level feedback, yet unconstrained generation often produces multiple lines or large segments of a file. As a result, reward triggered by the first correct line propagates through subsequent tokens, causing reward leakage. We observed substantial over-completion in experiments. Therefore, the length reward localizes the reward to the tokens responsible for the prediction. The strictness is intentional: it prevents reward leakage and leads to more stable training and faster convergence. While simple, it effectively ensures that the reward is assigned to the correct position, rather than to unrelated subsequent lines. In summary, these rewards are important and effective mechanisms to ensure reliable evaluation and proper reward assignment when applying RL to line-level code completion.

Identifier-Driven Intent Recognition.

Discussion on Identifier Overlap. Our method evaluates correctness based on the specific context of repo-level code completion rather than relying on shallow lexical similarity. We clarify that our task is repo-level code completion, an IDE-style workflow where a developer completes a single line inside an existing file of a repository. In this setting, the developer is not freely producing alternative implementations. The inserted line must reference concrete program entities available in the current scope and consistent with the surrounding control flow and data flow. The identifier appearing in a valid completion is therefore determined by the required binding, such as which object is in scope, which API provides the required return type, and which function is invoked at that program point, rather than by interchangeable naming choice. Replacing the ground-truth identifier with a different one, even if semantically similar in isolation, usually refers to a different object, breaks the intended call chain, or becomes unusable in the given context. The key issue is whether the completion binds to the intended program entity. For this reason, identifier overlap in the intent reward acts as a proxy for correct contextual binding and practical usability, rather than encouraging shallow lexical similarity.

Motivation Behind Using the Intent Reward as a Coefficient. In code completion, correctly capturing the developer’s intent is the primary objective, so we use the intent reward as a coefficient of the completion reward rather than an additive term. An additive reward treats intent as an independent bonus and may still assign a high total reward to outputs that are textually similar (e.g., small edit distance) but semantically incorrect. By contrast, using intent as a multiplicative coefficient makes it a gating signal: completions that better match the intended functionality receive proportionally higher reward, while superficially similar but semantically mismatched candidates are down-weighted. This is particularly important when two candidates have similar completion scores due to textual similarity, where the intent signal helps distinguish the one whose reasoning path aligns more closely with the expected behavior. From an optimization perspective, the multiplicative form also aligns the gradients of the completion objective with semantic correctness, encouraging the model to prioritize semantically consistent continuations rather than merely minimizing token-level differences.

Prompt

As show in Figure 4, we use this prompt as the LLMs’ input template.

Dataset

Completion Scenarios

We contain three completion scenarios as follows:

Cross-File-First (CF-F): In this setting, the position of completion is the first appearance of a cross-file line within a file and ensuring there is no identical or similar cross-file invocation for the model to reference.

Cross-File-Random (CF-R): In this setting, the position of completion is a random and non-first occurrence of a cross-file line.

In-File (IF): In this setting, the position of completion is an in-file line that does not involve any cross-file modules. This setting serves as a robustness test to ensure that the incorporation of cross-file context does not greatly affect the accuracy of predictions.

Dataset Construction

We collect 32K repository-level code completion examples from the RepoBench-C and RepoBench-P datasets [31], with an equal number of Python and Java examples. The original RepoBench dataset supports Python&Java with three tasks (Retrieval/Completion/Pipeline). Our dataset is derived from Completion(RepoBench-C) and Pipeline(RepoBench-P) as they directly correspond to in-file and cross-file scenarios. Specifically, the context fed into our IntentCoder includes three parts: Pre-target-line crossfile context, Pre-target-line infile context, and Post-target-line crossfile noise. The Pre-target-line crossfile context refers to import modules that appear before the target prediction line. The pre-target-line infile context refers to code content from the beginning of the current file up to the target line. The post-target-line crossfile noise refers to import modules that occur after the target line, which are regarded as noise since we can only use context before the target line for code completion task.

For repository-level line-level code completion, cross-file context is obtained by analyzing the classes, functions, identifiers, and other program entities related to the target completion line. Leverage their dependency relationships to locate relevant files and the corresponding definitions or functional implementations from the repository, providing repository-level context for completing the target line.

For the test dataset, We use Repobench-Java and Repobench-Python [31], each containing 3,000 samples, with 1,000 samples for each of the CF-F, CF-R, and IF settings. Regarding context size, following the RepoBnech, Our dataset is designed for LLMs with a token limit of 2048. The average context size fed into the model is 1346 tokens for training, 1399 tokens for evaluation, and 1121 tokens for testing.

Baselines

We consider the following open-source CodeLLMs for comparison, which are widely used in code completion tasks. To simplify the model names, we have removed the “instruct” suffix in the following introductions (e.g., Qwen2.5-coder, CodeLlama, Deepseek-Coder):

StarCoder2 (Lozhkov et al., 2024): The StarCoder2-15B is a 15 billion parameter model trained on 600+ programming languages using 4.3 trillion tokens, while the StarCoder2-7B is a 7 billion parameter model trained on 17 programming languages with 3.7 trillion tokens.

DeepSeek-Coder [16]: Deepseek-Coder-6.7B is a 6.7 billion parameter model initialized from Deepseek-Coder-6.7b-base and fine-tuned on 2B tokens of instruction data. DeepSeek-Coder-33B is the largest variant of the DeepSeek-Coder series.

CodeLlama (Rozier et al., 2023): CodeLlama-7B and CodeLlama-34B are designed to be safer for use in code assistants and generation applications, with a focus on general code synthesis, understanding, and instruction-following.

Qwen2.5-Coder [19]: Qwen2.5-Coder-7B and Qwen2.5-Coder-33B have performed in code generation, reasoning, and fixing, with the latter scaling up training tokens to 5.5 trillion. Qwen2.5-Coder-33B is the SOTA open-source CodeLLM, offering coding abilities comparable to GPT-4o and providing a more comprehensive foundation for real-world applications.

aiXcoder-7B-v2 [26]: aiXcoder-7B-v2 is currently the SOTA 7B model for repository-level code completion and uses the CoLT method to fine-tune the model to better utilize information within long contexts for code completion tasks.

Then we compare our method with the following methods in the training model for repository-level code completion:

fine-tuning [18]: A supervised fine-tuning (SFT) method where pre-trained models are further trained on a specific task or dataset, refining the model’s performance for the target application. We use Low-Rank Adaptation (LoRA) as the specific method.

GRPO [16]: Group Relative Policy Optimization (GRPO), which is a method of reinforcement learning. We utilizes reinforcement learning by exact match as a reward to improve code prediction accuracy for repository-level code completion.

CoLT [26]: CoLT is a novel training approach for repo-level code completion. It first performs supervised fine-tuning (SFT). After that, preference data is constructed by combining ground truth and model-generated data. Finally, the model is optimized using Direct Preference Optimization (DPO) (Rafailov et al., 2023).

Training Setup. In our experiment, we carefully control the training setup to make the experimental comparison as fair as possible. For the RL method, GRPO and our method use the same training configuration (e.g., the number of epochs, batch size, and rollout size for each prompt). CoLT also uses the same training configuration. For Preference Data Construction, we adopt the same count as the rollout size for each prompt to maintain the same data size. For supervised fine-tuning, we ensure the same overall amount of training data is observed by adjusting the number of epochs, since the RL method samples multiple answers for each prompt. The model is trained until convergence, and we report the best checkpoint according to validation performance.

Implementation Details

The rank and alpha values of LoRA (Hu et al., [18]) are set to 16 and 32, respectively. We use a learning rate of 1×10−41 \times10^{-4} with a linear scheduler and warm-up, with the warm-up ratio set to 0.1. The model was trained for 1 epoch, consisting of 500 steps, with a batch size of 192. The model πθ\pi_\theta is trained on 4 NVIDIA A800 (80GB) GPUs for training and 1 GPU for inference simultaneously, which takes 10 hours. The hyperparameters μ\mu, ϵ\epsilon, and β\beta are set to 1, 0.2, and 1×10−31 \times10^{-3}, respectively. The hyperparameters for reward weights, αic\alpha_{\mathrm{ic}}, αf\alpha_{\mathrm{f}}, and αL\alpha_{\mathrm{L}}, are set to 1.0, 0.2, and 0.1, respectively. We provide well-structured code interface, where reward functions can easily be configured and used. Other Details about the implementation are provided in our released code.

During the inference stage, the backbone models we use (e.g., Qwen2.5-Coder and DeepSeek-6.7B-Coder) support substantially longer context windows (up to 128K/16K tokens). Since RepoBench’s maximum input tokens are 2048, we calculate the runtime latency based on this setting. However, if the cross-file context exceeds 2048 tokens, we can still feed larger contexts to the model and perform reasoning without being constrained by this RepoBench-specific limit.

Evaluation Metrics

Following the previous work (Liu et al., 2023; Eghbali and Pradel, 2024; Zhang et al., 2023; Jiang et al., 2024b}), we report performance in two main categories to evaluate the performance of codeLLMs:

Code Match: To evaluate the accuracy of the predicted code lines, we use Exact Match (EM) and Edit Similarity (ES) (Levenshtein et al., [25]), which respectively represent the proportion of exact matches and the degree of similarity to the ground truth.

Identifier Match ([13, 12]): To evaluate the model’s ability to understand the completion intent and predict the correct application programming interfaces (APIs), we use the following metrics:(1) Exact Match (EM): EM represents the proportion of exact matches to the ground truths in code identifiers. (2) Precision: Precision measures the proportion of correctly predicted code identifiers out of all predicted identifiers. (3) Recall: Recall measures the proportion of correctly predicted code identifiers out of all actual code identifiers in the ground truth. (4) F1: F1 is the harmonic mean of precision and recall metrics, providing a balanced metric for the model’s performance.

Method Analysis and Generalization Experiments

Method Analysis

We clarify why our reward method is more appropriate for repo-level code completion task than other methods as follows:

(1) Execution-based reward can provide a strong reward signal, but in repo-level completion, they are difficult to obtain reliably. Our task involves line-level completion in code repositories, where the suffix code context is often missing, so independent compilation is infeasible. Constructing executable environments at scale requires complex dependencies and specific library versions, and building and testing large projects can take minutes or hours, which is impractical for RL training. Thus, while execution-based rewards are valuable, they are not scalable for large cross-file completion datasets. Our rewards are based on edit similarity and identifier matching, which is a more flexible and scalable design. The reward is lightweight and can be applied efficiently across large datasets, ensuring practical use even when execution or validation is unavailable. In our task, such signals are difficult to obtain reliably: (1) Our task focuses on line-level completion over incomplete files extracted from repositories. These samples typically lack surrounding complete code context or project context (e.g., dependent modules compile) and therefore cannot be compiled independently, making compiler feedback unstable. (2) Lightweight abstract execution could be applied to a subset of cases, which would substantially reduce coverage. and introduce bias toward easily executable examples. For these reasons, we adopt structured static signals (e.g., edit similarity and identifier matching) as a scalable proxy for correctness.

ModelRepoBench-Java Code Match EMRepoBench-Java Code Match ESRepoBench-Java Identifier Match EMRepoBench-Java Identifier Match PrecisionRepoBench-Java Identifier Match RecallRepoBench-Java Identifier Match F1RepoBench-Python Code Match EMRepoBench-Python Code Match ESRepoBench-Python Identifier Match EMRepoBench-Python Identifier Match PrecisionRepoBench-Python Identifier Match RecallRepoBench-Python Identifier Match F1
IntentCoderQwen_{\mathrm{Qwen}}42.4777.5248.5769.9569.6068.7535.9371.5842.7065.5763.8063.37
w/o Identifier-Driven Intent Recognition40.2776.7746.4768.7368.6267.6734.6069.9841.2763.7463.1062.05
w/o Reward-driven Completion Learning40.8076.9447.6769.5668.4968.0334.3068.9740.2062.2460.8760.25
w/o Completion-focused RL40.9076.1347.0768.4967.6266.9534.4767.7340.7062.0659.6159.45
w/o Selective-Exploration Optimization40.8776.7547.0768.9268.4167.6934.4770.4340.9764.5862.9962.39
w/o ALL26.8062.7732.7054.7457.0653.8119.9752.3925.2046.5552.4546.51

Table 7. Results of Ablation study on the RepoBench-java and RepoBench-Python dataset. All results in the table are reported in percentage (%).

(2) SFT reward: Cross-file completion requires reasoning about how an API is used in a concrete program context. A model trained only with retrieval supervision may learn to associate related files rather than learning to generate correct completions via cross-file reasoning, which may bias the model toward lexical similarity rather than deeper semantic correctness. Our method allows the model to learn such reasoning through reinforcement learning by interacting with the repo-level code-completion environment.

(3) Preference-based learning reward: Preference-based learning methods (e.g., DPO) optimize the model using fixed positive–negative pairs. However, in our setting it is difficult for the model to consistently produce fully correct outputs during self-improvement, and relying on ground-truth pairs alone introduces an out-of-distribution gap between training signals and the model’s own generations. Our reward design instead provides graded feedback: multiple candidate completions are compared and rewarded according to how well they align with program behavior. This allows the model to improve even when none of the candidates is perfectly correct, offering a more stable learning signal than strict binary preference pairs and enabling incremental improvement during training.

The experimental results of fine-tuning and CoLT, when compared with our approach, demonstrate that our approach better facilitates the learning of reasoning abilities in the code repository environment. Moreover, the group data generated through model self-distribution can stably and effectively optimize the model’s output distribution, thereby obtaining stable and effective learning signals while avoiding unstable signals from ground truth code completion that fall outside the distribution.

Generalization Experiment.

We provide further analysis of how the model performs when the retrieved cross-file context contains irrelevant or distracting information outside the RepoBench distribution. Specifically, we experimented on the CrossCodeEval-Java (Ding et al., [12]) dataset using different retrievers (BM25 (Robertson and Zaragoza, 2009) and OpenAI text-embedding-ada-002). CrossCodeEval-Java has 2,139 samples and retrieves the top-5 code snippets for each query. This naturally introduces incomplete and potentially distracting context. Table 6 shows the experimental results of our model and other methods. The results show two trends. (1) Under the same retriever, training with our method substantially improves performance (Code EM increases from 10.33% to 15.33%), indicating that the model learns to better identify relevant information within a noisy retrieval cross-files context. (2) When retrieval quality improves (e.g., using a more accurate embedding retriever), performance further increases (Code EM increases from 15.33% to 26.79%), suggesting that the method can effectively utilize cleaner context even in retrieval noise. These observations indicate that IntentCoder is not dependent on a particular repository distribution and remains effective under varying levels of irrelevant or incomplete cross-file context.

ModelBM25OpenAI text-embedding-ada
Code EMCode ESIdentifier EMCode EMCode ESIdentifier EM
Qwen2.5-Coder-7B10.3357.9318.7515.7164.6626.13
Qwen2.5-Coder-7B fine-tuning11.8758.0919.3520.3567.8027.21
Qwen2.5-Coder-7B GRPO13.1860.3620.0622.0268.3329.41
Qwen2.5-Coder-7B CoLT12.9059.7219.5423.1869.7230.67
IntentCoderQwen_{\mathrm{Qwen}}15.3362.4622.3926.7973.4338.62

Table 6. Evaluation results on CrossCodeEval-Java.

Case Study

We illustrate the effectiveness of IntentCoder through a case study presented in Figure 5. In this case, IntentCoder correctly identifies the user’s intent, which is to set a custom renderer for the table corner header (e.g., MultiLineTableHeaderRenderer). In contrast, many other models (e.g., DeepSeek-Coder, Qwen2.5-Coder, CodeLlama-7B, Starcoder2-7B, aixcoder-7B-v2), mistakenly think the next step should be related to the corner setup. Additionally, due to the lack of intent recognition’s capability, many other models overly focus on the in-file context (e.g., colored in blue), such as FrozenColumnTable, setGridColor, getColumn. At the same time, these models lack cross-file reasoning ability and incorrectly invoke classes from other files (e.g., colored in purple), failing to understand that the appropriate multi-line header renderer should be used. By recognizing the overall functional requirements, IntentCoder accurately provides the correct code completion, avoiding the shortcomings of other models in completion intent recognition and cross-file reasoning.

Figure 5. The case study for IntentCodercompares it with other Code LLMs.

References

  1. [1]Lakshya A Agrawal, Aditya Kanade, Navin Goyal, Shuvendu Lahiri, and Sriram Rajamani. 2023. Monitor-guided decoding of code LLMs with static analysis of repository context. Advances in Neural Information Processing Systems, 36:32270–32298.DOI
  2. [2]Miltiadis Allamanis and Charles Sutton. 2013. Mining source code repositories at massive scale using language modeling. In 2013 10th working conference on mining software repositories (MSR), pages 207–216. IEEE.DOI
  3. [3]Ben Athiwaratkun, Sanjay Krishna Gouda, Zijian Wang, Xiaopeng Li, Yuchen Tian, Ming Tan, Wasi Uddin Ahmad, Shiqi Wang, Qing Sun, Mingyue Shang, and 1 others. 2022. Multi-lingual evaluation of code generation models. arXiv preprint arXiv:2210.14868.arxiv.org/abs/2210.14868
  4. [4]Ramakrishna Bairi, Athary Sonwane, Aditya Kanade, Vageesh D C, Arun Iyer, Suresh Parthasarathy, Sriram Rajamani, Balasubramanyan Ashok, and Shashank Shet. 2024. Codeplan: Repository-level coding using llms and planning. Proceedings of the ACM on Software Engineering, 1(FSE):675–698.arxiv.org/abs/2309.12499
  5. [5]Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, and 1 others. 2021. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374.arxiv.org/abs/2107.03374
  6. [6]Wei Cheng, Yuhan Wu, and Wei Hu. 2024. Dataflow-guided retrieval augmentation for repository-level code completion. arXiv preprint arXiv:2405.19782.arxiv.org/abs/2405.19782
  7. [7]Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, and 1 others. 2023. Palm: Scaling language modeling with pathways. Journal of Machine Learning Research, 24(240):1–113.arxiv.org/abs/2204.02311
  8. [8]Fenia Christopoulou, Gerasimos Lampouras, Milan Gritta, Guchun Zhang, Yinping Guo, Zhongqi Li, Qi Zhang, Meng Xiao, Bo Shen, Lin Li, and 1 others. 2022. Pangu-coder: Program synthesis with function-level language modeling. arXiv preprint arXiv:2207.11280.arxiv.org/abs/2207.11280
  9. [9]Zhenlong Dai, Bingrui Chen, Zhuoluo Zhao, Xiu Tang, Sai Wu, Chang Yao, Zhipeng Gao, and Jingyuan Chen. 2025. Less is more: Adaptive program repair with bug localization and preference learning. In Proceedings of the AAAI conference on artificial intelligence, volume 39, pages 128–136.arxiv.org/abs/2503.06510
  10. [10]Zhenlong Dai, Chang Yao, WenKang Han, Yuanying Yuanning, Zhipeng Gao, and Jingyuan Chen. 2024. Mpcoder: Multi-user personalized code generator with explicit and implicit style representation learning. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3765–3780.arxiv.org/abs/2406.17255
  11. [11]Zhenlong Dai, Zhuoluo Zhao, Hengning Wang, Xiu Tang, Sai Wu, Chang Yao, Zhipeng Gao, and Jingyuan Chen. 2026. Learner-tailored program repair: A solution generator with iterative edit-driven retrieval enhancement. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 40, pages 30467–30475.
  12. [12]Yangruibo Ding, Zijian Wang, Wasi Ahmad, Hantian Ding, Ming Tan, Nihal Jain, Murali Krishna Ramanathan, Ramesh Nallapati, Parminder Bhatia, Dan Roth, and 1 others. 2023. Crosscodeval: A diverse and multilingual benchmark for cross-file code completion. Advances in Neural Information Processing Systems, 36:46701–46723.
  13. [13]Yangruibo Ding, Zijian Wang, Wasi Uddin Ahmad, Murali Krishna Ramanathan, Ramesh Nallapati, Parminder Bhatia, Dan Roth, and Bing Xiang. 2022. Cocomic: Code completion by jointly modeling in-file and cross-file context. arXiv preprint arXiv:2212.10007.
  14. [14]Aryaz Eghbali and Michael Pradel. 2024. Dehallucinator: Iterative grounding for llm-based code completion. arXiv preprint arXiv:2401.01701.
  15. [15]Daniel Fried, Armen Aghajanyan, Jessy Lin, Sida Wang, Eric Wallace, Freda Shi, Ruiqi Zhong, Wen-tau Yih, Luke Zettlemoyer, and Mike Lewis. 2022. Incoder: A generative model for code infilling and synthesis. arXiv preprint arXiv:2204.05999.
  16. [16]Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, and 1 others. 2025. Deepseek-rl: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948.
  17. [17]Qi Guo, Xiaohong Li, Xiaofei Xie, Shangqing Liu, Ze Tang, Ruitao Feng, Junjie Wang, Jidong Ge, and Lei Bu. 2024. Ft2ra: A fine-tuning-inspired approach to retrieval-augmented code completion. In Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis, pages 313–324.
  18. [18]Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, and 1 others. 2022. Lora: Low-rank adaptation of large language models. ICLR, 1(2):3.
  19. [19]Binyuan Hui, Jian Yang, Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Lei Zhang, Tianyu Liu, Jiajun Zhang, Bowen Yu, Keming Lu, and 1 others. 2024. Qwen2. 5-coder technical report. arXiv preprint arXiv:2409.12186.arxiv.org/abs/2409.12186
  20. [20]Maliheh Izadi, Roberta Gismondi, and Georgios Gousios. 2022. Codefill: Multi-token code completion by jointly learning from structure and naming sequences. In Proceedings of the 44th international conference on software engineering, pages 401–412.
  21. [21]Juyong Jiang, Fan Wang, JiaSi Shen, Sungju Kim, and Sunghun Kim. 2024a. A survey on large language models for code generation. arXiv preprint arXiv:2406.00515.
  22. [22]Siyuan Jiang, Jia Li, He Zong, Huanyu Liu, Hao Zhu, Shukai Hu, Erlu Li, Jiaz​​heng Ding, Yu Han, Wei Ning, and 1 others. 2024b. aixcoder-7b: A lightweight and effective large language model for code completion. arXiv e-prints, pages arXiv–2410.
  23. [23]Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. 2023. Efficient memory management for large language model serving with pagedattention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles.
  24. [24]Yuhang Lai, Chengxi Li, Yiming Wang, Tianyi Zhang, Ruiqi Zhong, Luke Zettlemoyer, Wen-tau Yih, Daniel Fried, Sida Wang, and Tao Yu. 2023. Ds-1000: A natural and reliable benchmark for data science code generation. In International Conference on Machine Learning, pages 18319–18345. PMLR.
  25. [25]Vladimir I Levenshtein and 1 others. 1966. Binary codes capable of correcting deletions, insertions, and reversals. In Soviet physics doklady, volume 10, pages 707–710. Soviet Union.
  26. [26]Jia Li, Hao Zhu, Huanyu Liu, Xianjie Shi, He Zong, Yihong Dong, Kechi Zhang, Siyuan Jiang, Zhi Jin, and Ge Li. 2025. aixcoder-7b-v2: Training llms to fully utilize the long context in repository-level code completion. arXiv preprint arXiv:2503.15301.
  27. [27]Yujia Li, David Choi, Junyoung Chung, Nate Kushman, Julian Schrittwieser, Rémi Leblond, Tom Eccles, James Keeling, Felix Gimeno, Agustin Dal Lago, and 1 others. 2022. Competition-level code generation with alphacode. Science, 378(6624):1092–1097.DOI
  28. [28]Ming Liang, Xiaoheng Xie, Gehao Zhang, Xunjin Zheng, Peng Di, Hongwei Chen, Chengpeng Wang, Gang Fan, and 1 others. 2024. Repofuse: Repository-level code completion with fused dual context. arXiv preprint arXiv:2402.14323.
  29. [29]Dianshu Liao, Shidong Pan, Qing Huang, Xiaoxue Ren, Zhenchang Xing, Huan Jin, and Qinying Li. 2023. Context-aware code generation framework for code repositories: Local, global, and third-party library awareness. CoRR.
  30. [30]Fang Liu, Zhiyi Fu, Ge Li, Zhi Jin, Hui Liu, Yiyang Hao, and Li Zhang. 2024. Non-autoregressive line-level code completion. ACM Transactions on Software Engineering and Methodology, 33(5):1–34.DOI
  31. [31]Tianyang Liu, Canwen Xu, and Julian McAuley. 2023. Repobench: Benchmarking repository-level code auto-completion systems. arXiv preprint arXiv:2306.03091.

Paper details

vlnc:W_ya5s6mbn
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 14

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents