betaFeedback

WorksComputer Science

Precision–Recall Trade-off on the Inference Level for LLMs in Grammatical Error Correction

Abstract

Large language models (LLMs) have recently been adopted for Grammatical Error Correction (GEC), achieving strong performance via both full fine-tuning and prompt-based approaches. As probabilistic sequence models, LLMs expose inference-time parameters that directly modify token generation behavior; however, their potential for controlling GEC-specific trade-offs has not been systematically studied. In this work, we investigate decoding strategies for grammatical error correction. We show that the repetition penalty parameter enables controllable adjustment of the precision–recall trade-off at inference time, without modifying the model architecture, training procedure, or prompt. Through experiments on standard GEC benchmarks, we demonstrate that repetition-penalty-based decoding enables inference-time control of the precision–recall balance, conceptually related to methods previously proposed for sequence-to-sequence GEC models, such as Align-and-Predict Decoding (APD), while remaining simple and broadly applicable to existing LLMs that expose inference-time decoding penalties.

Introduction

Grammatical Error Correction (GEC) is a field in which systems are expected to identify and correct errors in texts. The task is fairly subjective – it depends on the context in which the system operates, and its goals can differ. For example, a system assisting language learners should provide fewer but more precise corrections to avoid harming or misteaching the learner [10]. In contrast, a GEC system for native speakers should find as many errors as possible, since most of them stem from omissions [18]. The system can thus be optimized for high precision or high recall, balancing the two objectives in a precision–recall trade-off. Table 1 illustrates the

Source textfor example, in math club, teachers usually teach us how to solve a math problem using vary ways.
Precision-oriented GECFor example, in math club, teachers usually teach us how to solve a math problem using various ways.
Recall-oriented GECFor example, in the math club, teachers usually teach us how to solve a math problem using various methods.

Table 1. Example illustrating the difference between precision-oriented and recall-oriented GEC. Precision-oriented GEC is more conservative and tends to make only minimal, high-confidence edits, whereas recall-oriented GEC is less strict and proposes a broader set of changes, including optional stylistic improvements.

differences between precision-oriented and recall-oriented GEC systems.

In the pre-LLM era, the most effective approaches were sequence-to-edit (seq2edit) and sequence-to-sequence (seq2seq) models. The former tag the incorrect text and apply changes according to the tags [11], while the latter directly return the corrected version of the text [6].

LLMs showed decent performance on GEC, but tend to produce overcorrected outputs under standard greedy decoding [12, 4]. Their behavior can be controlled by modifying the decoding strategy via inference parameters such as top-kk, top-pp, temperature, or penalty mechanisms. Such parameters can improve performance, although the decoding strategy must be carefully chosen for the task (Wicher et al., 2022). Unfortunately, this has not been explored for GEC.

To address this gap, we analyze the available inference parameters in the context of GEC to identify those that can benefit LLMs adapted to the task. Our experiments show that the repetition penalty can successfully control the precision–recall trade-off. We analyze how this parameter behaves for both a general instruction-tuned LLM and an LLM fine-tuned for the task, and whether using a prompt with the fine-tuned LLM affects the decoding strategy under the repetition penalty and the overall performance.

Finally, we provide practical guidance on using the repetition penalty parameter across different contexts in which GEC systems can be applied.

Precision–recall Trade-off in GEC

The precision–recall trade-off characterizes how a correction model behaves. Models tuned toward higher precision act conservatively, producing fewer corrections, while recall-oriented models propose more changes and may introduce unnecessary edits.

For seq2edit models, the balance can be adjusted at inference time through a bias toward the KEEP tag and a minimum error probability threshold. For seq2seq models, Sun and Wang [16] propose Align-and-Predict Decoding (APD), which intervenes in decoding based on alignment with the source text, enabling a single model to produce more conservative or more aggressive corrections without retraining.

A separate line of work achieves similar control through post-processing. Sorokin [14] reranks the elementary edits of a first-stage model with a dedicated classifier and applies only those above a threshold; raising it discards unreliable edits and substantially increases precision, while recall stays bounded by the first-stage model’s coverage. Qorib and Ng (2023) enlarge the candidate pool by scoring complete hypotheses built from the edits of several base systems, at the cost of running multiple systems. Similarly, Park et al. [13] propose Post-Correction via Overcorrection (PoCO), which uses smaller models to filter edits from over-correcting LLMs. Such methods offer largely one-sided control and add both a second model and extra latency.

Other work adapts LLMs to GEC directly: Zeng et al. (2024) analyze prompting strategies, while Staruch et al. [15] and Liang et al. [8] investigate training-time adaptation under precision-oriented evaluation. Prompting, however, is unstable and insufficient for mitigating overcorrection, and training-based approaches require substantial compute and access to model weights.

These limitations motivate our work on inference-time control of the precision–recall trade-off in LLM-based GEC. Unlike APD, which requires a custom decoding implementation, and post-correction pipelines, which add processing stages, our approach relies only on inference parameters exposed by standard LLM decoding interfaces, making it broadly compatible with existing deployment frameworks.

Decoding Strategies for LLM-based Grammatical Error Correction

Different tasks require task-specific decoding strategies when using large language models. GEC is a constrained generation task in which the model should introduce necessary corrections while preserving the structure and meaning of the source text. This favors conservative generation behavior and motivates careful tuning of inference-time decoding parameters.

LLM Inference Parameters

The available inference parameters depend on the deployment framework. In this work, we focus on vLLM (Kwon et al., [7]), a widely used open-source library for efficient LLM inference.

Temperature controls the randomness of token generation. Since GEC expects a single high-confidence correction, we set it to zero. Beam search could further improve sequence likelihood, but it significantly increases inference time and is therefore not used. For the same reason, we do not consider stochastic sampling strategies such as top-pp, top-kk, or min-pp: they are intended to promote output diversity, which is inherently unsuitable for error correction, as selecting tokens beyond the most probable ones lacks justification without additional information.

Common token-level penalties include the frequency, presence, and repetition penalties. The frequency penalty grows with the number of times a token has already appeared in the generated output, while the presence penalty penalizes tokens based solely on whether they have appeared at least once. Both apply only to tokens generated during inference and ignore prompt tokens, so they cannot directly discourage copying from the source text in GEC.

In contrast, the repetition penalty also takes into account tokens present in the prompt, allowing it to be applied to all tokens, including those in the original uncorrected text.

Let zt∈Rz_t \in\mathbb{R} denote the logit of token tt and let r>0r > 0 be the repetition penalty. We define an indicator function

I(t)={1if t∈prompt∪current output,0otherwise.\mathbb{I}(t) = \begin{cases} 1 & \text{if } t \in\text{prompt} \cup\text{current output},\\ 0 & \text{otherwise.} \end{cases}

The repetition-penalized logit zt′z'_t is given by

zt′={ztrif zt>0 and I(t)=1,zt⋅rif zt≤0 and I(t)=1,ztif I(t)=0.z'_t = \begin{cases} \frac{z_t}{r} & \text{if } z_t > 0 \text{ and } \mathbb{I}(t) = 1,\\ z_t \cdot r & \text{if } z_t \le0 \text{ and } \mathbb{I}(t) = 1,\\ z_t & \text{if } \mathbb{I}(t) = 0. \end{cases}

Since the penalty modifies logits multiplicatively, values of rr greater than 1 discourage reusing tokens already seen in the prompt or output, whereas values below 1 encourage such repetition (effectively becoming a repetition reward).

In GEC, the penalty adjusts the probabilities of all source-text tokens and thus influences generation. Since introducing changes usually involves adding new tokens, the penalty can steer the model toward a more conservative output or one with more extensive changes.

Its advantage is ease of use: the penalty is set through a single argument in standard inference interfaces – example code is provided in Appendix § 7.

Repetition Penalty Comparison to the APD Method

Applying a repetition penalty during LLM inference is conceptually related to the APD method; Table 2 compares the two. Both use multiplicative transformations to adjust token scores, but in different score spaces: the repetition penalty is applied directly to logits, whereas APD modifies log probabilities.

PropertyAPDRepetition Penalty
Inference-time method✓✓
Score modificationMultiplicativeMultiplicative
Score spaceLog-probabilitiesLogits
Source awarenessExplicit alignmentPrompt token presence
GranularityPosition-awareToken-identity-based
Targets only the aligned source token✓✗
LLM API Compatibility✗✓

Table 2. Comparison of APD method and repetition-penalty-based decoding strategy.

A key distinction is that APD operates on the exact token selected at each decoding step, while the repetition penalty considers all tokens present in the prompt and the current output at every step. The repetition penalty thus offers a simpler alternative to APD, at the cost of adjusting all previously seen tokens rather than only the aligned source token. When the source text contains mostly unique tokens and corrections introduce mainly novel tokens, we hypothesize the practical difference is small; a direct empirical comparison, however, requires a custom APD implementation for autoregressive LLMs and is left for future work.

Experiments

To investigate the impact of the repetition penalty parameter on LLM inference for the GEC task, we conducted experiments on three benchmarks: BEA-test (W&I+LOCNESS) (Bryant et al., [2]) for English, Falko-MERLIN (Boyd, [1]) for German, and UA-GEC (Syvokon et al., [17]) for Ukrainian. For more comprehensive results, we additionally report performance on two common English benchmarks, CoNLL-2014 (Ng et al., [10]) and JF-LEG (Napoles et al., [9]), in Table 8–Table 10 in the Appendix. Dataset statistics are given in Table 7.

BenchmarkLangSplit# examples# references
JFLEGentest7474
CoNLL-2014entest1.3k2
W&I-LOCNESSentrain34.3k1
W&I-LOCNESSentest4.5k5
UA-GECuktrain31.0k1
UA-GECuktest2.7k2
Falko-MERLINdetrain19.2k1
Falko-MERLINdetest2.3k1

Table 7. Statistics on GEC datasets used in our work. # References indicates the number of reference corrections provided per input sentence.

ModelRep. PenaltyCoNLL-2014-test PCoNLL-2014-test RCoNLL-2014-test F0.5_{0.5}BEA-test PBEA-test RBEA-test F0.5_{0.5}JFLEG GLEU
Qwen3-4B-Prompted0.9057.3252.0556.1853.8262.6955.3961.34
0.9556.1353.6455.6152.4763.7454.4061.81
0.9755.4653.7855.1251.6764.1853.7762.04
0.9954.7753.9654.6051.1464.2453.3161.94
1.0054.7554.1454.6350.8564.4153.0961.95
1.0154.2154.4654.2650.5264.6252.8361.83
1.0353.8554.7554.0349.7664.8952.1962.16
1.0553.2055.1253.5848.8264.9651.3762.13
1.1051.2855.7252.1146.4665.2349.2961.90
Qwen3-4B-FT-Prompted0.9069.2538.6559.7971.1451.3766.0658.58
0.9570.6245.7463.6972.5059.6169.4961.03
0.9770.2047.8964.2271.8062.2269.6561.83
0.9968.9649.4563.9270.4864.2769.1562.40
1.0068.0650.2163.5469.5765.0068.6162.53
1.0167.2950.8863.2168.5466.0568.0362.54
1.0365.0553.3562.3265.9567.3266.2262.34
1.0562.6855.1961.0262.2268.0863.3161.94
1.1054.2458.6055.0648.6669.0351.7159.12
Qwen3-4B-FT-NoPrompt0.9067.6941.1959.9772.1755.9168.2059.27
0.9569.5547.1163.5072.2561.3369.7761.60
0.9768.5648.1063.1871.9363.1569.9862.28
0.9968.1050.0063.5070.8464.5969.5062.91
1.0067.7550.9463.5570.1165.4669.1363.02
1.0166.9351.2963.0869.0865.9568.4363.16
1.0365.0552.7162.1467.3266.6667.1963.14
1.0562.0854.0460.2864.1967.5564.8362.84
1.1054.6357.6055.2051.5767.8754.1760.34

Table 8. Effect of the repetition penalty on English GEC performance for Qwen3-4B models. A repetition penalty value of 1 indicates that no penalty is applied.

ModelRep. PenaltyCoNLL-2014-testBEA-testJFLEG
PRF0.5F_{0.5}PRF0.5F_{0.5}GLEU
EuroLLM-22B-Prompted0.9059.7347.2756.7457.4159.0057.7260.29
0.9560.7951.7658.7457.6764.3358.8962.92
0.9760.1953.8958.8155.9865.9557.7263.68
0.9957.8955.9057.4853.1266.9955.4163.92
1.0056.8756.5956.8251.5267.0954.0363.89
1.0155.1757.4255.6149.3967.1852.1663.44
1.0351.9858.5053.1644.5767.6647.8363.12
1.0548.5459.2950.3738.4766.8642.0461.34
1.1039.0760.8342.0823.3263.3826.7050.49
EuroLLM-22B-FT-Prompted0.9066.0832.0154.4871.3442.0062.5953.30
0.9571.7244.3563.8474.2357.7570.2360.67
0.9772.4048.9366.0674.3763.7271.9662.39
0.9971.4453.6466.9974.2868.1372.9663.82
1.0070.4955.3166.8272.9369.7072.2663.88
1.0169.5257.5066.7370.4971.0470.6064.03
1.0364.2359.8463.3064.9572.3866.3163.55
1.0558.7162.2659.3957.3172.7459.8561.20
1.1042.8264.6745.9230.2667.3134.0048.07
EuroLLM-22B-FT-NoPrompt0.9067.4237.9958.3772.0651.8066.8355.42
0.9570.8446.6364.1874.4162.1971.5961.52
0.9771.6249.8265.8674.6865.6972.6962.54
0.9971.1552.8066.5374.4768.5373.2063.49
1.0070.3154.4666.4473.2470.0072.5763.67
1.0169.3056.2766.2371.4871.0371.3964.01
1.0365.2758.5063.7966.4771.6267.4463.68
1.0560.6760.6760.6759.7372.1561.8662.26
1.1044.6863.2247.4634.1067.2137.8351.72

Table 10. Effect of the repetition penalty on English GEC performance for EuroLLM-22B models. A repetition penalty value of 1 indicates that no penalty is applied.

To assess its applicability to different LLMs, we use the following models:

  • Qwen3-4B,1

  • Gemma2-9B,2

  • EuroLLM-22B.3

The model selection criteria were based on general public interest, model size, multilingual capabilities, country of origin and license openness.

To analyze different usage scenarios, we evaluate the models under three configurations:

  • *-Prompted: a model without task-specific fine-tuning, evaluated using a detailed prompt that explicitly describes the GEC task.

Figure 1. Effect of repetition penalty on English (BEA-test) GEC benchmark. A repetition penalty value of 1 indicates no penalty.

  • *-FT-Prompted: a version of the model fine-tuned for GEC and trained with the detailed task prompt format; evaluation is performed using the same detailed prompt.

  • *-FT-NoPrompt: a separate version of the model fine-tuned for GEC without explicit task prompts; during evaluation, only the input text to be corrected is provided in the user part of the chat template.

Comparing *-Prompted with *-FT-Prompted allows us to assess how changes in the repetition penalty affect a general-purpose LLM versus a model fine-tuned for GEC. In turn, the comparison between *-FT-Prompted and *-FT-NoPrompt examines whether tokens originating from the task description prompt influence the behavior of the repetition penalty in the GEC setting. It also allows us to examine whether using a prompt for the GEC task is necessary when the model has been fine-tuned.

All experiments are performed using greedy decoding with a repetition penalty applied. The temperature is set to 0, and beam search is not used, as this configuration is the most practical for real-time LLM deployment. Details of the prompt design and the fine-tuning procedure are provided in the Appendix.

The evaluation for BEA-test is conducted using ERRANT (Bryant et al., [3]) on the CodaBench platform, as the benchmark labels are not publicly available. The Falko-MERLIN and UA-GEC benchmarks were evaluated using ERRANT within the gec-metrics tool (Goto et al., [5]). CoNLL-2014 and JFLEG were evaluated using their official scorers.

We note, however, that our experiments are not intended to tune the model toward precision- or recall-oriented evaluation as measured by the FβF_{\beta}-score for a particular β\beta. Different scenarios and application contexts may require different β\beta values, so we do not optimize specifically for F0.5F_{0.5}, even though it is reported as the primary metric in the evaluated benchmarks. Instead, our goal is to demonstrate that the precision–recall trade-off can be robustly controlled via the repetition penalty

Figure 2. Effect of repetition penalty on German (Falko-MERLIN) and Ukrainian (UA-GEC) GEC benchmarks. A repetition penalty value of 1 indicates no penalty.

ModelBEA-testFalko-MERLINUA-GEC
RPΔF0.5\Delta F_{0.5}ΔP\Delta PRPΔF0.5\Delta F_{0.5}ΔP\Delta PRPΔF0.5\Delta F_{0.5}ΔP\Delta P
Qwen3-4B-Prompted0.90†0.90^{\dagger}+2.30+2.30+2.97+2.970.90†0.90^{\dagger}+1.22+1.22+2.99+2.990.90†0.90^{\dagger}+3.99+3.99+5.49+5.49
Qwen3-4B-FT-Prompted0.970.97+1.04+1.04+2.23+2.230.990.99+0.07+0.07+0.47+0.470.950.95+0.74+0.74+5.70+5.70
Qwen3-4B-FT-NoPrompt0.970.97+0.85+0.85+1.82+1.820.990.99+0.07+0.07+0.40+0.400.990.99+0.14+0.14+1.44+1.44
Gemma2-9B-Prompted0.90†0.90^{\dagger}+2.67+2.67+3.05+3.050.90†0.90^{\dagger}+1.70+1.70+2.83+2.830.90†0.90^{\dagger}+3.43+3.43+4.38+4.38
Gemma2-9B-FT-Prompted0.970.97+0.51+0.51+1.36+1.360.970.97+0.05+0.05+1.03+1.030.970.97+0.59+0.59+3.42+3.42
Gemma2-9B-FT-NoPrompt0.970.97+1.07+1.07+2.27+2.270.990.99+0.70+0.70+1.12+1.120.990.99+0.47+0.47+1.40+1.40
EuroLLM-22B-Prompted0.950.95+4.86+4.86+6.15+6.150.970.97+1.46+1.46+2.15+2.150.90†0.90^{\dagger}+7.20+7.20+11.75+11.75
EuroLLM-22B-FT-Prompted0.990.99+0.70+0.70+1.35+1.350.990.99+0.05+0.05+0.45+0.450.990.99+0.56+0.56+1.62+1.62
EuroLLM-22B-FT-NoPrompt0.990.99+0.63+0.63+1.23+1.230.990.99+0.38+0.38+0.94+0.940.970.97+0.33+0.33+4.02+4.02

caption: Repetition penalty value maximizing F0.5F_{0.5} (RP) and the corresponding gains in F0.5F_{0.5} and precision over no penalty (RP = 1.0) for each model configuration and benchmark. ΔP\Delta P is the precision change at the same RP that maximizes F0.5F_{0.5}, not at the precision-maximizing RP. †\dagger: optimum lies at the boundary of the tested range (0.90); lower values were not evaluated and may perform better.

:::

across different models and benchmarks. In particular, we show that increasing the penalty above 1.0 consistently steers the model toward producing more changes, whereas lowering it below 1.0 results in stricter behavior.

Results

Quantitative Results

Figures 1 and 2 present the precision, recall, and F0.5F_{0.5} curves for different repetition penalty values – exact values are provided in Table 8–Table 13 in the Appendix. A consistent trend holds across all models and benchmarks: penalty values below 1 increase precision while reducing recall, whereas values above 1 lower precision and raise recall, in line with the expected behavior of the penalty. Extreme values, however, can degrade both metrics simultaneously; within 0.97–1.03 this was rare, occurring only for some non–fine-tuned configurations (e.g., Gemma2-9B-Prompted on BEA-test). Metric changes are also smoother for the non–fine-tuned models than for the fine-tuned variants, possibly because fine-tuning alters the magnitude or distribution of logits and thereby amplifies the effect of logit-level penalties.

ModelRepetition PenaltyFalko-MERLIN PFalko-MERLIN RFalko-MERLIN F0.5_{0.5}UA-GEC PUA-GEC RUA-GEC F0.5_{0.5}
EuroLLM-22B-Prompted0.9053.1345.8351.4945.0135.7142.78
0.9552.7152.3152.6340.2541.3240.46
0.9752.3454.5552.7738.1144.5539.25
0.9951.0855.9751.9934.5947.5736.59
1.0050.1956.3451.3133.2649.3735.58
1.0148.9756.6550.3431.6251.0534.23
1.0346.3957.0948.1927.7652.8230.67
1.0542.7156.5644.9124.7155.9227.81
1.1030.8952.3833.6516.5657.8519.31
EuroLLM-22B-FT-Prompted0.9064.9841.3458.3183.6135.0365.45
0.9568.0753.1064.4482.9644.5370.75
0.9768.2755.9765.4081.1648.0171.31
0.9967.7358.7765.7279.8152.0872.12
1.0067.2859.9265.6778.1953.4271.56
1.0166.4460.8065.2375.3355.2370.22
1.0364.3562.7164.0171.0058.1467.99
1.0560.8763.0761.3066.0960.9965.00
1.1049.3362.3951.4848.7266.7551.50
EuroLLM-22B-FT-NoPrompt0.9064.8240.2257.7684.7431.3063.17
0.9567.4551.4463.5083.2941.9169.56
0.9767.5055.0964.5982.4247.7771.98
0.9966.8157.7764.7879.4551.3171.60
1.0065.8759.1464.4078.4053.3171.65
1.0164.7159.6363.6375.8255.4370.62
1.0362.4261.4362.2272.1359.0569.07
1.0559.6962.5560.2466.3662.5565.56
1.1048.0661.7450.2948.3268.1151.30

Table 13. Effect of the repetition penalty on German/Ukrainian GEC performance for EuroLLM-22B models. A repetition penalty value of 1 indicates that no penalty is applied.

The trends of the *-FT-Prompted and *-FT-NoPrompt models are highly similar, suggesting that task-specific instructions in the prompt have limited influence on the penalty’s behavior and that fine-tuned models can operate effectively without explicit instructional prompts, as the task behavior appears to be largely encoded in the model parameters.

Precision-Oriented Analysis

While we do not optimize for any particular FβF_{\beta} score, the benchmarks report F0.5F_{0.5} as their primary metric, so we examine how the penalty affects it. Table 3 reports, for each model configuration and benchmark, the penalty value that maximizes F0.5F_{0.5} and the corresponding gains over standard decoding. In all 27 cases in Table 3 the optimum lies below 1.0, indicating that the penalty’s direction of action is systematically aligned with precision across models, adaptation regimes, and languages.4 The two model groups behave differently: fine-tuned configurations peak in a narrow 0.95–0.99 range, most often at 0.97–0.99, with modest gains (up to +1.07 F0.5F_{0.5}), whereas non–fine-tuned configurations peak at lower values (0.90†–0.97) with substantially larger gains (up to +7.20 for EuroLLM-22B on UA-GEC), consistent with their stronger over-correction; in seven of nine such cases the optimum lies at the boundary of the tested range, so even lower values may be beneficial.

Notably, the F0.5F_{0.5} curves are flat in a wide region around the optimum: the penalty substantially shifts the precision–recall composition of the score, with precision gains of up to +5.70 points for fine-tuned models and up to +11.75 points for non–fine-tuned models at the F0.5F_{0.5}-optimal value, while the aggregate metric changes far less. Precision-oriented adjustment is therefore available at negligible F0.5F_{0.5} cost; conversely, recall-oriented adjustment above 1.0 appears as mild F0.5F_{0.5} degradation only because of the metric’s precision weighting. The width of this flat region varies by model: EuroLLM-22B exhibits a markedly narrower plateau and larger metric swings than the other models in all setups, so the penalty value must be selected more carefully in each individual case.

Impact on Error Types

To examine which error types drive the trade-off, we compare the Gemma2-9B-FT-NoPrompt model (one of the best-performing models) against standard decoding on BEA-test: precision at a repetition penalty of 0.95 and recall at 1.05. We report only error types with at least 20 gold instances (TP+FN≥20TP + FN \ge20), as categories with lower support exhibit high variance.

Table 4 shows the most affected error types. As expected, none gains in both precision and recall simultaneously. The largest precision gains at 0.95 are observed for ADJ, PART, and CONJ (each exceeding 10 percentage points), with notably smaller improvements elsewhere; precision gains are not clearly correlated with recall losses, as some types lose more recall than they gain in precision. Recall gains at 1.05 are smaller overall, led by VERB, NOUN:NUM, and PRON. Several types (ADJ, NOUN, VERB:TENSE, NOUN:NUM, ORTH) appear in both lists, indicating particular sensitivity to the penalty, but no single type is consistently and disproportionately affected. We report this analysis for a single model; whether these patterns generalize across models remains to be verified.

Top Δ\DeltaPrecision (repetition penalty=0.95)Top Δ\DeltaRecall (repetition penalty=1.05)
ErrorΔP\Delta PΔR\Delta RErrorΔP\Delta PΔR\Delta R
ADJ15.34-6.30VERB-10.998.27
PART14.41-10.75NOUN:NUM-9.406.90
CONJ13.33-18.10PRON-8.946.90
NOUN6.32-9.80NOUN:POSS-1.335.81
VERB:TENSE5.33-11.72ADV-11.955.75
NOUN:NUM4.50-8.41NOUN-5.495.04
VERB:SVA4.34-3.53ADJ-9.024.94
ORTH3.69-7.20VERB:TENSE-9.223.97
PREP3.58-10.96ORTH-3.203.69
VERB:FORM3.43-3.87DET-2.631.99

Table 4. Precision–recall differences for the Gemma2-9B-FT-NoPrompt model across error types on the BEA-test set. The left part reports precision and recall changes at repetition penalty 0.95 compared with standard decoding, while the right part reports changes at repetition penalty 1.05. Only the ten most affected error types with at least 20 gold instances are shown.

Qualitative Analysis

Table 5 shows examples of how the repetition penalty affects model outputs for English sentences. The most obvious correction, changing Unforturntly to Unfortunately, is produced consistently across all configurations. In the first example, lowering the penalty causes the model to retain the word almost, compensating by inserting all afterwards to preserve grammaticality; only the highest penalty leads the model to replace in spite of with despite. In the second example, lowering the penalty leaves to uncorrected (instead of with), yielding more conservative edits, whereas the increased penalty produces a fluency-oriented rewrite, replacing criminal life with criminal lifestyle with that of. From a minimal-edit perspective such changes may be considered unnecessary, illustrating how higher penalty values encourage broader paraphrasing and thereby contribute to the observed precision–recall trade-off. The use of hardworking instead of hard-working is most likely due to the presence of this token in the model’s vocabulary.

Source text: Unfortunately, almost older people can not use internet, in spite of benefit of internet.
Standard decoding: Unfortunately, most older people can not use the internet, in spite of the benefits of the internet.
Repetition penalty=0.95: Unfortunately, almost all older people can not use the internet, in spite of the benefits of the internet.
Repetition penalty=1.05: Unfortunately, most older people can not use the internet, despite the benefits of the internet.
Source text: Let’s take time to compare the criminal life to the life of the hard working person.
Standard decoding: Let’s take time to compare the criminal life with the life of the hard-working person.
Repetition penalty=0.9: Let’s take time to compare the criminal life to the life of the hard-working person.
Repetition penalty=1.1: Let’s take time to compare the criminal lifestyle with that of the hardworking person.

Table 5. Examples of repetition penalty parameter impact on English GEC task during inference.

Practical Recommendations

Based on the results analysis, we propose several recommendations for different GEC scenarios. The appropriate repetition penalty value depends primarily on whether the model has been fine-tuned for the GEC task: fine-tuned models exhibit their F0.5F_{0.5} optima in a narrow range close to 1.0, whereas non–fine-tuned models overcorrect more strongly and benefit from lower penalty values.

For systems that favor higher precision, such as language-learning assistance tools or fully automatic systems without human supervision, values below 1 should be used. For fine-tuned models, the analysis in § 5.2 shows that values in the range 0.950.95–0.990.99 coincided with the F0.5_{0.5} optimum across all three main benchmarks, providing precision improvements at negligible F0.5_{0.5} cost. For non-fine-tuned models, stronger penalties of 0.900.90–0.970.97 were optimal – since in most cases the optimum lay at the boundary of the tested range, even lower values may be beneficial, although they require per-case evaluation, as both precision and recall may eventually decrease.

For systems aimed at higher recall, such as book or formal text proofreading by natives where the primary goal is to correct as many errors as possible, values above 1 increase recall at the cost of precision. Unlike the precision-oriented setting, no single value can be recommended: the standard benchmarks define no recall-weighted target metric, and the appropriate weighting of recall against precision is application-dependent. Nevertheless, the range 1.011.01–1.051.05 can be perceived as a safe starting point for increasing recall, as within it recall rises consistently while both metrics rarely degrade simultaneously; beyond 1.051.05, precision and recall may deteriorate jointly, and such values require additional evaluation.

We did not conduct detailed experiments with values lower than 0.900.90 or higher than 1.101.10; however, our initial findings suggest that such extreme values are more likely to degrade system performance and introduce strong biases resulting in unnatural model behavior.

Conclusion

In this work, we analyze inference-time decoding strategies for large language models in the context of grammatical error correction. Our results show that the selection and configuration of inference parameters are task-dependent and can be effectively adapted to meet specific task requirements without modifying model training or prompting.

Building on the prior work on precision–recall trade-off adaptation for GEC systems with different architectures, such as Align-and-Predict Decoding, we demonstrate that the precision–recall trade-off can likewise be controlled at inference time for LLM-based GEC, using only standard decoding parameters. In particular, our experiments show that the repetition penalty parameter provides a simple and effective mechanism for adjusting the precision–recall balance during decoding.

This work opens several directions for future research, including extending the analysis to additional LLMs, languages, and domains, as well as exploring the applicability of inference-time control mechanisms to related text editing tasks such as text simplification.

Limitations

This work explores inference-time decoding strategies for LLM-based GEC; however, the availability of decoding parameters varies between APIs and deployment frameworks. We do not evaluate proprietary models such as GPT-4 or GPT-5, as they currently do not expose a repetition penalty parameter.

The repetition penalty parameter behaves differently across models and benchmarks; therefore, it must be carefully selected for each specific situation and context.

Our experiments focus on three LLMs evaluated under three configurations, which may limit the generalizability of our findings across models of different sizes or architectures.

Finally, we evaluate our approach only on English, German, and Ukrainian GEC benchmarks. Extending the analysis to a larger number of languages would help better assess whether inference-time penalty behavior differs across languages.

Training & Inference Details

We trained our models on a 1–4×A100 (80GB) GPU setup, depending on the model size, and evaluated them on either a 2×RTX 3090 GPU setup or a 1×A100 (80GB) setup. The models were trained on the training portions of the datasets. Training a single model took around one hour. The models were trained separately for each language used in the experiments. The total inference time was approximately 2–3 hours. For inference, we used vLLM in version 0.9.1, however the repetition mechanism is still valid and unchanged in newer vLLM versions. The hyperparameter values are described in Table 6. The following prompt was used during training our models and during inference:

Hyperparameter nameValue
learning rate5e-6
total batch size16
warmup steps50
lr schedulerlinear
epochs1
optimizerAdamW8bit
weight decay0.01

Table 6. Hyperparameter values used to train our models.

You are a professional grammar and language
    correction system.
Your task is to correct all errors in the given text
    .
Do not change the original meaning, tone, or style.
Do not add new information or remove existing
    content unless required for grammatical
    correctness.

Output requirements:

Return only the corrected version of the text.

Do not include explanations, comments, annotations,
    or formatting changes.

Do not restate the original text.

Text to correct:

<text to correct>

Listing 1 shows two options for controlling the repetition penalty value during LLM inference: directly through vLLM’s SamplingParams and through an OpenAI-compatible API endpoint. In both cases, applying the penalty requires only setting a single parameter, without any modification of the decoding algorithm.

# Option 1
sampling_params = SamplingParams(n=1,
        temperature=0, max_tokens=2000,
        repetition_penalty=1.05)
output = model.generate(input_texts,
        sampling_params)

# Option 2
client.chat.completions.create(
        model=model_name,
        messages=messages,
        temperature=0.0,
        max_tokens=2000,
        extra_body={
                "repetition_penalty": 1.05
        }
)

Listing 1: Python code that allows control over repetition penalty values.

ModelRepetition PenaltyCoNLL-2014-test PCoNLL-2014-test RCoNLL-2014-test F0.5_{0.5}BEA-test PBEA-test RBEA-test F0.5_{0.5}JFLEG GLEU
Gemma2-9B-Prompted0.9054.7056.6655.0848.4468.3851.4464.03
0.9553.7157.2054.3747.1468.9950.3364.06
0.9753.3357.7554.1646.5269.2849.7963.95
0.9952.9458.2453.9245.8969.4749.2363.89
1.0052.7558.4053.7945.3969.4248.7763.80
1.0152.8658.8353.9544.9269.2448.3163.73
1.0352.3758.8953.5643.9369.0247.3763.45
1.0551.7258.9153.0242.8668.8446.3663.33
1.1050.1959.7351.8540.3668.3143.9662.63
Gemma2-9B-FT-Prompted0.9072.4446.6165.2175.7562.0972.5660.78
0.9573.2552.6667.9476.4168.0274.5762.79
0.9773.4854.5168.7076.1269.9874.8163.46
0.9972.6756.6868.7975.0171.8874.3664.00
1.0072.5757.5368.9674.7672.5074.3064.19
1.0172.1558.5068.9373.7473.1273.6264.23
1.0369.4360.4667.4371.1673.9771.7063.98
1.0567.1461.5965.9568.1374.2369.2763.55
1.1059.1363.2459.9157.2074.0359.9361.20
Gemma2-9B-FT-NoPrompt0.9072.4243.0563.7376.3957.9071.8059.94
0.9573.7551.2867.8177.3366.6374.9362.59
0.9773.9653.8268.8176.8269.6275.2663.40
0.9973.3556.8169.3175.4472.0274.7364.24
1.0072.6057.6669.0374.5572.8074.1964.32
1.0171.6458.5768.5873.8073.2773.6964.32
1.0369.2960.8467.4170.5874.5371.3364.07
1.0566.2962.6265.5267.2174.8768.6263.57
1.1057.7664.3258.9654.8574.4857.9160.08

Table 9. Effect of the repetition penalty on English GEC performance for Gemma2-9B models. A repetition penalty value of 1 indicates that no penalty is applied.

ModelRepetition PenaltyFalko-MERLIN PFalko-MERLIN RFalko-MERLIN F0.5F_{0.5}UA-GEC PUA-GEC RUA-GEC F0.5F_{0.5}
Qwen3-4B-Prompted0.9050.8342.0248.7835.8138.7136.36
0.9549.7244.7048.6332.8441.3834.26
0.9749.0545.5948.3231.6942.6333.40
0.9948.0646.1747.6730.9043.6932.82
1.0047.8446.4647.5630.3244.3632.37
1.0147.3846.8347.2729.7045.1131.88
1.0346.1046.9946.2828.0646.0630.44
1.0544.8947.1345.3226.1546.6828.67
1.1041.0347.7642.2221.1747.7623.83
Qwen3-4B-FT-Prompted0.9059.7036.3752.9182.0432.4862.86
0.9560.9243.5156.4079.8637.2965.01
0.9760.9146.3257.3077.4038.7464.52
0.9960.4348.4857.5975.4241.3064.73
1.0059.9649.4657.5274.1641.9064.27
1.0159.5450.1857.4072.5243.0763.80
1.0358.2351.7756.8168.5945.4762.26
1.0556.2352.8555.5264.2347.4259.98
1.1045.9453.2447.2450.5552.7450.97
Qwen3-4B-FT-NoPrompt0.9060.0435.5352.7679.8428.8458.98
0.9560.9242.7756.1578.8435.1163.12
0.9760.5645.2056.7177.1237.7263.79
0.9960.1247.4357.0675.6240.9264.65
1.0059.7248.1856.9974.1842.4164.51
1.0159.3049.3256.9973.2443.7464.54
1.0357.7051.0056.2268.4146.0162.34
1.0555.5451.8254.7664.0948.6260.26
1.1046.3952.2147.4548.0053.8349.06

Table 11. Effect of the repetition penalty on German/Ukrainian GEC performance for Qwen3-4B models. A repetition penalty value of 1 indicates that no penalty is applied.

ModelRepetition PenaltyFalco-MERLINUA-GEC
Gemma2-9B-Prompted0.9051.1451.3751.1930.6739.8932.16
Gemma2-9B-Prompted0.9549.9553.0850.5529.0443.1931.08
Gemma2-9B-Prompted0.9749.4353.6850.2227.8843.9930.08
Gemma2-9B-Prompted0.9948.6354.4349.6926.9245.2829.30
Gemma2-9B-Prompted1.0048.3154.8749.4926.2945.6728.73
Gemma2-9B-Prompted1.0147.4754.6448.7525.8746.9728.43
Gemma2-9B-Prompted1.0346.5555.0448.0324.9349.0027.65
Gemma2-9B-Prompted1.0545.3054.9046.9423.8150.0226.59
Gemma2-9B-Prompted1.1042.4354.8344.4420.6351.7423.45
Gemma2-9B-FT-Prompted0.9067.8446.8862.2786.1643.2671.90
Gemma2-9B-FT-Prompted0.9569.5456.4266.4585.8250.5475.31
Gemma2-9B-FT-Prompted0.9769.7759.3667.4085.4053.4176.26
Gemma2-9B-FT-Prompted0.9969.0061.5067.3583.2256.7776.13
Gemma2-9B-FT-Prompted1.0068.7462.3467.3581.9857.8675.67
Gemma2-9B-FT-Prompted1.0167.8963.0666.8780.5159.7775.29
Gemma2-9B-FT-Prompted1.0366.3664.7566.0376.2562.1972.96
Gemma2-9B-FT-Prompted1.0563.9064.8664.0971.4964.9270.07
Gemma2-9B-FT-Prompted1.1055.8864.1657.3656.4769.1758.62
Gemma2-9B-FT-NoPrompt0.9068.7446.8062.8584.9639.1968.87
Gemma2-9B-FT-NoPrompt0.9569.6656.3766.5286.1949.2374.94
Gemma2-9B-FT-NoPrompt0.9769.9859.6167.6384.3252.7075.28
Gemma2-9B-FT-NoPrompt0.9969.4561.9267.8082.7556.8375.83
Gemma2-9B-FT-NoPrompt1.0068.3362.6067.1081.3558.2075.36
Gemma2-9B-FT-NoPrompt1.0167.9663.2566.9680.0259.8774.98
Gemma2-9B-FT-NoPrompt1.0366.4664.8266.1375.5763.1072.70
Gemma2-9B-FT-NoPrompt1.0564.0065.0764.2170.9765.5569.82
Gemma2-9B-FT-NoPrompt1.1055.1064.0556.6855.9569.3358.20

Table 12. Effect of the repetition penalty on German/Ukrainian GEC performance for Gemma2-9B models. A repetition penalty value of 1 indicates that no penalty is applied.

References

  1. [1]Adriane Boyd. 2018. Using wikipedia edits in low-resource grammatical error correction. In Proceedings of the 4th Workshop on Noisy User-generated Text. Association for Computational Linguistics.DOI
  2. [2]Christopher Bryant, Mariano Felice, Øistein E. Andersen, and Ted Briscoe. 2019. The BEA-2019 shared task on grammatical error correction. In Proceedings of the Fourteenth Workshop on Innovative Use of NLP for Building Educational Applications, pages 52–75, Florence, Italy. Association for Computational Linguistics.DOI
  3. [3]Christopher Bryant, Mariano Felice, and Ted Briscoe. 2017. Automatic annotation and evaluation of error types for grammatical error correction. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 793–805, Vancouver, Canada. Association for Computational Linguistics.DOI
  4. [4]Christopher Davis, Andrew Caines, Øistein E. Andersen, Shiva Taslimipoor, Helen Yannakoudakis, Zheng Yuan, Christopher Bryant, Marek Rei, and Paula Buttery. 2024. Prompting open-source and commercial language models for grammatical error correction of English learner text. In Findings of the Association for Computational Linguistics: ACL 2024, pages 11952–11967, Bangkok, Thailand. Association for Computational Linguistics.DOI
  5. [5]Takumi Goto, Yusuke Sakai, and Taro Watanabe. 2025. gec-metrics: A unified library for grammatical error correction evaluation. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations), pages 524–534, Vienna, Austria. Association for Computational Linguistics.
  6. [6]Marcin Junczys-Dowmunt, Roman Grundkiewicz, Shubha Guha, and Kenneth Haefield. 2018. Approaching neural grammatical error correction as a low-resource machine translation task. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 595–606, New Orleans, Louisiana. Association for Computational Linguistics.
  7. [7]Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica. 2023. Efficient memory management for large language model serving with pagedattention. In Proceedings of the 29th Symposium on Operating Systems Principles, SOSP ’23, page 611–626, New York, NY, USA. Association for Computing Machinery.arxiv.org/abs/2309.06180
  8. [8]Jiehao Liang, Haihui Yang, Shiping Gao, and Xiao-jun Quan. 2025. Edit-wise preference optimization for grammatical error correction. In Proceedings of the 31st International Conference on Computational Linguistics, pages 3401–3414, Abu Dhabi, UAE. Association for Computational Linguistics.
  9. [9]Courtney Napoles, Keisuke Sakaguchi, and Joel Tetreault. 2017. JFLEG: A fluency corpus and benchmark for grammatical error correction. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers, pages 229–234, Valencia, Spain. Association for Computational Linguistics.
  10. [10]Hwee Tou Ng, Siew Mei Wu, Ted Briscoe, Christian Hadiwinoto, Raymond Hendy Susanto, and Christopher Bryant. 2014. The CoNLL-2014 shared task on grammatical error correction. In Proceedings of the Eighteenth Conference on Computational Natural Language Learning: Shared Task, pages 1–14, Baltimore, Maryland. Association for Computational Linguistics.DOI
  11. [11]Konstantin Omelianchuk, Vitaliy Atrasevych, Artem Chernodub, and Oleksandr Skurzhanskyi. 2020. GECToR – grammatical error correction: Tag, not rewrite. In Proceedings of the Fifteenth Workshop on Innovative Use of NLP for Building Educational Applications, pages 163–170, Seattle, WA, USA → Online. Association for Computational Linguistics.arxiv.org/abs/2005.12592
  12. [12]Konstantyn Omelianchuk, Andrii Liubonko, Oleksandr Skurzhanskyi, Artem Chernodub, Oleksandr Kornienko, and Igor Samokhin. 2024. Pillars of grammatical error correction: Comprehensive inspection of contemporary approaches in the era of large language models. In Proceedings of the 19th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2024), pages 17–33, Mexico City, Mexico. Association for Computational Linguistics.
  13. [13]Taehee Park, Heejin Do, and Gary Lee. 2025. Leveraging what’s overfixed: Post-correction via LLM grammatical error overcorrection. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 28183–28195, Suzhou, China. Association for Computational Linguistics. Muhammad Reza Qorib and Hwee Tou Ng. 2023. System combination via quality estimation for grammatical error correction. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 12746–12759, Singapore. Association for Computational Linguistics.
  14. [14]Alexey Sorokin. 2022. Improved grammatical error correction by ranking elementary edits. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 11416–11429, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
  15. [15]Ryszard Staruch, Filip Gralinski, and Daniel Dzienisiewicz. 2025. Adapting LLMs for minimal-edit grammatical error correction. In Proceedings of the 20th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2025), pages 118–128, Vienna, Austria. Association for Computational Linguistics.
  16. [16]Xin Sun and Houfeng Wang. 2022. Adjusting the precision-recall trade-off with align-and-predict decoding for grammatical error correction. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 686–693, Dublin, Ireland. Association for Computational Linguistics.DOI
  17. [17]Oleksiy Syvokon, Olena Nahorna, Pavlo Kuchmiichuk, and Nastassia Osidach. 2023. UA-GEC: Grammatical error correction and fluency corpus for the Ukrainian language. In Proceedings of the Second Ukrainian Natural Language Processing Workshop (UNLP), pages 96–102, Dubrovnik, Croatia. Association for Computational Linguistics.
  18. [18]Baoxin Wang, Xingyi Duan, Dayong Wu, Wanxiang Che, Zhigang Chen, and Guoping Hu. 2022. CCTC: A cross-sentence Chinese text correction dataset for native speakers. In Proceedings of the 29th International Conference on Computational Linguistics, pages 3331–3341, Gyeongju, Republic of Korea. International Committee on Computational Linguistics.DOI

Paper details

vlnc:W_9zd547eq
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents