WorksComputer Science
Accept or Repair: Preserving Passage-Supported Answers in Contrastive Decoding
Abstract
Large language models often answer from knowledge learned during training, even when a provided passage contradicts that knowledge. To address this problem, contrastive decoding contrasts the predictions under the full and a modified prompt at every token, but it also alters correct answers that the passage supports. We show that checking answers against the passage removes this trade-off. Based on this observation, we propose Accept-or-Repair (AoR), a training-free procedure. AoR keeps a sampled answer if the passage supports it. Otherwise, it tries up to two replacements with contrastive decoding, returning one as soon as it passes this check. It rejects replacements that repeat the answer generated without the passage when the passage favors another answer. Our experiments show that AoR raises the score over sampling by 12.5 points on counterfactual passages and 2.3 on ordinary ones, surpassing eight contrastive and prompting baselines on both, including when each receives the same rejection rule. Replacing only initial answers that the passage does not support improves the average score of every decoder. These results suggest that models can follow passages that contradict their knowledge while preserving accuracy on passages that agree with it. Code will be released at https://github.com/js-lee-AI/AoR.
Figure 1. Gain over plain sampling on counterfactual and ordinary passages, at the setting that each method chose on the development split. Test split, averaged over six models.
Introduction
Large language models (LLMs) often prioritize knowledge learned during training over the information in a provided passage, even when the two conflict (Longpre et al., 2021; Xie et al., 2024). However, models should base their answers on the passage, whether it provides updated or private information or conflicts with their learned knowledge (Ram et al., 2023; Zyskind et al., 2024; Shi et al., 2024; Lee et al., 2025).
To address this problem, contrastive decoding encourages models to follow the passage without additional training (Shi et al., 2024; Wang et al., 2025a). At each decoding step, it contrasts token predictions under the full and a modified prompt to favor information from the passage. Context-aware decoding (CAD), for instance, removes the passage from the contrasting prompt (Shi et al., 2024). Methods also use signals from the model to adjust the contrast strength or decide when to apply it (Wang et al., 2025a; Yuan et al., 2024; Tao and Agrawal, 2026). These methods control answer generation without first checking whether a plain answer already follows the passage. On ordinary passages, which agree with the model’s parametric knowledge, CAD turns correct answers into errors more often than plain resampling. The baselines that gain most on counterfactual passages reduce scores on ordinary passages, whereas the others leave ordinary scores nearly unchanged, as Figure 1 shows.
For extractive questions, checking whether an answer occurs in the passage provides a cue for deciding whether to keep it. On ordinary passages, most plain answers occur in the passage, and these answers are more accurate than those that do not. We find that contrastive decoders lower the scores of these answers, with larger F1 losses for CAD when the model assigns the answer a higher probability without the passage.
Based on this observation, we propose Accept-or-Repair (AoR), a training-free procedure that checks an initial answer before deciding whether to repair it. AoR samples a plain answer and accepts it when the passage supports it. For an unsupported initial answer, AoR draws an answer without the passage and uses it to screen replacements. The memory veto of AoR rejects replacements that repeat this answer when the passage favors another answer. AoR tries up to two replacements with contrastive decoding at the baselines’ selected strengths and returns the first that passes both the support check and the veto. Figure 2 illustrates these decisions on two test questions. CAD changes the correct answer 111 to two, whereas AoR keeps 111 and replaces the memorized 1922 with 1971 from the passage.
Figure 2. Two test questions. (a) CAD contrasts every answer. (b) AoR accepts supported answers (Supp). When the passage favors another answer, the memory veto rejects repairs that repeat , generated without the passage. Gray steps are skipped.
Our experiments show that AoR improves average scores over plain sampling on both counterfactual and ordinary passages and outperforms contrastive and prompting baselines. The lead remains when baselines receive the same memory veto, and LLM judges also rank AoR highest on average.
Replacing only unsupported plain answers improves the average score of each baseline. The support test selects answers to repair more effectively than using the model’s confidence, asking it to check its own answer or applying an entailment model. These alternatives often accept answers from the model’s memory even when the passage states a different answer.
We make three contributions.
We show that contrastive decoders improve scores on unsupported initial answers but reduce scores on supported answers in ordinary passages, identifying a source of the trade-off between the two passage types.
We propose Accept-or-Repair, a training-free procedure that keeps supported answers and tries up to two contrastive repairs for the others, subject to the memory veto.
We show that AoR retains its lead on held-out benchmarks and across model families and sizes, and improves scores on both passage types when answers are given as one sentence.
Related Work
Knowledge conflicts. Models can favor memorized knowledge over conflicting passages and be misled by irrelevant context (Longpre et al., 2021; Xie et al., 2024; Shi et al., 2023; Xu et al., 2024; Lee et al., 2026e). FaithEval and ConFiQA evaluate whether models bait their answers on counterfactual passages ([4] (Ming et al. 2025)). Prompting improves faithfulness to the passage through opinion-based questions and counterfactual demonstrations (Zhou et al., 2023).
Decoding for context faithfulness. Contrastive decoding adjusts token scores by comparing predictions from the original input with those obtained without the passage, with an irrelevant passage, or with a modified instruction (van der Poel et al., 2022; Shi et al., 2024; Zhao et al., 2024; Kim et al., 2024a). Methods control contrastive decoding using distribution divergence, entropy, confidence signals, or a tunable factor ([13] (Wang et al. 2025a; Yuan et al. 2024; Qiu et al. 2025; Khandelwal et al. 2025; Kim et al. 2024b; Li et al. 2026)). Gating and routing determine when to apply the contrast or which decoding regime to use (Tao and Agrawal, 2026; Jiang et al., 2026; Zhou et al., 2026; Lee et al., 2026c; Kang et al., 2026; Lee et al., 2026d). Other methods encourage models to follow the passage by adjusting token probabilities or steering hidden representations ([11] (Zhang et al. 2026; Zhao et al. 2025; Minder et al. 2025; Wang et al. 2025b)). AoR combines answer selection with contrastive decoding to keep supported initial answers and repair unsupported ones.
Answer selection and revision. Methods select among sampled answers by frequency or a trained verifier’s score ([7 f] (Wang et al. 2023; Lee et al. 2026b)), and model cascades use learned estimates of answer quality to decide whether to query another model [5]. Verification methods use self-critique, retrieved evidence, or attention signals to select or revise generated text [2, 10, 8, 6] (Lee et al. 2026a; Lee and Park 2026). AoR selects initial answers for repair using a lexical test against the passage, without a trained verifier, self-verification prompts, or access to attention weights.
Accept-or-Repair
Setting. A model generates a short answer from a prompt containing a passage and a question . We write for an answer generated from this prompt using decoder . AoR keeps the initial answer from plain sampling if the passage supports it. Otherwise, it generates replacements with contrastive decoders and checks them against the passage, as shown in Algorithm 1.
Support test. For short free-form answers, the support test accepts when its normalized form is nonempty and occurs in the normalized passage as a contiguous sequence of whole tokens. Normalization lowercases the text, replaces ASCII punctuation with spaces, preserves non-ASCII punctuation and symbols as separate tokens, and removes articles. For free-form questions, an answer that normalizes to yes or no passes the support test even if it does not occur in the passage. For multiple-choice questions, the answer is an option letter. It passes the support test when the selected option’s text occurs in the passage and no other option’s text does. For sentence answers, we exclude words that occur in the question and function words such as articles and prepositions. The answer passes the test when at least one word remains and all remaining words occur in the passage.

Figure 1 (PDF p. 3).
Algorithm 1 Accept-or-Repair
Require: passage , question , repair decoders
plain sampling
if then return ▷ accept
plain sampling without the passage
if the passage favors another answer over , else
for do ▷ repair
, with the letter of masked if on a multiple-choice question
if and then return
return the first that states an answer, else
Accept. AoR first draws a plain answer and returns it unchanged when holds, without applying contrastive decoding.
Repair. If the initial answer fails the support test, AoR draws candidate answers from in order. It returns the first candidate that passes the test and is not rejected by the memory veto. If no candidate passes both checks, AoR returns the first that states an answer and is not rejected by the veto, or the initial answer if no such candidate exists. A candidate states an answer when its normalized text is nonempty for a free-form question or it gives a valid option letter for a multiple-choice question. We use , with CAD (Shi et al., 2024) followed by Instructive Decoding (ID) (Kim et al., 2024a), and select the decoders and their order on the development split. For ID, the contrasting prompt instructs the model to ignore the passage. We write AoR() for the variant that uses as its only repair without the memory veto.
Memory veto. For an unsupported initial answer, AoR draws a closed-book answer using plain sampling without the passage to screen repairs. For free-form questions, if fails the support test, the veto rejects repairs that match after normalization. Such repairs already fail the support test, so the veto prevents them from being returned as fallbacks. For multiple-choice questions, an option scores 2 if its text occurs in the passage. Otherwise, we exclude function words and use the fraction of the option’s remaining words that occur in the passage. If another option scores higher than the option selected by , the veto masks the letter of during repair decoding. For example, if is A and only option C’s text occurs in the passage, the veto masks A and leaves B, C and D unmasked. An option whose text occurs in the passage receives the maximum score, so the veto does not mask it. On a paraphrased passage, the veto leaves unmasked when no other option scores higher.
Supported and unsupported answers. We analyze how retaining supported answers changes a decoder’s score. Let be the event that the initial plain answer passes the support test, and let and denote the scores in of and a draw from decoder . The expected score difference between and plain sampling is
where and is the same difference on unsupported initial answers. By retaining whenever holds, AoR preserves the score on supported answers and can change a correct initial answer only if that answer fails the support test. For AoR(), which uses one repair without the memory veto, the output on unsupported initial answers is the draw from , or if that draw states no answer. With zero score for a draw that states no answer, the expected gain over is
where averages on these fallback cases and zero on all other cases. (2) implies that AoR() has a higher expected score than when lowers the average score on supported initial answers.
Experimental Setup
Tasks. We evaluate whether AoR follows counterfactual passages while preserving accuracy on ordinary passages. In the main comparison, NQ-Swap (Longpre et al., 2021) replaces the answer entity in a Natural Questions passage (Kwiatkowski et al., 2019) with another entity of the same type, and FaithEval’s counterfactual task (Ming et al., 2025) asks multiple-choice questions whose passage supports a counterfactual answer. The ordinary tasks are NQ-Original, which uses the original passages and answers from NQ-Swap, SQuAD (Rajpurkar et al., 2016) and HotpotQA (Yang et al., 2018) in the distractor setting. For HotpotQA, we concatenate two supporting paragraphs with up to eight distractors. We also evaluate on the counterfactual subsets of ConFiQA (Bi et al., [4]). For ordinary passages, we use ConFiQA’s original passages and answers, TriviaQA (Joshi et al., (2017); Fisch et al. [9]) and SciQ (Welbl et al., 2017).
Models. The main comparison uses Qwen3 models with 1.7B, 4B, 8B and 14B parameters (Qwen Team, 2025), Llama-3.1-8B-Instruct (Llama Team, 2024) and Mistral-7B-Instruct-v0.3 (Jiang et al., 2023; Mistral AI, 2024). We also evaluate Qwen3-32B, Llama-3.1-70B-Instruct, OLMo-2-13B-Instruct (Team OLMo et al., 2024) and Yi-1.5-9B-Chat (Young et al., 2024; 01.AI, 2024) to examine performance across model sizes and families. All models are instruction-tuned, and Qwen3 runs in non-thinking mode. For each model, the methods share the temperature and top- sampling settings (Holtzman et al., 2020) listed in Table 10, with top- filtering enabled for Qwen3. Temperature scaling and filtering follow any contrastive adjustment of token scores. We share sampling noise across methods within each run and average the scores over three seeds.
| Model | Temperature | Top- | Top- |
| Qwen3 (all sizes) | 0.7 | 0.8 | 20 |
| Llama-3.1 (8B, 70B) | 0.6 | 0.9 | off |
| Mistral-7B | 0.7 | 0.9 | off |
| OLMo-2-13B | 0.7 | 0.9 | off |
| Yi-1.5-9B | 0.7 | 0.9 | off |
Table 10. Sampling settings of every model, shared by all methods.
Baselines. CAD, AdaCAD, COIECD and No-Worse CAD (NWCAD) use the question alone as the contrastive input (Shi et al., 2024; Wang et al., 2025a; Yuan et al., 2024; Tao and Agrawal, 2026). Random-passage CD (RPCD) contrasts with another example’s passage (Zhao et al., 2024), and ID with an instruction to ignore the passage (Kim et al., 2024a). To compare their contrastive inputs, we use the CAD scoring rule for both baselines. Strict prompt asks the model to answer only from the passage (Zhou et al., 2023), and Strict CAD applies CAD to that prompt. Contrastive settings are selected on the development split and kept fixed across models and tasks. We choose the setting with the highest average gain over sampling across the two task groups, retaining a recommended setting when it is within 0.1 points of the best. Strict CAD uses CAD’s selected strength, and AoR uses the selected CAD and ID strengths for its repairs.
Evaluation. We use token-level F1 for free-form answers and accuracy for multiple-choice answers. For the main comparison, we average task scores within the counterfactual and ordinary groups and report Mean as the average of the two group scores. We assess score differences with 95% confidence intervals from paired bootstrap resampling ([3]). Cost is the number of tokens processed across decoding calls, including prompts, relative to plain sampling. Appendices A through D give the prompts, settings and scoring procedures.
Results
Main results. We find that AoR is the only method to improve significantly over sampling in both task groups, by 12.5 points on counterfactual passages and 2.3 on ordinary passages, as Table 1 shows. It scores highest on each task and significantly outperforms the baselines in Mean. Strict CAD, CAD and COIECD make the largest counterfactual gains among the baselines but lose on ordinary passages, whereas the other baselines leave ordinary scores nearly unchanged and gain less on counterfactual passages. The lead in Mean holds on each model, as reported in Appendix H. AoR has a token cost of 1.71 times that of sampling, lower than RPCD and ID.
| Method | NQ-Swap | FaithEval | Avg. | NQ-Orig. | SQuAD | HotpotQA | Avg. | Mean | Cost |
| Sampling | 56.9 | 63.6 | 60.3 | 76.0 | 84.5 | 63.5 | 74.7 | 67.5 | 1.00 |
| CAD ((Shi et al., 2024)) | 62.4 | 73.7 | 68.1 (+7.8) | 72.5 | 84.6 | 60.8 | 72.6 (−2.1) | 70.3 | 1.18 |
| AdaCAD ((Wang et al., 2025a)) | 61.4 | 65.5 | 63.4 (+3.2) | 75.6 | 85.5 | 63.2 | 74.8 (+0.1) | 69.1 | 1.18 |
| COIECD ((Yuan et al., 2024)) | 62.2 | 73.7 | 68.0 (+7.7) | 72.7 | 84.5 | 61.1 | 72.8 (−1.9) | 70.4 | 1.18 |
| NWCAD ((Tao and Agrawal, 2026)) | 57.3 | 63.8 | 60.6 (+0.3) | 76.0 | 84.5 | 63.7 | 74.8 (+0.1) | 67.7 | 1.18 |
| RPCD ((Zhao et al., 2024)) | 61.0 | 71.1 | 66.1 (+5.8) | 75.4 | 85.3 | 62.5 | 74.4 (−0.3) | 70.2 | 2.00 |
| ID ((Kim et al., 2024a)) | 64.3 | 67.8 | 66.0 (+5.8) | 74.8 | 85.6 | 63.6 | 74.7 (0.0) | 70.4 | 2.01 |
| Strict prompt ((Zhou et al., 2023)) | 60.3 | 64.8 | 62.5 (+2.2) | 76.0 | 84.9 | 63.1 | 74.6 (0.0) | 68.6 | 1.03 |
| Strict CAD ((Zhou et al., 2023; Shi et al., 2024)) | 64.7 | 75.1 | 69.9 (+9.6) | 70.9 | 84.1 | 58.3 | 71.1 (−3.6) | 70.5 | 1.21 |
| AoR (ours) | 65.2 | 80.4 | 72.8 (+12.5) | 78.2 | 87.5 | 65.2 | 77.0 (+2.3) | 74.9 | 1.71 |
Table 1. Test scores averaged over six models: F1 for free-form tasks, accuracy for FaithEval. Mean averages both groups. Parentheses show differences from sampling, colored when their 95% intervals exclude zero. Cost is the token count relative to sampling.
Where the gains come from. We find that the baselines improve on unsupported initial answers in both groups but lose on supported answers in ordinary passages, as Table 2 shows. Figure 4 shows larger F1 losses for Strict CAD, CAD and COIECD in bins where the model assigns higher probability to the initial answer without the passage. By retaining supported answers, AoR avoids this loss term in Equation (1). It turns 4 of the 24,639 correct initial answers on ordinary passages into errors and corrects more wrong initial answers on counterfactual passages than the baselines. These findings suggest that retaining supported answers helps turn contrastive repairs into gains on ordinary passages.
| Method | Counterfactual | Counterfactual | Fixed (%) | Ordinary | Ordinary | Broken (%) | Lowered (%) | Repaired part CF | Repaired part Ord. |
| Sampling | – | – | – | – | – | – | – | 37.2 | 35.5 |
| CAD | −0.2 | +8.0 | 22.0 | −3.4 | +1.3 | 12.2 | 12.9 | 52.7 | 43.7 |
| AdaCAD | +0.3 | +2.9 | 9.4 | −0.9 | +0.9 | 3.9 | 5.3 | 44.7 | 41.1 |
| COIECD | −0.2 | +7.9 | 21.6 | −3.3 | +1.4 | 11.9 | 12.6 | 52.5 | 43.9 |
| NWCAD | −0.1 | +0.4 | 1.2 | −0.1 | +0.2 | 0.9 | 1.4 | 38.2 | 36.6 |
| RPCD | −0.7 | +6.5 | 17.6 | −2.1 | +1.8 | 9.7 | 10.1 | 50.6 | 46.3 |
| ID | 0.0 | +5.8 | 19.2 | −2.1 | +2.1 | 8.7 | 9.4 | 51.6 | 48.0 |
| Strict prompt | −0.2 | +2.5 | 9.3 | −0.9 | +0.8 | 3.7 | 4.9 | 43.9 | 40.4 |
| Strict CAD | 0.0 | +9.6 | 28.0 | −5.2 | +1.6 | 15.8 | 15.9 | 56.5 | 45.1 |
| AoR (ours) | 0.0 | +12.5 | 31.2 | 0.0 | +2.3 | 0.0 | 3.3 | 60.5 | 49.5 |
Table 2. Where the gains come from. , : share-weighted difference to sampling on supported and unsupported plain answers; Fixed, Broken: wrong plain answers made right, correct ones made wrong; Lowered: F1 below the plain answer; Repaired part: scores where AoR repairs.
Figure 4. F1 change on the supported plain answers of the ordinary tasks, binned by the probability that the model gives the same answer without the passage.
Wrapping other decoders. We find that replacing only unsupported initial answers significantly improves the Mean of each baseline, as Table 3 shows. Each baseline serves as AoR’s only repair in this comparison, without the memory veto. On ordinary passages, the gains recover the losses on supported answers in Table 2, as Equation (2) predicts. Counterfactual scores remain nearly unchanged.
| Wrapped decoder | CF | Ord. | Mean |
| CAD | 68.3 | 76.0 (+3.4) | 72.2 (+1.8) |
| AdaCAD | 63.2 | 75.6 (+0.9) | 69.4 (+0.3) |
| COIECD | 68.2 | 76.1 (+3.3) | 72.1 (+1.7) |
| NWCAD | 60.7 | 74.9 (+0.1) | 67.8 (+0.1) |
| RPCD | 66.8 | 76.5 (+2.1) | 71.6 (+1.4) |
| ID | 66.2 | 76.7 (+2.1) | 71.5 (+1.1) |
| Strict prompt | 63.0 | 75.5 (+0.9) | 69.3 (+0.7) |
| Strict CAD | 70.1 | 76.3 (+5.2) | 73.2 (+2.7) |
Table 3. Each decoder wrapped by AoR as its only repair, without the veto. On ordinary passages and in Mean, parentheses give the difference to alone, colored when its 95% interval excludes zero.
We also compare four contrast strengths for CAD, ID and RPCD to test whether this benefit depends on the selected setting. Without AoR, all three decoders score below sampling on ordinary passages at the strongest setting, as Figure 5 shows. With AoR, counterfactual scores rise with contrast strength while ordinary scores stay above sampling, giving each decoder its highest Mean at the strongest setting. These findings suggest that retaining supported answers allows stronger contrastive repairs while preserving performance on
Figure 5. Counterfactual and ordinary averages of CAD, ID and RPCD at four strengths, alone and wrapped by AoR as the only repair. Dotted lines mark sampling.
Results
Ablations. Keeping supported initial answers improves ordinary scores and reduces token cost, as Table 4 shows. Allowing partial matches between answers and passage spans lowers the Mean and admits more memorized answers on NQ-Swap. Removing the memory veto lowers FaithEval accuracy, and removing either repair lowers scores on both passage types.
| Variant | CF | Ord. | Mean | Cost |
| AoR | 72.8 | 77.0 | 74.9 | 1.71 |
| Components | ||||
| Repair every answer | +0.7 | −1.3 | −0.3 | 2.71 |
| No memory veto | −3.2 | 0.0 | −1.6 | 1.63 |
| Any occurring option supported | −0.9 | 0.0 | −0.5 | 1.65 |
| Repairs | ||||
| CAD only | −1.6 | −1.0 | −1.3 | 1.46 |
| ID only | −2.4 | −0.3 | −1.3 | 1.73 |
| ID, then CAD | −1.2 | +0.1 | −0.6 | 1.86 |
| CAD, then Strict CAD | −0.4 | −0.5 | −0.4 | 1.61 |
| CAD, ID, then CAD | +0.2 | +0.1 | +0.1 | 1.81 |
| CAD, ID, then span Strict CAD | +1.4 | +0.9 | +1.2 | 1.81 |
| Repairs without contrast | ||||
| Resample | −6.1 | −1.6 | −3.9 | 1.56 |
| Resample, strict prompt | −4.9 | −1.2 | −3.0 | 1.56 |
| Majority votes | ||||
| Three plain answers | −12.4 | −2.1 | −7.2 | 3.00 |
| Plain, CAD and ID | −7.8 | −1.6 | −4.7 | 4.19 |
Table 4. Variants of AoR and majority votes. Score differences are relative to AoR; color indicates a 95% interval excluding zero. Cost is the token count relative to sampling.
Replacing contrastive repairs with samples from the strict or plain prompt lowers AoR’s Mean by 3.0 to 3.9 points, with larger losses on counterfactual passages. The strict-prompt variant still out-performs the baselines in Mean. Majority voting ((Wang et al., 2023)) yields lower scores than AoR on both passage types and uses more tokens. These comparisons suggest that AoR benefits from both checking answers against the passage and using contrastive decoding to generate repairs.
Adding Strict CAD restricted to passage spans as a third repair raises the Mean at a higher token cost. The lead of AoR over the baselines in Mean also holds across the 16 combinations of CAD and ID strengths, as Appendix § 14 shows.
Choosing what to repair. The support test gives the largest gains on NQ-Swap and ordinary passages among the selection criteria in Table 5. We compare it with answer probability, the probability gain from adding the passage, a model self-check (Kadavath et al., 2022) and an entailment score. To isolate the selection criterion, each alternative
| Gate | NQ-Swap | Ord. | Mean |
| CAD on every answer | +5.5 | −2.1 | +1.7 |
| Same as closed-book answer | +1.5 | −0.8 | +0.3 |
| Answer probability | +4.2 | +0.2 | +2.2 |
| Probability gain from passage | +4.8 | −0.8 | +2.0 |
| Self-check | +3.0 | −0.3 | +1.4 |
| Entailment model | +3.7 | −0.2 | +1.7 |
| Support test | +5.9 | +1.3 | +3.6 |
Table 5. Gain over sampling when CAD replaces selected answers on free-form tasks. Four ranked gates match the support test’s selection count. Mean averages NQ-Swap and Ord. Color indicates 95% intervals excluding zero.
replaces the same number of initial answers as the support test, using the same CAD outputs without further checks.
On NQ-Swap, the support test selects 99.5% of initial answers that repeat the answer from the original passage, compared with 54–55% for answer probability, self-check and entailment. This difference suggests that passage support helps identify memorized answers that these three criteria miss.
Analysis
Beyond the main setting. AoR outperforms the baselines in average score across the comparisons in Table 6. The lead remains when we exclude FaithEval and when the baselines receive the same memory veto, indicating that neither the multiple-choice task nor the veto alone accounts for the advantage. Across model families, we find that AoR improves on sampling on OLMo-2-13B and Yi-1.5-9B, where most contrastive baselines score below sampling. Its lead over the best baseline grows from about 3 points for Qwen3 models up to 8B to 5.7 points at 32B, as Figure 3b shows. This
Figure 3. (a) How AoR ends on each task over six models: the plain answer is accepted, a repair is supported, or no draw is supported. (b) Mean on Qwen3 models of five sizes.
| Method | No FaithEval | With veto | CF | Ord. | OLMo | Yi | 32B | 70B | Greedy | Distractors | Qwen | Gemma |
| Sampling | 65.8 | 70.2 | 63.1 | 83.7 | 68.6 | 67.9 | 72.3 | 68.9 | 66.6 | 63.3 | – | – |
| CAD | 67.5 | 71.7 | 66.6 | 81.9 | 60.1 | 64.2 | 74.4 | 68.9 | 67.3 | 63.6 | −1.5 | +1.1 |
| AdaCAD | 68.1 | 71.6 | 65.7 | 83.7 | 68.5 | 67.5 | 74.0 | 71.6 | 67.6 | 64.9 | +0.6 | +1.6 |
| COIECD | 67.5 | 71.8 | 66.7 | 82.0 | 60.0 | 64.6 | 74.3 | 69.6 | 66.5 | 63.5 | −1.2 | +1.3 |
| NWCAD | 66.0 | 68.6 | 63.4 | 83.7 | 68.7 | 67.8 | 72.3 | 69.1 | 66.6 | 63.5 | 0.0 | +0.1 |
| RPCD | 67.7 | 72.2 | 67.8 | 83.2 | 63.5 | 68.2 | 73.5 | 72.7 | 67.8 | 65.8 | −0.8 | +1.1 |
| ID | 69.5 | 72.4 | 69.4 | 83.2 | 64.3 | 56.1 | 72.8 | 66.4 | 67.6 | 66.2 | +0.5 | +2.1 |
| Strict prompt | 67.5 | 70.8 | 66.7 | 83.7 | 70.7 | 67.3 | 74.2 | 73.4 | 68.3 | 63.9 | +0.5 | +1.2 |
| Strict CAD | 67.9 | 71.3 | 69.5 | 80.9 | 59.4 | 61.4 | 73.1 | 68.8 | 69.1 | 62.5 | −2.0 | +1.1 |
| AoR (ours) | 71.1 | 74.9 | 70.4 | 84.9 | 75.0 | 75.3 | 80.0 | 78.2 | 73.9 | 68.4 | +2.1 | +3.4 |
caption: Beyond the main setting: Mean, or group average for held-out data. No FaithEval: NQ-Swap, ordinary average; With veto: veto for all; Greedy: four models, retuned, first row greedy; Distractors: passages of four other questions; Judge: free-form Mean gain over sampling.
:::
suggests that checking which answers need repair remains useful as model size increases. With the decoder settings retuned for greedy decoding, AoR retains its lead over the baselines. It also improves over sampling when answers are given as one sentence, as detailed in Appendix § 15. Under two LLM judges that accept paraphrased answers, AoR ranks highest on average, suggesting that its gains are not explained by word-overlap scoring alone.
Where AoR acts. AoR accepts 82 to 86% of initial answers on ordinary passages and 70% on NQ-Swap, as Figure 3a shows. It returns them unchanged, without repair decoding or a closed-book draw. Across free-form tasks, AoR finds a supported repair for more than half of the rejected initial answers. On most FaithEval questions, the passage contains no option text or multiple option texts, so no answer passes the support test and AoR returns a fallback answer. In these cases, the memory veto can still mask the closed-book option during repair decoding when the passage favors another. Table 7 illustrates acceptance, CAD and ID repairs, and the memory veto.
| Task, model | Passage, excerpt | Question | Sampling | CAD | AoR |
| NQ-Orig. Qwen3-8B | UConn owns the two longest winning streaks . . . The longest streak, 111 straight wins, started . . . | how many games in a row have the uconn women’s basketball team won | 111 ✓ | two ✗ | 111 ✓ accepted |
| NQ-Swap Qwen3-8B | In 1971, after the Irish War of Independence . . . most of Ireland seceded from the United Kingdom . . . | when did ireland gain its independence from england | 1922 ✗ | 1971 ✓ | 1971 ✓ CAD repair |
| NQ-Swap Llama-3.1-8B | Neal Dahlen as Hiccup Horrendous Haddock III, the son of the Viking chief Stoick the Vast and Valka. | who voices hiccup in how to train your dragon 2 | Jay Baruchel ✗ | Jay Baruchel ✗ | Neal Dahlen ✓ ID repair |
| FaithEval Qwen3-14B | . . . recent studies have shown that minerals hold a pivotal role in the fossilization process. . . . | Which of the following would most likely form a fossil? A. a rock, B. a crystal, C. a mineral, D. a bone | D ✗ | D ✗ | C ✓ D vetoed |
Table 7. Test answers from three models, with passage excerpts, illustrating acceptance, CAD and ID repairs, and the memory veto. A check marks a correct answer; labels below AoR’s answers indicate the action taken.
Conclusion
We propose Accept-or-Repair, a training-free procedure that keeps passage-supported initial answers and uses contrastive decoding to repair unsupported ones. AoR also applies the memory veto to reject repairs that repeat the answer generated without the passage when the passage favors another answer. AoR improves average scores over plain sampling on counterfactual and ordinary passages across ten models. In the main evaluation on six models and five benchmarks, it yields large gains on counterfactual passages and small gains on ordinary ones. Replacing only unsupported initial answers improves the average score of every decoder in this evaluation, compared with using that decoder to answer every question. Our findings suggest that checking passage support helps models follow passages that contradict their knowledge while preserving accuracy on passages that agree with it.
Limitations
AoR uses word matching to check passage support for short answers and single sentences. Long-form or abstractive answers call for support tests that account for paraphrasing and information combined across the passage. We examine model self-checking and an entailment model as alternative tests on short-answer tasks in Appendix § 14. Our experiments cover English benchmarks and models of up to 70B parameters.
The contrastive repair decoders require access to the model’s next-token distributions. We also evaluate strict-prompt resampling as an alternative repair generator in § 5.
Ethics Statement
All datasets and models are public, and we use them for research under the terms of their releases: NQ-Swap and ConFiQA under the MIT license, SQuAD and HotpotQA under CC BY-SA 4.0, SciQ under CC BY-NC 3.0, Qwen3, Qwen2.5-32B-Instruct, Gemma 4, Mistral-7B-Instruct, OLMo-2 and Yi-1.5 under Apache-2.0, Llama-3.1 under the Llama 3.1 Community License, and the DeBERTa-v3 entailment model under the MIT license.
AoR helps models base their answers on the provided passage. Applications should assess source reliability, since passage support alone does not establish factual correctness (Lee et al., 2022).
References
Prompts and Decoders
Initial answers and contrastive decoders use single-turn user prompts rendered with the model’s chat template. We add no system message, and the Llama-3.1 template adds its default system header. The Qwen3 models run with thinking disabled.
Table 8 gives the prompts of the free-form tasks. The instruction, passage and question are separated by blank lines, with the labels Passage and Question before the corresponding text. For HotpotQA, the label is Passages, directives refer to multiple passages, and each paragraph begins with its title and a colon. For the multiple-choice tasks the directive reads “Answer the multiple-choice question according to the passage”, the format sentence asks for only the letter of the correct option, and the question is followed by the options, one on each line in the form “A. text”. The contrastive prompts make the same changes to the instruction or passage as their free-form counterparts in Table 8. The closed-book prompt reads “Answer the multiple-choice question. Reply with only the letter of the correct option.” followed by the question and the options.
| Prompt | Text | Used by |
| Standard | Answer the question using the passage. Reply with only the answer, a few words at most. + passage + question | sampling, AoR, every contrastive decoder except Strict CAD |
| Closed-book | Answer the question. Reply with only the answer, a few words at most. + question | the memory veto of AoR, contrast of CAD, AdaCAD, COIECD, NWCAD and Strict CAD |
| Other passage | standard instruction + passage of another example + question | contrast of RPCD |
| Opposite directive | Answer the question using your own knowledge and ignore the passage. Reply with only the answer, a few words at most. + passage + question | contrast of ID |
| Strict | Answer the question using only the passage, even if the passage contradicts what you know. Reply with only the answer, a few words at most. + passage + question | Strict prompt, Strict CAD |
| Copy | Answer the question using the passage. Reply with only the answer, copied from the passage in as few words as possible. + passage + question | copy prompt, alone and with CAD |
| Feedback | standard prompt + plain answer + Your answer does not occur in the passage. Answer the question again using the passage. Reply with only the answer, a few words at most. | feedback repair |
Table 8. Prompts of the free-form tasks. A plus sign joins the parts of a prompt. Shading marks the two prompts that AoR samples from. Its repairs use the prompts of CAD and ID.
CAD, Strict CAD, RPCD and ID score a token at each step by , where is the prompt with the passage, is the contrasted prompt in Table 8 and is the contrast strength. They sample from this score using the model’s temperature and truncation settings. CAD and Strict CAD contrast with the closed-book prompt, RPCD with the passage of another example and ID with the opposite directive. Strict CAD uses the strict prompt as . For RPCD, we take the passage from the example places later in the same pool of examples, wrapping around at the end.
AdaCAD scales at every step by the Jensen–Shannon divergence between the two distributions in bits. COIECD adds the contrast to the log probability under when a token violates its entropy constraint, and to the closed-book log probability otherwise. NWCAD uses the closed-book distribution when it is confident and close to the distribution under . Otherwise, it uses the distribution under when that distribution is confident, and the CAD score when neither condition holds. Confidence is measured by the gap between the two highest token probabilities. We use the paper’s margins of 0.30 for the closed-book distribution and 0.05 for the distribution under . We implement COIECD and NWCAD from their papers and also consult the released COIECD code.
Support Test and Veto
Support test. For short free-form answers, the support test reads the first line. Normalization lowercases the answer and passage, replaces ASCII punctuation with spaces and preserves other punctuation and symbols as separate tokens. It then removes the articles a, an and the, and collapses whitespace. A normalized answer passes when it is nonempty and occurs in the normalized passage as a contiguous sequence of whole tokens. Answers that normalize to yes or no also pass. For multiple-choice questions, we extract the option letter as in evaluation and apply the same normalized token matching to each option’s text. The answer passes only when the selected option is the sole match.
Memory veto. We sample one closed-book answer for each question and reuse it across the three runs. For free-form questions, the veto rejects repairs that match after normalization if fails the support test.
For multiple-choice questions, an option scores 2 when its text passes the token-matching rule above. Otherwise, its score is the fraction of its normalized words, excluding stop words, that occur in the normalized passage. If another option scores higher than the option selected by , we mask tokens representing the letter of by setting their scores to at every repair decoding step. These tokens are identified by stripping spaces, tabs, line breaks, parentheses, periods and colons from the ends of their decoded text and checking whether the remainder equals the letter. If no option can pass both the support test and the veto, AoR returns the first repair that names a non-vetoed option as its fallback and stops generating repairs.
Cost and latency. Repairs and the closed-book answer are generated only for unsupported initial answers. The token cost in Table 1 counts prompt and generated tokens across decoding calls, including tokens processed under both prompts in contrastive decoding.
We measure latency with Qwen3-8B at batch size one on an A100 GPU, using 100 test examples from each ordinary task. For these free-form questions, we generate the closed-book answer alongside the CAD repair because generating the repair does not depend on the closed-book answer. AoR takes 1.30 times as long as sampling, compared with 1.32 for CAD.
Evaluation Details
Data. Table 9 reports the sizes of the data splits. For the main tasks, we use the validation splits of SQuAD and HotpotQA’s distractor setting, the development split of NQ-Swap, which also supplies NQ-Original, and the counterfactual test split of FaithEval version 1.0. The held-out tasks use the test splits of the three ConFiQA subsets, the TriviaQA web development set from MRQA 2019, and SciQ’s validation and test splits with their support paragraphs. For ConFiQA, we keep questions whose changed answer shares no alias with the original answer and whose passage has at most 1,000 words. We score F1 against the answer stated in the passage. The original versions pair the same questions with the unchanged passages and answers. We randomly sample 1,000 examples from each task, reserving 200 for development and 800 for testing. We use development examples to select decoding settings and AoR’s repair sequence, keeping model weights fixed.
| Task | Metric | Dev. | Test | Used |
| Counterfactual passages | ||||
| NQ-Swap | F1 | 200 | 800 | 724 |
| FaithEval | accuracy | 200 | 800 | 800 |
| Ordinary passages | ||||
| NQ-Original | F1 | 200 | 800 | 724 |
| SQuAD | F1 | 200 | 800 | 724 |
| HotpotQA | F1 | 200 | 800 | 800 |
| Held-out counterfactual passages | ||||
| ConFiQA-QA, -MR, -MC | F1 | 200 | 800 | 800 |
| Held-out ordinary passages | ||||
| ConFiQA original | F1 | 200 | 800 | 800 |
| TriviaQA web | F1 | 200 | 800 | 800 |
| SciQ | accuracy | 200 | 800 | 800 |
Table 9. Counts of development and test examples. Used excludes test examples whose question, or for SQuAD whose passage, occurs in the development split. ConFiQA rows represent each subset.
Scoring. We score the first line of short free-form answers with token-level F1 after lowercasing and removing punctuation and articles, taking the highest score across reference answers. For multiple-choice tasks, we score accuracy after extracting an option letter. We check for a valid option letter first at the start of the response, then after “answer”, “option” or “choice”, and finally in parentheses or an option label at the start of a line. Responses without a recognized option letter are incorrect. Generation stops after at most 48 new tokens for short free-form answers and 32 for multiple-choice answers.
Sampling. All runs use bfloat16 weights, PyTorch (Paszke et al., 2019) and Transformers (Wolf et al., 2020) on RTX A6000, A100 and H100 GPUs. Table 10 lists the sampling settings for each model. To pair sampled outputs across methods, we generate Gumbel noise with a 32-bit integer hash of the run seed, example identifier, decoding step and token index. After contrastive adjustment, temperature scaling and filtering, we add this noise to the token scores and select the token with the highest score. This samples from each method’s distribution while giving methods with identical distributions the same token. AoR uses the initial answer’s noise for its repairs, so its CAD repair matches the CAD baseline unless the memory veto masks an option. A second draw from the same decoder uses the next run’s noise.
Significance. For the main comparison, confidence intervals use 2,000 bootstrap resamples of questions, except for SQuAD, where we resample passages together with all their questions. The resampling indices are shared across models and runs, and across the paired NQ-Swap and NQ-Original tasks. For each resample, we average score differences over the tasks in each group and take the mean of the two group differences to compute Mean. Task-level comparisons use a Holm correction.
Baseline Settings
Table 11 reports development gains over sampling for each setting, averaged across the counterfactual and ordinary groups. We select the setting with the highest average gain, retaining the original paper’s recommendation when it is within 0.1 points of the best. Strict CAD uses CAD’s selected strength.
| Method | Parameter | Settings tried | Scores |
| CAD | strength | 0.25, 0.5, 1, 2 | 1.81, 2.67, 2.66, −0.49 |
| AdaCAD | strength | 0.5, 1, 2, 4 | 1.26, 1.67, 1.54, 0.35 |
| RPCD | strength | 0.25, 0.5, 1, 2 | 1.85, 2.75, 3.11, 1.00 |
| ID | strength | 0.25, 0.5, 1, 2, 4, 8 | 0.92, 1.73, 2.82, 3.10, 0.76, −10.89 |
| NWCAD | divergence threshold | 0.05, 0.1, 0.2, 0.3, 0.5 | 0.17, 0.23, 0.20, −0.16, −0.76 |
| COIECD | strength, constraint scale | (0.5, 0.25), (1, 0.25), (2, 0.25), (1, 0.1), (1, 0.5) | −12.55, 1.19, 2.60, 1.54, 0.51 |
Table 11. Settings tried on the development split and their scores in the same order. Scores average the counterfactual and ordinary gains over sampling, in points. Bold marks the chosen setting.
Table 12 reports test gains across these settings, with Strict CAD evaluated at the same strengths as CAD. Gains on ordinary passages remain small, and the settings with the largest counterfactual gains lower ordinary scores. This pattern suggests that the trade-off is not confined to the selected settings.
| Method | Settings | Counterfactual | Ordinary |
| CAD | 0.25, 0.5, 1, 2 | 3.7, 5.9, 7.8, 8.0 | 0.2, −0.1, −2.1, −7.7 |
| AdaCAD | 0.5, 1, 2, 4 | 2.3, 3.2, 3.7, 3.2 | 0.2, 0.1, −0.5, −2.5 |
| RPCD | 0.25, 0.5, 1, 2 | 2.8, 4.5, 5.8, 5.7 | 0.8, 0.7, −0.3, −4.2 |
| ID | 0.25, 0.5, 1, 2, 4, 8 | 1.5, 2.7, 4.5, 5.8, 4.2, −7.7 | 0.4, 0.5, 0.8, 0.0, −4.3, −15.3 |
| NWCAD | 0.05, 0.1, 0.2, 0.3, 0.5 | 0.2, 0.3, 0.1, −0.1, −0.7 | 0.0, 0.1, −0.3, −0.6, −1.2 |
| COIECD | (0.5, 0.25), (1, 0.25), (2, 0.25), (1, 0.1), (1, 0.5) | −22.8, 1.9, 7.7, 2.7, 1.0 | −4.3, 0.2, −1.9, 0.1, 0.1 |
| Strict CAD | 0.25, 0.5, 1, 2 | 6.0, 7.9, 9.6, 8.8 | −0.2, −1.1, −3.6, −10.3 |
| Strict prompt | none | 2.2 | 0.0 |
Table 12. Gain over sampling at each setting of Table 11, on the counterfactual and the ordinary average of the test split. Bold marks the chosen setting.
For NWCAD, we trace decoding on 200 test examples from each task using Qwen3-8B at the selected setting. The share of steps using the CAD score reaches a maximum of 0.5% across tasks, with the remaining steps sampling from the prompt with or without the passage.
Further Analyses
Main comparison. Against Strict CAD, the strongest baseline, AoR gains 2.9 points on counterfactual and 5.9 on ordinary passages. Every baseline trails it significantly in both groups, and its smallest lead in Mean on one of the six models is 2.9 points. On individual tasks, 44 of the 45 gains of AoR over a baseline remain significant after Holm correction. Without FaithEval, it leads the closest baseline, ID, by 1.6 points in Mean.
Supported answers. In Table 2, support is determined by the test of AoR. An answer is correct when it matches a gold answer exactly, or for FaithEval when it names the right option. On ordinary passages, 83.6% of the plain answers are supported and score 82.5 F1. On these answers CAD scores 4.1 points lower and Strict CAD 6.2. CAD changes 18.5% of them, and 41% of these changes give an answer that the passage does not contain. The 4 correct plain answers that AoR breaks on ordinary passages are unsupported because the passage contains the gold answer only inside a longer word, as Ghana inside Ghanaian.
Precision of the support test. Of the plain answers that the test accepts on ordinary passages, 84.9% have an F1 of at least 0.5. Using this threshold to define correctness, every baseline breaks more supported initial answers than it corrects, from 1.6 times as many for NWCAD to 4.9 times for Strict CAD. Within this supported subset, retaining the initial answers improves the binary score over the decoder exactly when this ratio exceeds one. Across free-form tasks, 72 to 95% of supported plain, CAD and ID draws have an F1 of at least 0.5, compared with 16 to 53% of unsupported draws. 1.7% of the plain answers that AoR accepts on NQ-Swap equal the closed-book answer. On the 6.6 to 20.5% of ordinary examples whose passage also contains a wrong closed-book answer, AoR scores highest on each ordinary task.
Repair term. For AoR itself, of Equation (1) splits into a repair term and a fallback term,
where is the share of unsupported answers that a supported repair replaces, and are the mean scores of the supported repairs and of the fallbacks, and and those of the plain answers they replace. AoR thus gains over sampling exactly when the repair term exceeds the fallback loss . On the free-form tasks, the supported repairs of AoR score 50 to 78 F1, where the plain answers they replace score 19 to 48. The repair term is 12 to 25 points, while the fallback loss stays within 2.2 points of zero. On ordinary passages, AoR raises the F1 of 2.3 answers for each one it lowers, and no baseline raises more than 1.1.
Where contrastive decoders break answers. Table 13 gives the share and the plain F1 of each bin of Figure 4, with the probability taken under the closed-book prompt. Strict CAD, CAD and COIECD lose more F1 in the higher closed-book probability bins, while AdaCAD, which weakens the contrast where the two predictions agree, loses little across bins. CAD, COIECD and Strict CAD break correct supported answers more often when the closed-book answer is also correct. With correctness defined as in Table 2 and the six models pooled, CAD breaks 15.1% of these answers, compared with 11.5% when the closed-book answer is wrong. The corresponding rates are 14.7 and 11.3% for COIECD, and 18.8 and 15.1% for Strict CAD. RPCD and ID, whose contrasting prompt also contains a passage, show the opposite pattern, with rates of 9.0 versus 9.9% for RPCD and 7.1 versus 9.1% for ID. On ordinary passages, a second plain sample breaks 2.7% of correct initial answers, compared with 12.2% for CAD using the same noise as the initial sample.
| Closed-book probability | ||||
| – | – | |||
| Share (%) | 72.7 | 8.8 | 7.2 | 11.3 |
| Plain F1 | 80.5 | 85.1 | 86.3 | 89.3 |
| F1 change | ||||
| CAD | −3.0 | −5.4 | −7.1 | −7.4 |
| AdaCAD | −1.0 | −1.3 | −1.2 | −0.7 |
| COIECD | −2.9 | −5.5 | −6.6 | −7.0 |
| RPCD | −1.9 | −2.8 | −4.0 | −2.9 |
| ID | −2.1 | −3.0 | −3.7 | −3.1 |
| Strict CAD | −4.8 | −8.4 | −10.6 | −9.7 |
Table 13. Supported plain answers on ordinary tasks, grouped by their closed-book probability . Rows give the share, plain F1 and decoder F1 changes. Test split, six models.
Wrapping other decoders. Every gain in Mean of Table 3, from 0.1 to 2.7 points, is significant. Draws that state no answer are rare, so in Equation (2) stays below 0.3 points. The counterfactual score of each wrapped decoder stays within 0.7 points of its score when used alone.
Ablations. Table 14 gives six variants of Table 4 on each task, together with a fallback to the plain answer. Without the initial acceptance test, AoR still scores 75.7 on ordinary passages, above sampling. The support checks on repairs remain in place. Accepting an answer that some span of the passage matches with a token F1 of at least 0.5 or 0.8 gives a Mean of 73.1 or 74.4. On 4.4% of NQ-Swap questions, the 0.5 threshold accepts an initial answer containing the answer from the original passage. CAD or ID alone lowers the Mean by 1.3 points, and at the 16 combinations of CAD and ID strengths the Mean lies between 72.5 and 75.4.
| Variant | NQ-Swap | FaithEval | NQ-Orig. | SQuAD | HotpotQA | CF | Ord. | Mean |
| Sampling | 56.9 | 63.6 | 76.0 | 84.5 | 63.5 | 60.3 | 74.7 | 67.5 |
| AoR | 65.2 | 80.4 | 78.2 | 87.5 | 65.2 | 72.8 | 77.0 | 74.9 |
| Repair every answer | 66.6 | 80.4 | 76.5 | 86.8 | 63.8 | 73.5 | 75.7 | 74.6 |
| No memory veto | 65.2 | 74.0 | 78.2 | 87.5 | 65.2 | 69.6 | 77.0 | 73.3 |
| CAD only | 62.8 | 79.5 | 76.9 | 86.7 | 64.5 | 71.2 | 76.0 | 73.6 |
| ID only | 64.2 | 76.5 | 78.1 | 86.9 | 65.2 | 70.4 | 76.8 | 73.6 |
| Resample | 58.1 | 75.2 | 76.8 | 85.3 | 64.1 | 66.7 | 75.4 | 71.0 |
| Resample, strict prompt | 61.2 | 74.7 | 77.4 | 85.7 | 64.5 | 67.9 | 75.8 | 71.9 |
| Plain answer as fallback | 64.5 | 67.2 | 78.5 | 87.4 | 65.2 | 65.9 | 77.0 | 71.5 |
Table 14. Variants of AoR on each task, test split, six models. The plain answer as fallback returns the plain answer, instead of the first answer not rejected by the veto, when no draw is supported.
Memory veto. Removing the memory veto lowers FaithEval accuracy by 6.4 points and the counterfactual average by 3.2 points. Table 15 separates the veto from the decoders on FaithEval, and Table 16 gives the veto to every baseline. The first row of Table 15 picks the option with the highest veto score and splits ties evenly. A decoder with the veto returns the answer of its run with the masked letter where the veto holds and its usual answer elsewhere. On free-form tasks, the veto does not alter a baseline’s answer because it filters only the fallback candidates used by AoR. The baselines with the veto trail AoR by at least 1.2 points on counterfactual and 2.2 on ordinary passages.
| Decoder | Without veto | With veto |
| Veto score as a solver | 61.4 | – |
| Sampling | 63.6 | 74.5 |
| ID | 67.8 | 76.1 |
| CAD | 73.7 | 79.2 |
| AoR | 74.0 | 80.4 |
Table 15. FaithEval accuracy of each decoder without and with the memory veto, which masks the letter of a vetoed closed-book answer. The first row picks the option with the highest veto score. Test split, six models.
| Method | CF | Ord. | Mean |
| Sampling + veto | 65.7 | 74.7 | 70.2 |
| CAD + veto | 70.8 | 72.6 | 71.7 |
| AdaCAD + veto | 68.4 | 74.8 | 71.6 |
| COIECD + veto | 70.8 | 72.8 | 71.8 |
| NWCAD + veto | 62.4 | 74.8 | 68.6 |
| RPCD + veto | 69.9 | 74.4 | 72.2 |
| ID + veto | 70.2 | 74.7 | 72.4 |
| Strict prompt + veto | 66.9 | 74.6 | 70.8 |
| Strict CAD + veto | 71.6 | 71.1 | 71.3 |
| AoR (ours) | 72.8 | 77.0 | 74.9 |
Table 16. Each baseline with the memory veto of AoR, which on FaithEval masks the letter of a vetoed closed-book answer. Test split, six models.
Repairs without contrast. With repairs sampled from the strict prompt, AoR achieves a Mean of 71.9, 1.4 points above Strict CAD. On NQ-Swap, the gain over sampling is 4.2 points, compared with 8.3 with contrastive repairs.
Majority voting. The voting variants in Table 4 select the most frequent answer after applying the support test’s normalization. One votes among the plain answers of three runs, and the other among the plain, CAD and ID answers of one run. Both return the initial plain answer when no answer occurs twice.
Feedback repair. The feedback repair adds the unsupported plain answer as an assistant turn and the feedback instruction from Table 8 as a new user turn, then samples another answer with the same noise. Asked again, the model repeats 45% of the rejected answers, after the normalization of the support test, and 15% of its new answers are supported, against 33% of the CAD answers to the same questions. As the only repair of AoR, feedback gains 1.6 points over sampling on NQ-Swap and 0.2 on ordinary passages. A CAD repair gives larger gains of 5.9 and 1.3 points, respectively, at a similar token cost.
Spans of the passage. Table 17 compares other ways to answer in the words of the passage, and AoR with one repair, on the four free-form tasks, since a multiple-choice answer is a letter. Span decoding samples the plain answer under the constraint that it is a span of the passage. At each step, it considers the 64 highest-probability tokens and keeps those that extend a prefix of a passage span starting at a word boundary, or a prefix of yes or no. The answer can end only at a complete passage span or at yes or no. If no token satisfies the constraint before the answer is complete, decoding continues without the constraint. The copy prompt of Table 8 asks for the answer copied from the passage, for sampling and for CAD, which keeps its closed-book prompt. The copy prompt lowers the ordinary score by 1.3 points, and span decoding trails AoR by 3.8 points on NQ-Swap and 0.7 on ordinary passages.
| Method | NQ-Swap | Ord. | Mean | Cost |
| Sampling | 56.9 | 74.7 | 65.8 | 1.00 |
| Span decoding | 61.4 | 76.3 | 68.8 | 1.00 |
| Copy prompt | 59.6 | 73.4 | 66.5 | 1.02 |
| Copy prompt, CAD | 59.7 | 66.2 | 63.0 | 1.20 |
| AoR with one repair | ||||
| Feedback | 58.5 | 74.9 | 66.7 | 1.28 |
| CAD | 62.8 | 76.0 | 69.4 | 1.27 |
| AoR (ours) | 65.2 | 77.0 | 71.1 | 1.53 |
Table 17. Span and copy methods and AoR variants on free-form tasks. Ord. averages NQ-Original, SQuAD and HotpotQA, and Mean averages NQ-Swap and Ord. Cost counts processed tokens relative to sampling. Test split, six models.
Span-restricted repair. Span Strict CAD in Table 4 is Strict CAD decoded under the constraint of span decoding. Among six repair sequences with a span-restricted draw that we compared on the development split, the sequence that ends with it scores highest on NQ-Swap and on the ordinary tasks. On the test split it raises NQ-Swap from 65.2 to 67.6 and the Mean of each of the six models. Without the constraint, Strict CAD as the third repair raises the Mean by 0.4 points.
Gates. Table 5 compares gates for selecting which initial answers to replace on the four free-form tasks. All gates use the CAD answer from the same run as the only repair and return it without a support check. For each model, task and run, four gates rank initial answers and flag the same number as the support test rejects. The first two rank by mean token log-probability and by the gain in log-probability from adding the passage to the prompt. The third uses the probability of yes when the answering model is asked whether the passage supports its answer ((Kadavath et al., 2022)). The fourth uses entailment probability from DeBERTa-v3-large ([14]) fine-tuned for natural language inference by (Laurer et al., 2023). Each gate flags the answers with the lowest scores.
The self-check prompt reads “Read the passage and decide whether it supports the proposed answer to the question.”, followed by the passage, the question and the answer and by “Does the passage support the proposed answer? Reply with only yes or no.”, in the plural on HotpotQA. The probability of yes is taken against that of no at the first token of the reply. The entailment model reads overlapping 400-token windows of the passage as premises and takes the highest entailment probability across windows. The hypothesis is the answer, preceded by the question when the answer has at most three words. The closed-book gate flags the plain answers that equal the closed-book answer, 12% of the answers against 20% for the support test.
The Mean of Table 5 averages NQ-Swap and the ordinary average. The support test raises this Mean by 3.6 points, significantly more than the other gates, which gain at most 2.2 points. Under the two LLM judges, its gains over sampling are 1.1 and 2.3 points, compared with at most 0.5 and 1.8 for the other gates. Its lead over each gate is at least 0.6 points, with 95% intervals above zero. With the self-check or the entailment model as its test, AoR stays 1.6 and 2.0 points above the closest baseline in Mean.
Held-out data and further models. On the counterfactual subsets of ConFiQA, AoR gains 7.3 points over sampling, significantly more than every baseline. Across the held-out ordinary tasks, AoR gains 1.2 points over sampling on average, while no baseline improves significantly. On OLMo-2-13B, Yi-1.5-9B, Qwen3-32B and Llama-3.1-70B, AoR gains significantly over sampling in both groups. Its Mean exceeds the closest baseline by 4.2, 7.1, 5.7 and 4.8 points, respectively. Tuning every baseline on the development split of each of OLMo-2-13B, Yi-1.5-9B and Qwen3-32B leaves the closest one 4.2, 5.6 and 3.5 points behind AoR. On the Qwen3 models of Figure 3b, the supported answers cost the best baseline 3.7 points of its Mean at 32B, against 0.7 at 8B.
Passage length and distractors. Split into quartiles by the length of the passage, AoR gains 0.8 to 9.1 points over sampling in every quartile of every free-form task, including the longest quartile of HotpotQA, whose passages have at least 1,060 words.
For Table 18, every question of NQ-Swap and NQ-Original keeps its passage and receives the passages of four other NQ-Swap questions, in the same version, that contain neither its gold answer nor its original answer as whole words. The question’s own passage occupies a random one of the five positions, shared across its original and counterfactual versions along with the distractors. We separate passages by blank lines and use the HotpotQA prompts. This setting runs once, with the strengths of the main runs. AoR scores highest on both tasks, indicating that its lead persists with the added distractors.
| Method | NQ-Swap | NQ-Orig. | Mean |
| Sampling | 53.8 | 72.7 | 63.3 |
| CAD | 58.8 | 68.5 | 63.6 |
| AdaCAD | 57.9 | 71.9 | 64.9 |
| COIECD | 58.4 | 68.6 | 63.5 |
| NWCAD | 54.3 | 72.7 | 63.5 |
| RPCD | 59.1 | 72.6 | 65.8 |
| ID | 60.7 | 71.8 | 66.2 |
| Strict prompt | 56.0 | 71.8 | 63.9 |
| Strict CAD | 59.6 | 65.4 | 62.5 |
| AoR (ours) | 61.2 | 75.5 | 68.4 |
Table 18. NQ-Swap and NQ-Original with the passage of each question placed among four passages of other questions, at a random position. Test split, six models.
Other scorers. Table 19 scores a free-form answer as correct when a gold answer occurs in it as whole words after the normalization of the scorer. For Table 20, Qwen2.5-32B-Instruct (Qwen Team, 2024), a model outside the main comparison, judges the free-form answers with greedy decoding. It reads “You grade answers to a reading-comprehension question. A candidate answer is correct if it means the same as one of the reference answers, even when it is worded differently, for example with an alias, an abbreviation, a number written in words, or extra words around the answer. It is wrong if it names something else or adds a claim that contradicts the references. Judge only against the references.”, followed by the question, the gold answers and the answer, and replies with yes or no. When a method’s answer equals the plain answer of the same run after the support test’s normalization, it receives the same judgment. Differences from sampling therefore reflect changed answers alone.
| Method | CF | Ord. | Mean |
| Sampling | 64.2 | 76.6 | 70.4 |
| CAD | 72.8 | 75.5 | 74.1 |
| AdaCAD | 67.4 | 76.8 | 72.1 |
| COIECD | 72.7 | 75.6 | 74.1 |
| NWCAD | 64.6 | 76.6 | 70.6 |
| RPCD | 70.7 | 76.5 | 73.6 |
| ID | 68.8 | 74.0 | 71.4 |
| Strict prompt | 66.2 | 76.1 | 71.1 |
| Strict CAD | 74.1 | 74.0 | 74.1 |
| AoR (ours) | 75.8 | 77.2 | 76.5 |
Table 19. Scores when a free-form answer counts as correct if it contains a gold answer as whole words, which does not penalize longer answers. FaithEval keeps accuracy. Test split, six models.
| Method | NQ-Orig. | SQuAD | HotpotQA | Ord. | NQ-Swap |
| CAD | −4.6 | −1.8 | −4.9 | −3.8 | +0.9 |
| AdaCAD | −0.6 | 0.0 | −1.2 | −0.6 | +1.8 |
| COIECD | −4.2 | −1.8 | −4.5 | −3.5 | +1.0 |
| NWCAD | −0.1 | −0.2 | 0.0 | −0.1 | 0.0 |
| RPCD | −2.3 | −1.7 | −3.0 | −2.3 | +0.8 |
| ID | −3.4 | −1.0 | −1.0 | −1.8 | +2.8 |
| Strict prompt | −0.8 | −0.3 | −0.9 | −0.7 | +1.7 |
| Strict CAD | −7.1 | −3.3 | −7.9 | −6.1 | +2.0 |
| AoR (ours) | 0.0 | +0.8 | +0.2 | +0.3 | +3.9 |
Table 20. Accuracy differences from sampling on free-form answers, judged by Qwen2.5-32B-Instruct. Test split, six models.
Among 300 random plain answers with an exact match, the judge accepts 93%, and among 300 with an F1 of zero, 8%. The second judge, Gemma-4-31B-it [12], reads the same prompt. A reply counts as yes when its first word is yes. It accepts 100% and 9% of the same two samples.
The judge columns of Table 6 report gains over sampling in the Mean of NQ-Swap and the ordinary average. Intervals for judged differences use 2,000 resamples of the questions in each task.
Of the 167 judged NQ-Swap answers that equal the question’s original answer, Qwen2.5-32B-Instruct accepts 0 and Gemma-4-31B-it 1. Under Qwen2.5-32B-Instruct, AoR gains 3.9 points over sampling on NQ-Swap and 0.32 on ordinary passages, both significantly, and it leads every baseline significantly on both. Under Gemma-4-31B-it, AoR gains 7.1 points on NQ-Swap, more than any other method, while CAD loses 2.7 and Strict CAD 4.6 points on ordinary passages. Under this judge, the wrapper of Table 3 raises the judged ordinary accuracy of the seven decoders that lose on ordinary passages by 0.5 to 5.7 points.
Under the two judges, AoR breaks 1.6% and 1.7% of the ordinary plain answers that the judge accepts, and CAD 7.8% and 5.9%. Of the ordinary plain answers that the test rejects but a judge accepts, AoR replaces 86% and 87% by answers that the same judge accepts.
Qwen2.5-32B-Instruct rejects 35% of the NQ-Swap answers that equal the substituted gold answer, and Gemma-4-31B-it 0.5%. With such exact answers counted as correct, AoR gains 7.8 points on NQ-Swap under the first judge.
Sentence Answers and Greedy Decoding
Sentence answers. We evaluate sentence answers on NQ-Swap, NQ-Original and SQuAD using the same examples and gold answers as the short-answer experiments. We replace the format instruction in Table 8 with “Reply with one short sentence.” and allow up to 64 new tokens. An answer is correct when a gold answer occurs in its first line as whole words after scorer normalization.
The support test normalizes the answer’s first line and the passage as in Appendix § 11, then removes question words and function words from the answer. The answer passes when at least one word remains and all remaining words occur in the passage. Answers normalized to yes or no also pass. We selected the requirement to match all remaining words using the development split. We generate the closed-book answer in the same sentence format and retain the short-answer procedures for the memory veto, repairs and fallback.
We find that AoR improves over sampling on all three tasks in Table 21, with a larger gain on NQ-Swap than on the ordinary tasks.
| Method | NQ-Swap | NQ-Orig. | SQuAD | Ord. | Mean |
| Sampling | 64.9 | 86.3 | 90.9 | 88.6 | 76.7 |
| AoR (ours) | 73.3 | 86.7 | 91.9 | 89.3 | 81.3 |
| Difference | +8.4* | +0.4 | +1.0 | +0.7* | +4.6* |
Table 21. Answers given as one sentence, correct when they contain a gold answer. Ord. averages NQ-Original and SQuAD, and Mean averages NQ-Swap and Ord. *: the 95% interval of the difference excludes zero. Test split, six models.
Greedy decoding. We run each method once on each task for the four models in Table 22, choosing the token with the highest score at each step. We reselect decoder settings by the rule in § 4, evaluating the grid in Table 11 with greedy
| Method | CF | Ord. | Mean | Llama | Mistral | OLMo | Yi |
| Greedy | 58.4 | 74.8 | 66.6 | 70.2 | 58.0 | 69.4 | 68.8 |
| CAD | 63.3 | 71.4 | 67.3 | 74.1 | 62.1 | 67.7 | 65.4 |
| AdaCAD | 61.7 | 73.5 | 67.6 | 72.3 | 61.1 | 69.0 | 68.1 |
| COIECD | 60.8 | 72.2 | 66.5 | 72.5 | 60.5 | 67.8 | 65.3 |
| NWCAD | 58.5 | 74.7 | 66.6 | 70.5 | 58.1 | 69.3 | 68.3 |
| RPCD | 62.5 | 73.0 | 67.8 | 72.6 | 63.8 | 68.5 | 66.2 |
| ID | 61.5 | 73.8 | 67.6 | 73.9 | 63.2 | 68.2 | 65.2 |
| Strict prompt | 61.9 | 74.7 | 68.3 | 74.1 | 60.0 | 71.8 | 67.4 |
| Strict CAD | 66.7 | 71.4 | 69.1 | 75.4 | 63.8 | 68.8 | 68.2 |
| AoR (ours) | 70.8 | 77.1 | 73.9 | 77.4 | 66.6 | 76.4 | 75.3 |
Table 22. Greedy decoding of every method on Llama-3.1-8B, Mistral-7B, OLMo-2-13B and Yi-1.5-9B, test split, with every setting chosen again under greedy decoding on the development split. The memory veto of AoR uses the sampled closed-book answer of the main runs.
decoding on the four models’ development split. The chosen strengths are 0.5 for CAD, Strict CAD and RPCD, and 1 for AdaCAD and ID. NWCAD uses a divergence threshold of 0.1, and COIECD uses strength 1 and a constraint scale of 0.1. AoR generates its initial answer and CAD/ID repairs greedily at these settings and reuses the sampled closed-book answer from the main runs to keep the memory veto’s reference answer fixed.
Under greedy decoding, AoR gains 7.3 points in Mean over plain greedy decoding and outperforms the baselines on each model in Table 22. Together, these results suggest that AoR’s gains extend beyond short answers generated by sampling.
Examples
Table 7 illustrates how AoR accepts an initial answer, returns a CAD or ID repair, and applies the memory veto. On the ordinary passage, CAD answers two, a count of winning streaks, to a question about consecutive wins. AoR keeps the supported initial answer 111.
On the first NQ-Swap passage, AoR replaces the initial answer 1922 with CAD’s answer 1971, the year given in the passage. On the second, CAD
repeats Jay Baruchel, whose name is absent from the passage. AoR rejects this repair and returns the ID answer Neal Dahlen, which occurs in the passage.
On FaithEval, sampling and CAD choose D, which is also the closed-book answer. AoR’s memory veto masks D because the passage favors C, and CAD then generates C as the repair.
Results by Model
Table 23 lists average scores for each method and each of the six models in the main comparison.
| Method | Qwen3-1.7B | Qwen3-4B | Qwen3-8B | Qwen3-14B | Llama-3.1-8B | Mistral-7B |
| Sampling | 65.1 | 68.1 | 72.2 | 72.2 | 69.5 | 57.7 |
| CAD | 65.3 | 70.6 | 76.2 | 75.5 | 72.3 | 62.3 |
| AdaCAD | 65.9 | 69.4 | 73.3 | 73.2 | 71.9 | 60.9 |
| COIECD | 65.5 | 70.6 | 76.1 | 75.5 | 72.4 | 62.2 |
| NWCAD | 65.4 | 68.3 | 72.3 | 72.2 | 69.8 | 57.9 |
| RPCD | 65.8 | 70.4 | 74.3 | 74.6 | 71.2 | 65.1 |
| ID | 62.9 | 71.9 | 75.5 | 75.4 | 71.4 | 65.1 |
| Strict prompt | 64.1 | 68.5 | 73.0 | 73.0 | 73.3 | 59.6 |
| Strict CAD | 64.1 | 70.5 | 76.1 | 75.8 | 72.7 | 63.9 |
| AoR (ours) | 69.0 | 74.7 | 79.0 | 80.3 | 77.7 | 68.6 |
Table 23. Mean of the counterfactual and ordinary averages for each model in the main comparison. Test split.
References
- [2]Akari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil, and Hannah Hajishirzi. 2024. Self-RAG: Learning to retrieve, generate, and critique through self-reflection. In The Twelfth International Conference on Learning Representations.arxiv.org/abs/2310.11511
- [3]Taylor Berg-Kirkpatrick, David Burkett, and Dan Klein. 2012. An empirical investigation of statistical significance in NLP. In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pages 995–1005, Jeju Island, Korea. Association for Computational Linguistics.
- [4]Baolong Bi, Shaohan Huang, Yiwei Wang, Tianchi Yang, Zihan Zhang, Haizhen Huang, Lingrui Mei, Junfeng Fang, Zehao Li, Furu Wei, Weiwei Deng, Feng Sun, Qi Zhang, and Shenghua Liu. 2025. Context-PDO: Aligning Language Models for Context-Faithfulness. In Findings of the Association for Computational Linguistics: ACL 2025.arxiv.org/abs/2412.15280
- [5]Lingjiao Chen, Matei Zaharia, and James Zou. 2024. FrugalGPT: How to use large language models while reducing cost and improving performance. Transactions on Machine Learning Research.arxiv.org/abs/2305.05176
- [6]Yung-Sung Chuang, Linlu Qiu, Cheng-Yu Hsieh, Ranjay Krishna, Yoon Kim, and James R. Glass. 2024. Lookback lens: Detecting and mitigating contextual hallucinations in large language models using only attention maps. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 1419–1436, Miami, Florida, USA. Association for Computational Linguistics.arxiv.org/abs/2407.07071
- [7]Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reichīro Nakano, Christopher Hesse, and John Schulman. 2021. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168.arxiv.org/abs/2110.14168
- [8]Shehzaad Dhuliawala, Mojtaba Komeili, Jing Xu, Roberta Raileanu, Xian Li, Asli Celikyilmaz, and Jason Weston. 2024. Chain-of-verification reduces hallucination in large language models. In Findings of the Association for Computational Linguistics: ACL 2024.arxiv.org/abs/2309.11495
- [10]Luyu Gao, Zhuyun Dai, Panupong Pasupat, Anthony Chen, Arun Tejasvi Chaganty, Yicheng Fan, Vincent Zhao, Ni Lao, Hongrae Lee, Da-Cheng Juan, and Kelvin Guu. 2023. RARR: Researching and revising what language models say, using language models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).arxiv.org/abs/2210.08726
- [11]Aryo Pradipta Gema, Chen Jin, Ahmed Abdulaal, Tom Diethe, Philip Alexander Teare, Beatrice Alex, Pasquale Minervini, and Amrutha Saseendran. 2025. DeCoRe: Decoding by contrasting retrieval heads to mitigate hallucinations. In Findings of the Association for Computational Linguistics: EMNLP 2025, pages 10003–10039, Suzhou, China. Association for Computational Linguistics.arxiv.org/abs/2410.18860
- [12]Gemma Team. 2026. Gemma 4 technical report. arXiv preprint arXiv:2607.02770.
- [13]Jerry Zhi-Yang He, Sashrika Pandey, Mariah L. Schrum, and Anca Dragan. 2025. Context steering: Controllable personalization at inference time. In The Thirteenth International Conference on Learning Representations.arxiv.org/abs/2405.01768
- [14]Pengcheng He, Jianfeng Gao, and Weizhu Chen. 2023. DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing. In The Eleventh International Conference on Learning Representations.




