WorksComputer Science
Better Answers, Weaker Evidence Control: Evidence Bypass in On-Policy Distillation
Abstract
On-policy distillation (OPD) can improve answer accuracy while weakening responsiveness to supporting evidence. We investigate this separation in scientific-document question answering through matched document interventions that change the supported answer while preserving the question and unrelated context. We identify evidence bypass: a student retains an originally correct answer after the document changes to support a different one. On QASPER with a Qwen3-4B student, standard OPD raises original-document accuracy by 3.4 percentage points, lowers counterfactual accuracy by 5.3 points, and increases bypass by 15.7 points; the same directional pattern appears on PeerQA. Controlled perturbations localize the shift to answer-critical evidence, and bypass grows with the student's pre-distillation closed-book preference for the original answer. A factorized study of teacher targets, privileged information, divergence, and supervision density finds that explicitly supplying the teacher with annotated answer-critical spans yields the largest joint improvement in evidence response and claim support. These findings motivate Evidence-Paired OPD (EP-OPD), which combines on-policy supervision on matched documents supporting different answers to the same question with an answer-level margin that favors each document's supported answer over its paired alternative. Across both datasets and two student sizes, EP-OPD recovers 54-66% of standard OPD's counterfactual-accuracy loss while preserving 80-85% of its original-document accuracy gain. Our results establish evidence responsiveness as a distinct training and evaluation target alongside answer accuracy. Our code is publicly available at: https://github.com/RuoranXu/EP-OPD.
Introduction
Research on reasoning faithfulness distinguishes answer correctness and explanation plausibility from whether a model’s stated reasons reflect the factors driving its decisions [1, 2 Chen] et al., 2025). Correct answers and coherent chains of thought (CoT) can coexist with a mismatch between stated justifications and actual decision influences. Interventions on reasoning traces and task inputs probe this relationship by measuring how predictions respond to changes in their stated basis [4, 5]. Reasoning distillation brings this issue into training. Students learn from teacher-generated rationales and CoT traces [6, 7, 8], raising a further question: does transferring reasoning behavior also transfer dependence on supporting evidence?
On-policy distillation (OPD) provides a setting for studying this question: the student generates its own trajectories, and the teacher supervises the prefixes it visits [9, 10]. In document-grounded tasks, these trajectories bring the student’s existing answer preferences into contact with evidence-conditioned teacher supervision. We investigate this question in scientific-document QA. Document evidence and parametric answer preferences often support the same answer on ordinary examples. We examine their relationship through controlled document variation: we change one answer-critical fact and update dependent mentions, producing a matched document that supports a different answer. Fresh responses to both versions measure how distillation changes the student’s responsiveness to supporting evidence.
We identify evidence bypass: a model retains an originally correct answer after the document changes to support a different one. On QASPER with a Qwen3-4B student, standard OPD increases original-document accuracy from 49.0% to 52.4%, while counterfactual accuracy falls from 44.5% to 39.2% and bypass rises from 20.1% to 35.8%. The same directional pattern appears on PeerQA. These results reveal a separation between improvements in answer accuracy and changes in evidence dependence during distillation.
Further analysis connects this pattern to answer-critical information and pre-existing answer preferences. Paraphrases and irrelevant insertions preserve most of OPD’s original-document gain, while evidence deletion reduces correct abstention. To examine how the training signal shapes evidence dependence, we vary four OPD design dimensions: teacher target, teacher privileged information, divergence loss, and supervision density. Among these interventions, providing the teacher with the annotated answer-critical span yields the largest gains in counterfactual accuracy and claim support, together with the largest reduction in bypass.
These findings motivate Evidence-Paired OPD (EP-OPD), which makes document-dependent answer variation explicit during training. Each question is paired with two documents supporting different answers. Both receive on-policy teacher supervision, and a paired answer-level objective encourages the student to favor each document’s supported answer over its paired alternative. Across QASPER and PeerQA and two student sizes, EP-OPD recovers 54–66% of standard OPD’s counterfactual-accuracy loss while preserving 80–85% of its original-document accuracy gain. These results establish evidence dependence as a training and evaluation target alongside answer accuracy.
Our contributions are threefold:
We identify evidence bypass in document-grounded OPD through matched evidence interventions and characterize its relationship to the student’s pre-distillation answer preferences.
We organize OPD design along teacher target, teacher privileged information, divergence loss, and supervision density, showing how each shapes accuracy, evidence response, and claim support.
We introduce Evidence-Paired OPD, which couples on-policy supervision with document-dependent answer preferences to strengthen evidence response while preserving most of distillation’s accuracy gains.
Related work
Chain-of-thought faithfulness and evidence interventions
A correct answer accompanied by a plausible chain of thought (CoT) does not establish that the stated reasoning faithfully reflects the model’s decision process. Ye and Durrett [1] show that generated explanations can lack input grounding. Turpin et al. [2] demonstrate that CoT traces can omit biasing features influencing answers, while Chen et al. [3] document analogous omissions of influential hints in reasoning models. Interventions on reasoning traces and task inputs probe whether predictions depend on their stated basis [4, 5], complementing evaluations of rationale sufficiency, comprehensiveness, and explanation–label association [11, 12]. Counterfactual data and contrast sets probe decision rules through targeted input changes [13, 14], while attribution research assesses whether claims are supported by source material [15, 16]. We apply these perspectives to scientific-document QA using answer-changing document pairs, evidence deletion, and claim-support evaluation. These diagnostics measure behavioral evidence responsiveness rather than directly verify internal reasoning.
On-policy and reasoning distillation
Reasoning distillation transfers teacher behavior through generated rationales and CoT traces [6, 7, 8]. On-policy distillation (OPD) supervises student-generated trajectories: GKD applies teacher distributions at student-visited states [9], MiniLLM optimizes reverse KL using on-policy samples (Gu et al., [10]), and Li et al. [17] analyze progressive teacher–student alignment along these trajectories. CoT faithfulness raises a further question: does imitating teacher reasoning also transfer dependence on supporting evidence? Agreement on observed answers and traces alone need not establish such dependence. SCOTT addresses a related rationale-level issue through counterfactual rationale–answer pairs (Wang et al., [18]). We instead examine document-level evidence dependence under OPD, varying teacher target, teacher privileged information, divergence loss, and supervision density. Evidence-Paired OPD (EP-OPD) trains on matched documents supporting different answers to the same question, coupling answer preferences to source evidence rather than varying supplied rationales.
Context faithfulness and parametric knowledge
Knowledge-conflict research examines how models balance supplied context against parametric knowledge. Entity substitution and counterfactual QA probe reliance on these sources ([19, 20]). FaithEval evaluates context-following under controlled evidence conditions (Ming et al. [21]), and Context-DPO trains preferences for context-faithful answers (Bi et al. [22]). Scientific-reasoning research further shows that stronger reasoning can make existing parametric knowledge more accessible (Li et al. [23]). We connect these observations to distillation by relating the base student’s closed-book answer preference to post-OPD responses under paired document interventions, without interpreting this association as direct identification of an internal reasoning mechanism.
Experimental Framework
We separate a model’s accuracy on the supplied document from its response when answer-critical evidence changes, and compare how training alters each. This section defines the matched document interventions, the metrics and statistical comparisons built on them, and the models and OPD design variations we compare.
Datasets and Document Interventions
We evaluate on two scientific document QA datasets: QASPER comprises 5,049 questions over 1,585 NLP papers (Dasigi et al. [24]), and PeerQA comprises 579 questions drawn from reviews of 208 papers (Baumgaertner et al. [25]). Both provide answers and supporting evidence, with distinct question-collection procedures and answer formats. Splits are disjoint by source paper. The primary evaluation includes answerable questions whose answers can be represented as an entity, value, polarity, comparison, or bounded set; unanswerable examples contribute to abstention evaluation. We use full papers when they fit the context window. Otherwise, a fixed document pack combines annotated evidence with retrieval-ranked distractor paragraphs and is shared across methods.
We measure document response by changing the evidence that determines an answer while keeping the question fixed. For each eligible example, we edit one answer-critical span, changing an entity, value, comparison, polarity, or reported direction. We update dependent mentions consistently and preserve the question, section order, and unrelated context:
We generate a fresh response to each document version and validate each held-out pair so that the edited document uniquely supports its new answer. We also construct a meaning-preserving paraphrase , an evidence-deleted version , and a length-matched irrelevant insertion . The answer-changing edit preserves answer type and scientific syntax and updates dependent mentions. Validators check label uniqueness, residual support for the original answer, and edit locality; held-out pairs also receive manual review before model evaluation. These versions measure response to semantic preservation, evidence availability, distraction, and answer-changing evidence (Appendix § 10).
Evidence-Response Metrics and Evaluation
Three primary quantities summarize answer behavior. Original accuracy measures correctness on ; counterfactual accuracy (CFAcc) measures correctness on . Bypass is computed over the examples each model answers correctly on : the fraction on which the same model retains the original answer on . Because this condition is model-specific, bypass differences compare conditional rates rather than changes on a fixed question set. The three measures capture performance on the observed document, adaptation to changed evidence, and persistence of the original answer. CFAcc and bypass are complementary: avoiding the original answer does not by itself mean producing the answer the edited document supports.
We also measure correct abstention after evidence deletion. We score original accuracy, counterfactual accuracy, and bypass with deterministic, item-specific answer canonicalizers. We split responses into independently evaluable claims and assess source support. A blinded evaluator receives the question, one randomized claim, its cited spans, and minimal neighboring context, with method and outcome metadata hidden. Labels are supported, contradicted, absent from the source, invalid composition, or irrelevant. Claim support precision (CSP) is the fraction of answer-relevant claims labeled supported. Experts evaluate a disjoint subset using the same units and definitions. Segmentation rules, prompts, and evaluation controls appear in Appendix § 11. Appendix § 9.§ 1 gives the metric definitions, paired interaction estimand, and regression form.
Each matched question pair is the comparison unit, and source papers define resampling clusters. We aggregate claim labels by question and report scores averaged across seeds. The primary comparison estimates how the training-induced change in correctness differs between counterfactual and original documents using 10,000 paired, paper-clustered bootstrap replicates. We apply Holm correction to secondary outcomes within each result family.
Models and Factorized OPD Design
The phenomenon analysis uses Qwen3-4B base students and a frozen Qwen3-30B-A3B-Instruct-2507 teacher ([26]). Training comparisons span the pretrained BASE checkpoint, answer-only supervision (ANS-SFT), imitation of teacher reasoning traces whose final answers pass the task verifier (answer-filtered CoT-SFT), and standard reverse-KL OPD. Principal runs share the question pool, optimizer-step count, retrieved context, and supervised output-token budget. We update all student parameters and run three seeds per principal configuration. Hyperparameters and compute matching appear in Appendix § 8.
We factorize standard OPD along four dimensions, teacher target, teacher privileged information, divergence, and supervision density, and vary one dimension at a time under an otherwise shared training setup. Teacher target: fixed reference or EMA with . Teacher privileged information: gold evidence IDs or the answer-critical span. Divergence: reverse KL or forward KL with . Supervised positions: all prefixes or the top 50% or 25% by pre-update student entropy. Questions, rollout settings, optimizer schedules, and student inputs remain fixed within each comparison. Exact objectives appear in Appendix § 8.§ 4.
Teacher privileged information is supplied only to the teacher; student rollouts and evaluation inputs never contain it. Entropy-mask controls hold rollouts and optimizer steps fixed, while the number of teacher-scored tokens intentionally varies with supervision density (Appendix § 8.§ 4). Results of the factorized study are reported in § 4.4 and Table 2.
| Axis | Single change | Orig. | CFAcc | Bypass | CSP |
| Reference | Standard OPD | 52.4 | 39.2 | 35.8 | 57.0 |
| Teacher | EMA, | 54.8 | 41.5 | 33.2 | 59.1 |
| Teacher | EMA, | 51.6 | 39.7 | 35.4 | 57.5 |
| Priv. info. | Weak PI: evidence IDs† | 57.3 | 47.8 | 22.8 | 68.2 |
| Priv. info. | Strong PI: critical span† | 60.6 | 52.6 | 14.8 | 76.0 |
| Divergence | Forward KL, | 50.7 | 43.1 | 28.8 | 63.2 |
| Divergence | Forward KL, | 48.9 | 45.7 | 24.5 | 66.4 |
| Density | Top-entropy 50% | 47.2 | 39.8 | 34.1 | 58.1 |
| Density | Top-entropy 25% | 43.8 | 36.9 | 38.2 | 54.3 |
Table 2. Factorized OPD interventions on QASPER with Qwen3-4B (%). Each intervention changes one design dimension relative to standard OPD. †Gold evidence information is provided only to the teacher.
Evidence Bypass under On-Policy Distillation
We examine how standard OPD changes evidence response in Qwen3-4B students, establishing the relationship between original-document accuracy and original-answer persistence under counterfactual evidence. We then characterize this change through document interventions and pre-distillation answer preferences, before examining how evidence response varies across the OPD design choices defined in § 3.3.
Accuracy Gains Accompany Stronger Original-Answer Persistence
On QASPER, standard OPD increases original-document accuracy from 49.0% to 52.4%. On the matched answer-changing documents, counterfactual accuracy falls from 44.5% to 39.2%, and conditional bypass rises from 20.1% to 35.8%. The paired interaction is −8.7 percentage points (95% CI, [−12.4, −5.1]), capturing the separation between the training-induced changes on original and counterfactual documents. OPD therefore improves accuracy on the original documents while increasing persistence of the original answer when the supplied evidence supports a different one.
The supervised baselines place this pattern in the context of different training signals. Answer-only SFT produces modest increases in original accuracy and bypass; answer-filtered CoT-SFT produces larger increases in both; and standard OPD exhibits the largest original-document gain and bypass increase among these baseline training conditions (first four rows of Table 1). The same direction appears on PeerQ with the 4B student: standard OPD raises original accuracy by 2.1 percentage points and lowers counterfactual accuracy by 2.6 points, yielding a paired interaction of −4.7 points. Across both datasets, the matched evaluations reveal a consistent shift toward retaining the original answer after an answer-changing edit.
| Method | Orig. ↑ | Para. ↑ | CFAcc ↑ | Bypass ↓ | Drop abst. ↑ | CSP ↑ |
| BASE | 49.0 | 48.7 | 44.5 | 20.1 | 22.0 | 69.8 |
| ANS-SFT | 50.2 | 49.9 | 43.8 | 23.7 | 19.2 | 67.5 |
| Answer-filtered CoT-SFT | 51.4 | 51.0 | 42.3 | 29.6 | 16.0 | 64.2 |
| Standard OPD | 52.4 | 51.9 | 39.2 | 35.8 | 14.4 | 57.0 |
| EP-OPD (ours) | 51.9 | 51.6 | 42.7 | 25.9 | 19.1 | 65.4 |
Table 1. QASPER with Qwen3-4B-Base (%). ↑/↓: higher/lower is better. Among the baseline training conditions, standard OPD attains the highest original and paraphrase accuracy while every evidence-sensitive outcome moves away from the base model. EP-OPD (last row; § 6) partially recovers these outcomes at a 0.5 pp original-accuracy cost.
Document Interventions Localize the Shift to Answer-Critical Evidence
The document controls connect this shift to the information needed to answer the question. On QASPER, accuracy on meaning-preserving paraphrases rises from 48.7% to 51.9%, closely tracking the original-document improvement. Irrelevant insertions likewise preserve most of the original-document gain. Removing the necessary evidence produces a larger change: correct abstention falls from 22.0% to 14.4%. The accuracy gain thus carries over to edits that preserve the answer, whereas abstention under evidence deletion and accuracy under answer-changing edits both decline.
Claim-level evaluation extends the analysis to the support for individual answer-relevant statements. Automatic evidence error rises from 30.2% to 43.0%, and error on the blinded expert subset rises from 28.5% to 38.3%. These changes accompany a 12.8-point reduction in claim support precision. Table 7 brings together the document-operator and claim-level outcomes, linking the change in final-answer behavior to weaker support for the claims used in the response.
| Outcome | Base | Standard OPD | OPD − Base |
| Original-document accuracy | 49.0 | 52.4 | +3.4 |
| Paraphrase accuracy | 48.7 | 51.9 | +3.2 |
| Paraphrase gap (Para. − Orig.) | −0.3 | −0.5 | −0.2 |
| Counterfactual accuracy | 44.5 | 39.2 | −5.3 |
| Evidence-drop abstention | 22.0 | 14.4 | −7.6 |
| Conditional bypass | 20.1 | 35.8 | +15.7 |
| Automatic step evidence error | 30.2 | 43.0 | +12.8 |
| Expert step evidence error | 28.5 | 38.3 | +9.8 |
Table 7. Document-operator and claim-level outcomes on QASPER–4B. Changes compare standard OPD with the frozen base. Scores are percentages, except the paraphrase gap and the change column, which are in pp. The expert row uses the blinded validation subset.
Bypass Increases with Pre-Distillation Answer Preference
The additional bypass induced by OPD grows monotonically across the resulting prior-strength quartiles, from approximately 5 percentage points in the weakest-prior quartile to 20 points in the strongest. Across the two datasets, the three-way interaction between OPD, counterfactual evaluation, and high prior strength lies between and points. The strongest change in evidence response thus occurs where the base student already favors the original answer.
The document interventions and prior stratification provide complementary views of evidence bypass: the former locate the shift in answer-critical evidence, and the latter relate its magnitude to an existing answer preference. We next examine how evidence response changes when individual OPD design choices are varied.
Evidence Response under OPD Design Variations
The preceding analysis concerns standard OPD. We now ask whether the separation between original accuracy and evidence response persists, and how it changes, when a single training choice is varied. Using the factorized design in § 3.3, we vary one dimension at a time on QASPER with Qwen3-4B. Table 2 reports the results, and Figure 1 gives an overview.

Figure 1. A coordinate view of how ordinary OPD choices affect document use. We vary supervision density, teacher target, divergence, and teacher privileged information one factor at a time around standard OPD (dashed contour). Each contour plots one metric’s change (pp): original accuracy and the counterfactual response, evidence adherence, and claim grounding of the three panels of Figure 3; outward is better. Teacher and density changes move the three evidence metrics almost identically, so those contours overlap. PI (†): teacher privileged information.
Teacher-target changes remain close to standard OPD: a slow EMA target reaches 54.8% original accuracy and 41.5% CFAcc, with 33.2% bypass and 59.1% CSP. A faster EMA target produces a smaller shift. Forward KL traces a different accuracy-response profile: moving from to raises CFAcc from 43.1% to 45.7% and reduces bypass from 28.8% to 24.5%, as original accuracy moves from 50.7% to 48.9%. Concentrating supervision on fewer high-entropy prefixes progressively lowers original accuracy.
Teacher privileged information produces the largest joint movement across the evidence-sensitive measures. Providing gold evidence IDs raises CFAcc to 47.8%, lowers bypass to 22.8%, and raises CSP to 68.2%. Providing the exact answer-critical span moves these quantities to 52.6%, 14.8%, and 76.0%, with original accuracy reaching 60.6%. Among the tested interventions, supplying answer-critical evidence to the teacher yields the largest joint improvement in CFAcc, bypass, and CSP.
Evidence-Paired OPD
When each question is paired with only one answer-supporting document during training, evidence-responsive and document-insensitive strategies may behave identically on the observed examples (Proposition 1). We therefore pair each question with documents supporting different answers, so that document-insensitive answer preferences cannot simultaneously satisfy both supervision targets.
Evidence-Paired OPD (EP-OPD) turns the matched-document intervention into a training signal. Each question appears with two validated documents that support distinct answers. The student receives on-policy teacher supervision under each document, together with an answer-level objective that couples its answer preferences to the evidence supplied in that document. Figure 2 shows the document intervention and the associated training outcomes.

Figure 2. Minimal evidence interventions identify bypass; paired evidence repairs it. One answer-critical edit flips the supported answer while the question and document structure stay fixed. Standard OPD sees only and may preserve under ; EP-OPD co-distills the pair and follows the edit. Values are Qwen–4B; deltas are relative to Base(top) and standard OPD(bottom).
Paired-document objective
For a validated pair, let and support answers . The student independently samples a response under each input. At every student-visited prefix, the frozen teacher conditions on the same question and document and supplies the next-token distribution used by the standard reverse-KL OPD objective.
Let denote the length-normalized log probability of canonical answer . We encourage each document to favor its supported answer over the answer associated with its partner:
where is the answer margin and . The full objective is
The OPD term transfers teacher behavior along student-generated trajectories, and the paired term couples answer preferences to document variation. When the pair loss is zero, each supported answer exceeds its paired alternative in score by at least under its own document. The two canonical answers consequently exchange their ranking across the pair. Appendix § 9.§ 3 formalizes this property, and Appendix § 8.§ 2 gives the token-level loss and answer-score definitions.
Algorithm 1 Evidence-Paired On-Policy Distillation
Require: Paired data , student , frozen teacher Require: Margin , weight , batch size , learning rate .
for each training iteration do
Sample , ; set .
for each do
for do
; sample .
for all .
.
for do
.
.
.
Update using at learning rate . Ensure: Updated student parameters .
Training procedure
We construct and validate document pairs before optimization and place both members of each pair in the same mini-batch. Each member receives an independent student rollout and the corresponding teacher supervision. The paired answer scores are computed by teacher forcing over the short canonical answer strings. The margin and pair weight are selected on the QASPER development set and transferred unchanged to the second student size and PeerQA.
At inference, the trained student receives a single question–document input and generates its response. Algorithm 1 summarizes the optimization procedure. Appendix § 8 specifies the hyperparameters and supervision-budget matching.
Evaluating Evidence-Paired OPD
We evaluate EP-OPD on held-out document pairs using the protocol in § 3. The main comparison uses the Qwen3-4B student from the phenomenon analysis; the cross-dataset and scale study additionally includes Qwen3-1.7B. Details appear in Appendix § 8.§ 3.
Answer accuracy and evidence response
On QASPER with Qwen3-4B (Table 1), EP-OPD achieves 51.9% original-document accuracy, preserving 85% of the gain obtained by standard OPD over the base student. The original-accuracy change relative to standard OPD is -0.5 percentage points, and paraphrase accuracy is 51.6%. Counterfactual accuracy increases by 3.5 points to 42.7%, recovering 66% of standard OPD’s counterfactual-accuracy loss. Conditional bypass falls from 35.8% to 25.9%, a reduction of 9.9 points.
The improvement extends to evidence availability and claim support. Correct abstention after evidence deletion rises by 4.7 points to 19.1%, and claim support precision rises by 8.4 points to 65.4%. Paired training therefore improves adaptation to answer-changing evidence, abstention when necessary evidence is missing, and support for answer-relevant claims while preserving most of the original-document accuracy gain.
Figure 3 places this response profile alongside the factorized OPD controls from § 4.4. Teacher targets, divergence choices, and supervision density yield different combinations of original accuracy and evidence response. The teacher-side privileged-information controls measure the effect of supplying annotated evidence directly to the teacher. EP-OPD introduces variation through matched training documents, with teacher and student sharing the document input for each member. Across the three panels, paired training improves counterfactual response, evidence adherence, and claim grounding relative to standard OPD.
Figure 3. One-axis OPD interventions trace different accuracy–evidence frontiers. Panels pair original accuracy with CFAcc, reversed bypass, or CSP (top-right is better). Arrows change one factor from standard OPD; dashed guides mark Base. PI (†) is an oracle and EP-OPD a symmetric-context reference.
Consistency across datasets and student sizes
Table 3 reports the standard-OPD change from the base student in each dataset–student setting and the fraction of its counterfactual-accuracy loss recovered by EP-OPD. On QASPER, paired training recovers 59% of the counterfactual loss for the 1.7B student and 66% for the 4B student. The corresponding recovery on PeerQA is 62% and 54%. Across the four settings, EP-OPD retains 80–85% of the original-document accuracy gain while recovering 54–66% of the counterfactual loss. The improvement in document response thus extends across both scientific QA datasets and both student sizes under the shared paired-training configuration.
| Dataset | Student | Orig. | CFAcc | CF recovery |
| QASPER | 1.7B | +2.6 | −3.4 | 59% |
| QASPER | 4B | +3.4 | −5.3 | 66% |
| PEERQA | 1.7B | +2.9 | −4.5 | 62% |
| PEERQA | 4B | +2.1 | −2.6 | 54% |
Table 3. Cross-dataset and scale replication. Entries are percentage-point changes of standard OPD relative to the frozen base, except the final column, which is the fraction of OPD’s CFAcc loss recovered by EP-OPD.
Evidence response across prior-strength strata
The frozen-base prior stratification connects the effect of pairing to the example-level pattern established in the phenomenon analysis. As shown in Appendix § 13.§ 2, EP-OPD lowers bypass in every quartile. In the weakest-prior quartile, bypass falls from 17% under standard OPD to 14%; in the strongest, it falls from 49% to 35%. Relative to the base student, these values correspond to additional bypass of 2 and 6 percentage points, compared with 5 and 20 points under standard OPD. Pairing thus improves evidence following throughout the prior distribution, with the largest absolute reduction in the strongest-prior quartile.
Component ablation. We compare three paired-document variants on QASPER with Qwen3-4B: Augmented OPD, Paired ANS-SFT, and Augmented OPD with answer cross-entropy. More details in Appendix § 12. Relative to Augmented OPD, EP-OPD improves CFAcc by 4.2 percentage points and CSP by 3.1 points, with a 0.2-point decrease in original accuracy. Replacing answer cross-entropy with the paired margin improves CFAcc and CSP by 1.8 and 5.7 points, respectively, with original accuracy changing from 52.5% to 51.9%. Paired ANS-SFT attains the lowest bypass rate (25.3%), while EP-OPD achieves higher original accuracy, CFAcc, and CSP.
Conclusion
We identify evidence bypass in document-grounded on-policy distillation: students improve their accuracy on original documents while increasingly retaining the original answer when the document supports a different one. Matched interventions on QASPER and PeerQA reveal that this shift tracks answer-critical evidence and grows with the student’s pre-distillation preference for the original answer. We decompose standard OPD along four dimensions—teacher target, teacher privileged information, divergence loss, and supervision density—and find that teacher-side oracle access to answer-critical evidence yields the largest joint improvement in evidence-sensitive outcomes.
Building on these findings, Evidence-Paired OPD trains on matched documents that support different answers to the same question, explicitly coupling answer preferences to document evidence. It improves counterfactual accuracy and claim support, reduces bypass, and preserves most of standard OPD’s original-document accuracy gain. Collectively, these results establish evidence response as a distinct dimension of distillation quality and motivate training objectives that strengthen both answer accuracy and responsiveness to answer-relevant evidence.
AI use statement
Generative AI tools were used solely for limited assistance with language polishing, formatting, routine coding, and debugging. The research, analysis, interpretation of results, and preparation of the final content were carried out by the human authors. All results and citations were independently verified by the authors, who take full responsibility for the work.
Ethics statement
Counterfactual documents are controlled research stimuli, not claims about the cited papers. They are clearly marked in released metadata and must not be indexed or redistributed as authentic scientific text. Human validation uses only document excerpts and model outputs; no sensitive personal data are collected. The method may improve evidence following, but it does not guarantee factuality when supplied evidence is itself false or adversarial, so counterfactual compliance should not be treated as a universal trustworthiness criterion.
Reproducibility statement
The paper specifies the document interventions, metrics, statistical unit, resampling protocol, datasets, models, and baselines (§ 3), the training objective and algorithm (§ 5), and the formal estimand (Appendix § 9.§ 1). The appendix provides proofs, construction validators, evaluation prompts, hyperparameters, and additional result tables. The source package includes a summary result manifest and uses shared macros for the main reported outcomes. An anonymous implementation and generated intervention manifests will accompany the submission.
| Setting | Value |
| Student checkpoints | Qwen3-1.7B-Base; Qwen3-4B-Base |
| Frozen teacher | Qwen3-30B-A3B-Instruct-2507 |
| Trainable parameters | all student parameters |
| Optimizer | AdamW, , weight decay 0.1 |
| Peak learning rate | (1.7B); (4B) |
| Schedule | 3% warmup, cosine decay, two epochs |
| Global batch | 128 documents (64 synchronized pairs for EP-OPD) |
| Maximum input / output | 32,768 / 768 tokens |
| Training rollout | temperature 0.8, top- |
| Primary evaluation | greedy decoding |
| Gradient clipping / precision | 1.0 / bfloat16 |
| OPD divergence | token-average reverse KL, full vocabulary |
| Pair margin / weight | 0.20 / 0.50 |
| Random seeds | 3 per principal configuration |
Table 4. Training and decoding configuration. Pair hyperparameters are selected once on QASPER development data and transferred unchanged.
Training and implementation details
Optimization
All methods use the same document serialization and answer delimiter. The frozen evidence-preserving pack keeps the retrieval channel fixed across methods, separating evidence use from retrieval-augmented generation ([27, 28]). OPD and EP-OPD compute the teacher distribution at every non-padding student token; no reward model or answer-only policy gradient is used. To match supervision tokens, we subsample longer off-policy traces and resample shorter answer-only targets. Training examples are shuffled at the paper level. Development selection uses original accuracy subject to a counterfactual-response constraint fixed before test evaluation; the test checkpoint is selected once per seed.
For EP-OPD, a batch contains the original and counterfactual member of each pair. The global batch size counts documents, so the number of pairs per update is half the number of standard-OPD documents. Both members receive independent on-policy rollouts. The margin score is teacher-forced only over the short canonical answer string and is length normalized as in Equation (2).
Evidence-Paired OPD details
Let and let the student sample . At each student-visited prefix , the frozen teacher provides a next-token distribution . The standard baseline uses length-normalized reverse KL,
For a validated pair and , the answer score used in the margin is
and the complete training objective is
with defined in Equation (2). The complete update is summarized in Algorithm 1 in the main text.
Compute matching
Teacher forward calls dominate OPD compute. Standard OPD therefore receives two independently sampled observational documents for each pair consumed by EP-OPD, equalizing teacher-scored tokens and optimizer steps. Reported method comparisons also match maximum input length, maximum generated length, decoding temperature, and checkpoint-selection schedule. Where EP-OPD is evaluated at both student sizes, the runs share data manifests and hyperparameters except for learning rate.
Factorized OPD control objectives
Every factor control starts from the same QASPER–4B training manifest and changes only the specified training choice. EMA targets are no-gradient moving teachers; if denotes the corresponding online teacher checkpoint after update , their parameters obey
with identical initialization and update ordering within the two EMA runs. The fixed-teacher row instead uses the external Qwen3-30B-A3B target throughout.
For privileged-context controls, the student’s context remains . The teacher receives , where weak PI makes the IDs of gold evidence paragraphs and strong PI makes the exact answer-critical evidence span. PI is never present in the student rollout or at evaluation. Because these rows consume information unavailable to the symmetric-context methods, they are oracle interventions in the learning-with-privileged-information sense [29] and estimate the value of resolving the teacher’s evidence-location uncertainty.
At a student-visited prefix , forward KL uses
where both distributions use temperature and the conventional factor compensates for temperature-dependent gradient scaling. Entropy masks are computed from the frozen pre-update student distribution for the complete rollout. Let select prefixes in the top entropy quantile. The masked loss is
Normalizing over the selected prefixes keeps the loss an average over supervised positions. The ablation therefore changes which positions receive supervision, rather than reducing the loss simply because fewer positions are selected. Rollouts and optimizer steps are held fixed; teacher-scored tokens intentionally fall with because supervision density is the manipulated variable.
Comparison and supporting results
Metric definitions and paired training comparison
Let denote answer correctness on document version . The primary answer-level metrics are
Original accuracy measures performance on the observed document, CFAcc measures adaptation to the answer-changing document, and bypass measures persistence of the original answer among examples answered correctly before the intervention.
For a training method , with base parameters and trained parameters , define
The paired quantity measures how the training-induced change differs between counterfactual and original documents. We report it together with the corresponding change in bypass to summarize how training changes document response.
Single-document ambiguity
Proposition 1 (Observational non-identifiability). Let be a training support in which each occurs with one answer-supporting document. If an evidence-responsive policy is correct on , then there exists a question-only policy with identical predictions and identical empirical answer or trace-imitation risk on . Consequently, observational supervision alone does not identify evidence response.
Proof. Let be any evidence-responsive policy that produces a conditional output distribution on each observed example. Because each observed is paired with a single , define a question-only policy on the empirical support by
Outside the observed questions, define arbitrarily. Then and induce identical distributions over answers on , hence identical empirical answer loss for every proper scoring rule. If their next-token distributions are matched at every prefix reached on , they also induce the same trace-imitation and on-policy token risk there. Nevertheless, is invariant to replacing with any , whereas need not be. The two policies therefore agree observationally but differ under document intervention.
The proposition characterizes the observed training support: one document per question leaves evidence-responsive and question-conditioned solutions observationally indistinguishable on that support.
Proposition 2 (Paired shortcut exclusion). Consider a balanced pair with fixed , distinct answers , and documents . Any stochastic question-only policy has average negative log-likelihood at least on the pair. Any deterministic question-only policy has pair accuracy at most .
Proof. Write and . Since , . The balanced pair negative log-likelihood is
By AM–GM, , so . A deterministic question-only policy returns the same answer for both documents, so it can match at most one of the two distinct labels.
For a balanced -way evidence set with distinct supported answers, the same argument gives NLL at least and deterministic accuracy at most .
What the paired margin guarantees locally
Corollary 3 (Local evidence-response certificate). If , then the paired response
satisfies . Every question-only policy has , so it cannot attain zero pair loss for .
Proof. Zero hinge loss in Equation (2) implies
Summing yields . For a question-only policy, for every , making the two brackets exact negatives and .
Corollary 4 (Paired answer-decision guarantee). Let . If and , then and . Consequently, candidate-set accuracy is one on both documents and pairwise bypass is zero.
Proof. Zero pair loss with implies and . The candidate-set argmax therefore selects the document-supported answer on both versions.
These statements characterize the two canonical alternatives used by the margin. The main evaluation extends the analysis to fresh document pairs under free generation.
A simple illustration of prior cancellation
Proposition 5 (Prior cancellation in a two-source model). Let index which of two answers a document supports, and let the logit for the positive answer be , where is constant across the pair and is the evidence contribution. For fixed , the balanced paired logistic risk has the unique minimizer . Observational risk on only depends on and therefore cannot identify their allocation.
Proof. For the fixed question, write . The balanced paired logistic risk is
For fixed , is strictly convex in , symmetric under , and has at ; hence its unique minimizer is . With only the positive document, the risk is , which is constant on every level set of and cannot identify how the logit is allocated.
Writing , original-answer persistence under the counterfactual document is . At fixed ,
Thus a larger positive prior increases persistence and also strengthens the paired gradient opposing it in this toy model. This relation motivates the pre-training prior stratification used in the main experiments.
Evidence intervention construction
Construction pipeline
Our minimal, label-changing revisions follow the local-control principle of counterfactually augmented data and contrast sets [13, 14]. For every answerable question with annotated evidence, we apply the following deterministic-first pipeline.
Evidence closure. Start from the annotated evidence spans and add coreferent or explicitly dependent sentences needed to interpret the target claim. This prevents an edit from leaving a contradictory duplicate elsewhere in the local context.
Atomic answer graph. Normalize the gold answer into typed slots (entity, scalar with unit, signed direction, Boolean polarity, comparison, or bounded set). Link each slot to the smallest evidence statement that determines it.
Type-preserving proposal. Apply an operator from Table 5. The proposed value is selected to remain locally plausible, preserve grammar and units, and differ from every answer alias in the original document.
| Operator | Example transformation | Required validation |
| Entity | Method A Method B | same entity class; no residual coreference |
| Numeric | 4.2 2.4 points | unit and precision preserved; table/text agree |
| Direction | improves degrades | predicate antonym valid in local syntax |
| Polarity | supported not supported | all scoped negation dependencies updated |
| Comparison | A B B A | metric and comparison set unchanged |
| Bounded set | {A,B} {A,C} | cardinality and entity types preserved |
Table 5. Counterfactual edit operators. All edits are restricted to the evidence closure and preserve answer type.
Dependent rewrite. Propagate the edit only through the evidence closure. Section headings, citations, surrounding methods, distractor paragraphs, and the question are frozen.
Automatic validation. Require answer change, type validity, lexical-diff locality, no remaining original-support span, no newly introduced contradiction, and agreement between the deterministic answer transformer and a document-only verifier.
Held-out validation. Review the complete evaluation pair without revealing training method or model outputs. Reject rather than repair ambiguous pairs after model evaluation begins.
Control documents
The four modified document versions distinguish sensitivity to answer-critical evidence from sensitivity to changes that should leave the answer unchanged.
: paraphrase only the answer-critical evidence while preserving every typed answer slot. This tests semantic invariance.
: insert paragraphs from another paper matched in length, section type, and broad topic but unrelated to . This tests distraction and context length.
: remove the full evidence closure and repair only dangling discourse markers. The correct behavior is to state that the document is insufficient.
: edit the answer-critical slots and all dependent mentions so that the document uniquely supports .
The context tokenizer length is matched between and up to the small unavoidable difference introduced by type-preserving replacements. For the conditional bypass computation, answer aliases are canonicalized before comparing a counterfactual output with .
Evaluation protocol
Deterministic answer scoring
Each primary item stores its answer type, canonical value, and allowed aliases. Text is Unicode-normalized, case-folded where appropriate, and stripped of citation markers before comparison. Numeric values are parsed with units and converted to the annotated unit; no tolerance is allowed unless the source itself reports rounding. Boolean, direction, and comparison answers use a fixed ontology. Set-valued answers require exact normalized set equality. Outputs that provide both the original and counterfactual answer without resolving the question are incorrect. This restricted atomic scorer is why headline Acc and CFAcc do not require an LLM judge.
Atomic claim segmentation
Claim-level evaluation asks whether a response is supported by its cited source, as in attribution and citation evaluation [15, 30, 16, 31]. Although the evaluation prompt requests one claim per step, we segment the output before assigning support labels. Coordinated propositions are split when either can be contradicted independently, and cited premises are separated from a derived conclusion. An equation, its substituted values, and its verbal interpretation are treated as separate units when each can fail independently; discourse markers and restatements are omitted.
Segmentation uses anonymized outputs without method, checkpoint, document-version, or answer-correctness metadata. Boundary cases are resolved with a fixed guide before support labels are revealed. This procedure reduces the extent to which differences in verbosity or sentence structure change the unit being evaluated. These labels assess source support, not whether the claims describe the model’s internal reasoning.
Structured evidence response prompt
You are answering a question using only the supplied scientific document.
For each reasoning step, return exactly:
CLAIM: <one atomic claim>
EVIDENCE: <paragraph IDs that support this claim, or NONE>
RELATION: direct | composed | uncertain
After the steps, return:
ANSWER: <answer, or INSUFFICIENT EVIDENCE>
Do not use facts that are absent from the document. If the necessary
evidence is missing, abstain.
All systems receive this response contract only at evaluation; it is not a method-specific prompt. Paragraph identifiers are assigned before document variants are generated so that unchanged paragraphs retain the same IDs.
Paired-document component ablation
Configurations. We examine the contributions of counterfactual document exposure, answer supervision, and the paired margin on Qasper with the Qwen3-4B student. Each paired-document configuration uses original and counterfactual documents supporting different answers to the same question. Let
denote the paired-document distillation term, and let denote cross-entropy supervision of each document’s supported canonical answer. The compared objectives are
where weights answer cross-entropy. Augmented OPD removes the answer-level objective from EP-OPD; Paired ANS-SFT uses answer supervision without distillation; and Augmented OPD + Answer CE replaces the paired margin with cross-entropy. Standard OPD provides the original-document reference.
Document exposure and the paired margin. Table 6 shows that adding counterfactual documents changes the evidence-response profile: Augmented OPD reduces bypass from 35.8% to 27.2% and increases CSP from 57.0% to 62.3%, while CFAcc changes from 39.2% to 38.5%. Lower original-answer persistence thus accompanies a different change in counterfactual correctness, highlighting the complementary information provided by these metrics. Adding the paired margin increases CFAcc from 38.5% to 42.7%, raises CSP from 62.3% to 65.4%, and further reduces bypass to 25.9%, with original accuracy changing from 52.1% to 51.9%.
| Method | Orig. | CFAcc | Bypass | CSP |
| Standard OPD | 52.4 | 39.2 | 35.8 | 57.0 |
| Augmented OPD | 52.1 | 38.5 | 27.2 | 62.3 |
| Paired ANS-SFT | 50.5 | 42.0 | 25.3 | 63.8 |
| Augmented OPD + Answer CE | 52.5 | 40.9 | 26.7 | 59.7 |
| EP-OPD | 51.9 | 42.7 | 25.9 | 65.4 |
Table 6. Paired-document component ablation on QASPER with Qwen3-4B. Scores are percentages. Standard OPD uses original documents; the remaining configurations use original and counterfactual documents. Bold denotes the best value in each column.
| Method | Lowest | Q2 | Q3 | Highest |
| BASE | 12 | 16 | 21 | 29 |
| Standard OPD | 17 (+5) | 24 (+8) | 34 (+13) | 49 (+20) |
| EP-OPD | 14 (+2) | 19 (+3) | 25 (+4) | 35 (+6) |
Table 8. Bypass rate (%) by frozen-base parametric prior quartile. Parentheses show the change from base.
Answer supervision and objective choice. Paired ANS-SFT achieves the lowest bypass rate, 25.3%. Relative to this configuration, EP-OPD raises original accuracy by 1.4 percentage points, CFAcc by 0.7 points, and CSP by 1.6 points, with a 0.6-point increase in bypass. Combining augmented OPD with answer cross-entropy yields the highest original accuracy, 52.5%. Compared with this objective, EP-OPD improves CFAcc by 1.8 points and CSP by 5.7 points, while reducing bypass by 0.8 points at a 0.6-point original-accuracy cost. These comparisons characterize the joint contribution of on-policy supervision and document-specific answer ranking: the complete objective attains the highest CFAcc and CSP in the ablation while retaining an original accuracy of 51.9%.
Additional results
Document-operator localization
Answer-preserving paraphrase and irrelevant-insertion controls retain most of the observed-document gain. Evidence deletion and answer-changing interventions produce the largest changes in evidence-sensitive outcomes, and the claim-level rows show the same source-support pattern.
Parametric-prior strata
The base model already shows greater original-answer persistence in the stronger-prior quartiles. Our main comparison concerns the additional increase after OPD, which grows from +5 to +20 pp across the quartiles. EP-OPD reduces most of this training-induced increase, while the differences already present in the base model remain.
Empirical Diagnostics
These comparisons bring together the main finding, its replication, and the effect of evidence pairing. The first four describe how answer accuracy and evidence response change after OPD, including their relationship to the student’s initial answer preference. The remaining comparisons assess whether EP-OPD improves evidence response and claim support while retaining the original-task gain. Together with the document controls in Table 7, they help assess alternatives such as general perturbation sensitivity or indiscriminate answer suppression. The direction column summarizes the qualitative relationships under examination; the observed column gives the reported magnitudes.
| Implication | Expected direction | Observed |
| 1. Observational competence rises | +3.4 pp (main); +2.1 to +3.4 pp across settings | |
| 2. Evidence response separates | −8.7 pp (main); pp | |
| 3. Original answer persists | +15.7 pp (main) | |
| 4. Reallocation replicates and increases with prior strength | Same signs across settings; larger bypass increment at higher prior | Three other settings; quartile increments +5 to +20 pp |
| 5. Pairing restores evidence response | 54–66% of CFAcc loss recovered | |
| 6. Repair preserves the original-task benefit | 80–85% of gain retained; −0.5 pp vs. OPD (main) | |
| 7. Claim support tracks reallocation and repair | Evidence error rises after OPD; CSP rises with repair | Expert error +9.8 pp; repair CSP +8.4 pp (main) |
Table 9. Implications and observed outcomes. Changes are relative to the frozen base unless a repair comparison is stated; main-setting results refer to QASPER–4B.
Illustrative intervention examples
Direction edit. “The proposed regularizer improves calibration” becomes “worsens calibration.” The question and experiment description are unchanged; the correct comparison reverses.
Numeric edit. “Ablation reduces F1 by 4.2 points” becomes “0.4 points.” Units, table position, and qualitative framing are preserved.
Evidence deletion. Sentences locating an out-of-domain split are removed while dataset names remain in unrelated paragraphs. A supported system must abstain.
These schematic examples use synthetic text for illustration. The manifest records each exact diff, answer transformation, evidence closure, and validation status.
References
- [1]Xi Ye and Greg Durrett. The unreliability of explanations in few-shot prompting for textual reasoning. In Advances in Neural Information Processing Systems, volume 35, pages 30378–30392, 2022.arxiv.org/abs/2205.03401
- [2]Miles Turpin, Julian Michael, Ethan Perez, and Samuel R. Bowman. Language models don’t always say what they think: Unfaithful explanations in chain-of-thought prompting. In Advances in Neural Information Processing Systems, volume 36, 2023.arxiv.org/abs/2305.04388
- [3]Yanda Chen, Joe Benton, Ansh Radhakrishnan, Jonathan Uesato, Carson Denison, John Schulman, Arushi Somani, Peter Hase, Misha Wagner, Fabien Roger, Vlad Miculik, Samuel R. Bowman, Jan Leike, Jared Kaplan, and Ethan Perez. Reasoning models don’t always say what they think. arXiv preprint arXiv:2505.05410, 2025. URL https://arxiv.org/abs/2505.05410.
- [4]Tamera Lanham, Anna Chen, Ansh Radhakrishnan, Benoit Steiner, Carson Denison, Danny Hernandez, Dustin Li, Esin Durmus, Evan Hubinger, Jackson Kernion, Kamile Lukosiute, Karina Nguyen, Newton Cheng, Nicholas Joseph, Nicholas Schiefer, Oliver Rausch, Robin Larson, Sam McCandlish, Sandipan Kundu, Saurav Kadavath, Shannon Yang, Thomas Henighan, Timothy Maxwell, Timothy Telleen-Lawton, Tristan Hume, Zac Hatfield-Dodds, Jared Kaplan, Jan Brauner, Samuel R. Bowman, and Ethan Perez. Measuring faithfulness in chain-of-thought reasoning. arXiv preprint arXiv:2307.13702, 2023. URL https://arxiv.org/abs/2307.13702.
- [5]Pepa Atanasova, Oana-Maria Camburu, Christina Lioma, Thomas Lukasiewicz, Jakob Grue Simonsen, and Isabelle Augenstein. Faithfulness tests for natural language explanations. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 283–294. Association for Computational Linguistics, 2023. doi: 10.18653/v1/2023.acl-short.25. URL https://aclanthology.org/2023.acl-short.25/.DOI
- [6]Cheng-Yu Hsieh, Chun-Liang Li, Chih-Kuan Yeh, Hootan Nakhost, Yasuhisa Fujii, Alex Ratner, Ranjay Krishna, Chen-Yu Lee, and Tomas Pfister. Distilling step-by-step! outperforming larger language models with less training data and smaller model sizes. In Findings of the Association for Computational Linguistics: ACL 2023, pages 8003–8017. Association for Computational Linguistics, 2023. doi: 10.18653/v1/2023.findings-acl.507.DOI
- [7]Namgyu Ho, Laura Schmid, and Se-Young Yun. Large language models are reasoning teachers. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 14852–14882. Association for Computational Linguistics, 2023. doi: 10.18653/v1/2023.acl-long.830.DOI
- [8]Lucie Charlotte Magister, Jonathan Mallinson, Jakub Adamek, Eric Malmi, and Aliaksei Severyn. Teaching small language models to reason. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 1773–1781. Association for Computational Linguistics, 2023. doi: 10.18653/v1/2023.acl-short.151.DOI
- [9]Rishabh Agarwal, Nino Vieillard, Yongchao Zhou, Piotr Stanczyk, Sabela Ramos Garea, Matthieu Geist, and Olivier Bachem. On-policy distillation of language models: Learning from self-generated mistakes. In International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=3zKtaqxLhW.arxiv.org/abs/2306.13649
- [10]Yuxian Gu, Li Dong, Furu Wei, and Minlie Huang. MiniLLM: Knowledge distillation of large language models. In International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=5h0qf7IBZz.DOI
- [11]Jay DeYoung, Sarthak Jain, Nazneen Fatema Rajani, Eric Lehman, Caiming Xiong, Richard Socher, and Byron C. Wallace. ERASER: A benchmark to evaluate rationalized NLP models. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4443–4458. Association for Computational Linguistics, 2020. doi: 10.18653/v1/2020.acl-main.408.DOI
- [12]Sarah Wiegreffe, Ana Marasović, and Noah A. Smith. Measuring association between labels and free-text rationales. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 10266–10284. Association for Computational Linguistics, 2021. doi: 10.18653/v1/2021.emnlp-main.804. URL https://aclanthology.org/2021.emnlp-main.804/.DOI
- [13]Divyansh Kaushik, Eduard Hovy, and Zachary C. Lipton. Learning the difference that makes a difference with counterfactually-augmented data. In International Conference on Learning Representations, 2020. URL https://openreview.net/forum?id=Sklgs0NFvr.
- [14]Matt Gardner, Yoav Artzi, Victoria Basmov, Jonathan Berant, Ben Bogin, Sihao Chen, Pradeep Dasigi, Dheeru Dua, Yanai Elazar, Ananth Gottumukkala, Nitish Gupta, Hannaneh Hajishirzi, Gabriel Ilharco, Daniel Khashabi, Kevin Lin, Jiangming Liu, Nelson F. Liu, Phoebe Mulcaire, Qiang Ning, Sameer Singh, Noah A. Smith, Sanjay Subramanian, Reut Tsarfaty, Eric Wallace, Ally Zhang, and Ben Zhou. Evaluating models’ local decision boundaries via contrast sets. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 1307–1323. Association for Computational Linguistics, 2020. doi: 10.18653/v1/2020.findings-emnlp.117.DOI
- [15]Hannah Rashkin, Vitaly Nikolaev, Matthew Lamm, Lora Aroyo, Michael Collins, Dipanjan Das, Slav Petrov, Gaurav Singh Tomar, Iulia Turc, and David Reitter. Measuring attribution in natural language generation models. Computational Linguistics, 49(4):777–840, 2023. doi: 10.1162/coli_a_00486.DOI
- [16]Tianyu Gao, Howard Yen, Jiatong Yu, and Danqi Chen. Enabling large language models to generate text with citations. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 6465–6488. Association for Computational Linguistics, 2023. doi: 10.18653/v1/2023.emnlp-main.398.DOI
- [17]Yaxuan Li, Yuxin Zuo, Bingxiang He, Jinqian Zhang, Chaojun Xiao, Cheng Qian, Tianyu Yu, Huan-ang Gao, Wenkai Yang, Zhiyuan Liu, and Ning Ding. Rethinking on-policy distillation of large language models: Phenomenology, mechanism, and recipe. arXiv preprint arXiv:2604.13016, 2026a.arxiv.org/abs/2604.13016
- [18]Peifeng Wang, Zhengyang Wang, Zheng Li, Yifan Gao, Bing Yin, and Xiang Ren. SCOTT: Self-consistent chain-of-thought distillation. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5546–5558. Association for Computational Linguistics, 2023. doi: 10.18653/v1/2023.acl-long.304.DOI
- [19]Shayne Longpre, Kartik Perisetla, Anthony Chen, Nikhil Ramesh, Chris DuBois, and Sameer Singh. Entity-based knowledge conflicts in question answering. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 7052–7063. Association for Computational Linguistics, 2021. doi: 10.18653/v1/2021.emnlp-main.565.DOI
- [20]Ella Neeman, Roee Aharoni, Or Honovich, Leshem Choshen, Idan Szpektor, and Omri Abend. DisentQΑ: Disentangling parametric and contextual knowledge with counterfactual question answering. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 10056–10070. Association for Computational Linguistics, 2023. doi: 10.18653/v1/2023.acl-long.559.DOI
- [21]Yifei Ming, Senthil Purushwalkam, Shrey Pandit, Zixuan Ke, Xuan-Phi Nguyen, Caiming Xiong, and Shafiq Joty. FaithEval: Can your language model say faithful to context, even if “the moon is made of marshmallows”? In International Conference on Learning Representations, 2025. URL https://openreview.net/forum?id=UeVx6L59fg.arxiv.org/abs/2410.03727
- [22]Baolong Bi, Shaohan Huang, Yiwei Wang, Tianchi Yang, Zihan Zhang, Haizhen Huang, Lingrui Mei, Junfeng Fang, Zehao Li, Furu Wei, Weiwei Deng, Feng Sun, Qi Zhang, and Shenghua Liu. Context-DPO: Aligning language models for context-faithfulness. In Findings of the Association for Computational Linguistics: ACL 2025, pages 10280–10300. Association for Computational Linguistics, 2025. doi: 10.18653/v1/2025.findings-acl.536.DOI
- [23]Alan Li, Yixin Liu, Arpan Sarkar, Doug Downey, and Arman Cohan. Demystifying scientific problem-solving in LLMs by probing knowledge and reasoning. In International Conference on Machine Learning, 2026b. URL https://arxiv.org/abs/2508.19202.
- [24]Pradeep Dasigi, Kyle Lo, Iz Beltagy, Arman Cohan, Noah A. Smith, and Matt Gardner. A dataset of information-seeking questions and answers anchored in research papers. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4599–4610. Association for Computational Linguistics, 2021. doi: 10.18653/v1/2021.naacl-main.365.DOI
- [25]Tim Baumgaertner, Ted Briscoe, and Iryna Gurevych. PeerQA: A scientific question answering dataset from peer reviews. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 508–544. Association for Computational Linguistics, 2025. doi: 10.18653/v1/2025.naacl-long.22.
- [26]An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Cheng'en Huang, Chenxu Lv, et al. Qwen3 technical report. arXiv preprint arXiv:2505.09388, 2025.arxiv.org/abs/2505.09388
- [27]Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. Retrieval-augmented generation for knowledge-intensive NLP tasks. In Advances in Neural Information Processing Systems, volume 33, pages 9459–9474, 2020.arxiv.org/abs/2005.11401
- [28]Akari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil, and Hannaneh Hajishirzi. Self-RAG: Learning to retrieve, generate, and critique through self-reflection. In International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=hSyW5go0v8.arxiv.org/abs/2310.11511
- [29]David Lopez-Paz, Léon Bottou, Bernhard Schölkopf, and Vladimir Vapnik. Unifying distillation and privileged information. In International Conference on Learning Representations, 2016. URL https://arxiv.org/abs/1511.03643.
- [30]Jacob Menick, Maja Trebacz, Vladimir Mikulik, John Aslanides, Francis Song, Martin Chadwick, Mia Glaese, Susannah Young, Lucy Campbell-Gillingham, Geoffrey Irving, and Nat McAleese. Teaching language models to support answers with verified quotes. arXiv preprint arXiv:2203.11147, 2022.arxiv.org/abs/2203.11147
- [31]Xiang Yue, Boshi Wang, Ziru Chen, Kai Zhang, Yu Su, and Huan Sun. Automatic evaluation of attribution by large language models. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 4615–4635. Association for Computational Linguistics, 2023. doi: 10.18653/v1/2023.findings-emnlp.307.DOI
