betaFeedback

WorksComputer Science

Consistency Training Suppresses Cue Following Without Teaching Cue Selectivity

Abstract

Language models often change answers because of a cue, a hint such as "I believe the answer is B" that gives no valid evidence. Consistency training (CT) trains a model to answer the same with and without the cue. Four CT methods apply this idea to output tokens (BCT), residual activations (ACT), attention patterns (AttCT) and sampled-answer rates (RMCT). Prior work suggests some learn different solutions. We train all four on Gemma-2-2B-it and Llama-3.1-8B-Instruct with the same data. We test predictions written in advance and inspect the models with a known circuit that carries cues to the answer. All four make the trained cue less effective, and their fixes share three properties. First, correct cues lose about as much influence as wrong ones. Second, the fix is usually large for cues before the answer options, as in training, and much smaller after them. Llama RMCT depends least on position. Third, the circuit heads still carry most remaining cue-induced errors. Making BCT targets depend on the cue's correctness mostly weakens the fix. CT thus weakens the trained cue without teaching models when a cue is right. Evaluations with only wrong cues at the trained position miss the first two properties.

Introduction

Language models often change their answer because of a cue in the prompt. By a cue we mean text that points to one answer option without giving valid evidence for it. Examples are the hint “I believe the answer is B”, an earlier answer in the conversation and few-shot examples labeled with the cued option. Models follow such cues (cue following) even when the cued answer is wrong, and they often do not mention the cue when they explain their answer ([26, 32, 3]).

Consistency training (CT) is a direct remedy. It trains the model to give the same answer with and without the cue. Four CT methods enforce this at different points in the model. Bias-augmented CT (BCT) matches the output tokens ([3]), activation CT (ACT) matches residual-stream activations ([15]) and attention CT (AttCT) matches per-head attention patterns ([4]). Rate-matching CT (RMCT) uses reinforcement learning so that the model gives the cued letter about as often with the cue as without it ([14]). Each method comes from its own paper, with its own data, models and tests, and no study has compared all four on equal terms. Two of these papers looked inside the trained models and concluded that BCT learns a different solution from the activation-level methods ([15, 4]).

We call the change that CT makes the fix. What does the fix actually teach the model? Cues are sometimes right. A good fix should therefore keep the model following a cue that names the correct answer while making it resist one that names a wrong answer. We call such a fix selective. A good fix should also hold when the cue moves to a new place in the prompt. The source papers test neither property, because their evaluations use only cues that name a wrong answer and never move the trained cue. We therefore ask three questions. Is the fix selective? Does it hold when the cue moves? Does CT remove the mechanism that carries the cue to the answer, or leave it in use?

We train all four methods with published code, on the same data and on 2 models. We wrote down each prediction before running its experiment (van Miltenburg et al., [33]). The original model is the instruction-tuned model before CT. For the third question we use the reader–writer circuit of Gupta et al. [10], a few attention heads that carry cues to the answer and already exist after pretraining. Its reader heads attend from the answer position to the cued letter, and its writer heads carry much of the cue’s effect on the answer. With it we can test whether the errors left after CT still pass through these heads, and in which layers the fix sits.

Figure 1. Consistency training weakens right and wrong cues alike, mostly before the answer options, and leaves the cue circuit in use. (a) Cue following on the same questions, cue before the options. (b) Harmful-cue following removed at each cue position. (c) Share of cue errors corrected by patching, which resets the circuit heads to uncued values. If CT had removed the circuit, the after-CT bar would drop toward the black lines, which show the best random heads (highest over models and methods). All bars and the lines in (b) average the two models. Bars in (a) and the after-CT bar in (c) also average the four methods. Whiskers show ranges, and Llama uses one seed in (c).

Findings. All four methods make the models follow the trained cue less (§ 4.1), and their fixes share three properties (Figure 1). (1) The fix is not selective (§ 4.2). A cue that names the correct answer loses about as much influence as one that names a wrong answer, even on the same questions. (2) The fix depends on where the cue sits (§ 4.3). It is usually large when the cue comes before the answer options, as in training, and smaller after them. Llama RMCT is affected least. (3) The circuit stays in use (§ 4.4). Its heads still carry most of the remaining cue-induced errors, and for ACT and AttCT the fix sits mainly in the layers before the circuit. Changing what BCT trains on does not make the fix selective either (§ 4.5). Targets that take the cue’s correctness into account mostly make the whole fix weaker. Training with cues at several positions, as in the BCT recipe of Chua et al. [3], changes where the fix works. These results suggest that CT teaches models to give the trained cue less weight without teaching them to judge whether the cue is right.

Implications. Every CT objective is minimized by ignoring the cue, so we expect correct cues to lose some weight (§ 4.2). Evaluations that use only wrong cues at the trained position show neither this loss nor the position effect. We therefore suggest that CT evaluations also test correct cues and new cue positions.

We also report two evaluation pitfalls that may affect other work. First, the answer read from the first output token disagreed with the generated answer on up to a fifth of prompts (cf. [35]). Second, merging LoRA adapters into bf16 weights changed the results of 48 of 51 checkpoints by more than a tolerance we set in advance (Appendix § 10).

Related work

Cue following and its mechanism. Language models repeat back users’ stated views ([22]), follow stated beliefs and other prompt cues against their own answers ([26, 36]), and often change answers without mentioning the cue ([32, 3]). Mechanistic studies locate this behavior in small sets of attention heads ([6, 29, 21]). Gupta et al. [10] report that the reader–writer circuit is already present after pretraining and that post-training changes when it is used and how strongly it acts. Gupta et al. [9] find that alignment tuning largely shapes decodable and steerable directions for such biases. We use the circuit only as a measurement tool, and to our knowledge no study has used it to examine a fix for cue following. The closest results show that alignment training “masks but does not remove” a related circuit ([21]) and that DPO bypasses the toxicity mechanism without removing it ([16]).

Comparisons of CT methods. Irpan et al. [15] compare training curves on one model and report evidence that BCT and ACT learn “mechanistically different solutions”. Gautam et al. [4] study one checkpoint per method and find that the activation-level methods share one linear pathway while BCT learns a different direction. Other work extends the family with on-policy CT (Han et al., [11]), CT against political manipulation (Phan et al., [23]) and ACT for reasoning models (Shah et al., [25]). Africa and Mani [1] compare BCT, ACT and other variants on model organisms that are already sycophantic and find that CT made them more sycophantic, significantly so for ACT. We run all four on the same data and add the self-training control of Chua et al. [3] (training without the cue).

Gaps in CT evaluations. The evaluations of the four methods use only cues that name a wrong answer, and they keep the trained cue where it was in training. Other anti-sycophancy methods have been tested on cues that name the truth. Training a model only to resist persuasion makes it worse at accepting useful persuasion ([28]). A benchmark measures whether deployed assistants accept valid corrections and reject wrong ones ([27]). Suppressing sycophancy can also make models worse at updating their answer for good reasons when a user pushes back ([20, 38]). Closest to our setting, Tang and Khanna [30] train two 32B reasoning models with a target equivalent to BCT’s and find that these models agree less with a user who suggests the correct answer. On position, the BCT recipe varies the cue position during training ([3]). The authors of ACT note that ACT’s loss covers only the tokens after the cue, which makes ACT weaker against jailbreaks that add tokens at the end ([15]). A BCT model trained against frustration helps against a persona attack when the facts come before a question but hurts when they come after it ([4], App. § 11.§ 6). Finally, benign fine-tuning can erode safety training ([24]). For consistency-style objectives, we know of only one test of such erosion, for a target that removes deception ([30]). We test correct cues, new cue positions and benign fine-tuning.

Setup

Data. We use Gemma-2-2B-it (Gemma; Gemma Team, [5]) and Llama-3.1-8B-Instruct (Llama; Grattafiori et al., [7]). The questions are multiple-choice questions from MMLU, HellaSwag, LogiQA and TruthfulQA ([12, 40, 18, 17]) in the cued form released by Chua et al. [3]. We ask for a direct letter answer (Figure 8 in Appendix § 9 shows an example). Training uses one cue type, a suggested answer (the trained cue), on 1,213 questions. ACT’s loss covers only the tokens after the cue. We therefore place every training cue before the question, so that this loss covers all of it. The data of Chua et al. instead vary the position. As in the BCT recipe, some training cues name the correct answer (239 of the 1,213). The test set has 1703 items, each pairing one question with one cue type, drawn from 465 questions that do not appear in training. It contains the trained cue and five held-out cue types. These are a planted earlier answer, a distractor fact, wrong few-shot examples, a distractor argument and spurious few-shot squares. It keeps the cue positions of the source data, which vary (Appendix § 8 describes the cues and gives full details).

Four cue positions in an example prompt

Figure 8. The four cue positions in an example prompt (a training question). Training places every cue at front. Here a harmful cue names B and a helpful cue names the correct answer C (options shown on one line).

Methods. We run all four methods with published code and train LoRA adapters ([13]) on the same prompts (Appendix § 8). BCT fine-tunes the model on cued prompts, with its own uncued answers as targets. It mixes these examples 1:1 with general instruction data. ACT pulls the residual activations of the cued prompt toward those of the original model on the uncued prompt. AttCT matches the per-head attention over the question between the cued and uncued prompts. RMCT rewards sampled answers so that the model gives the cued letter about as often with the cue as without it. Each method keeps its authors’ configuration, so the methods also differ in adapter placement and number of updates. We choose learning rates with the F1 score of Irpan et al. [15], which combines wrong-cue resistance with uncued accuracy and never rewards following correct cues.

Measures. We measure how often a cue moves the answer, separately for wrong and correct cues. Harmful-cue following (flip|knew in tables) starts from the items that the model answers correctly without a cue. It is the share of them on which a cue that names a wrong option makes the model give that option. Helpful-cue following starts from the items that the model answers wrongly without a cue. It is the share of them on which a cue that names the correct option makes the model answer correctly. The harmful drop and the helpful drop are how much each falls from the original model to the trained model, on the same items. On the test set, we parse greedy generations to a letter and report harmful-cue following for the trained cue and the mean of this rate over the five held-out cue types. For helpful-cue following and for position, we use a cue battery of 400 test questions with six cue wordings at four positions. These are before the question (front, the trained position), just before the answer options (pre), just after them (post) and after the answer instruction (end). Front and pre come before the options, and post and end after them. On the cue battery we read the first answer token, which we check against generation (Appendix § 10.§ 1). We count a fix as selective if its harmful drop is at least 0.05 and its helpful drop is at most half of it, the criterion we registered for the BCT variants on Llama. The relative reduction at a position is the share of harmful-cue following that a method removes there, 1−method/original1 - \text{method}/\text{original}.

Circuit instruments. We use the circuit in three ways (Appendix § 9 lists its heads). First, we measure how much the reader heads of Gemma in layers 13 and 14 attend to the cue. Second, we patch the circuit heads, replacing their activations with their values on the uncued prompt. We count how many of the remaining cue-induced errors this corrects and compare with random heads in the same layers. Third, we run a layer swap. We copy the attention weights of the trained model into the original model, one block at a time. There are four blocks, two before the circuit, one for the circuit and one after it. We then measure what share of the trained model’s fix each mixed model reproduces.

Predictions and statistics. Before each experiment, we wrote down our predictions and how we would score them. We kept them in a file that we only ever added to, so that these tests do not depend on choices made after seeing the results. Appendix § 7 reports all of them, including those that failed. Analyses that we did not plan this way are marked exploratory. We first saw on Gemma that the fix depends on the cue position, so we treat Llama as the fresh test of this pattern. Unless noted, results are means over three seeds, with 95% bootstrap intervals over test questions (Appendix § 8).

Results

All methods reduce following of the trained cue

We first check that each method does its job, which is to make the model resist a cue that names a wrong answer. Figure 2 shows harmful-cue following on the test set, for the trained cue and for five cue types not used in training. Every method reduces following of the trained cue on both models. On Gemma, BCT and RMCT remove three quarters or more of it, and ACT and AttCT remove about half. Our prediction that every method would at least halve it on Gemma fails narrowly. AttCT falls just short of half and ACT just passes it, and for both the threshold lies inside the error bars. On Llama, RMCT removes nearly all of it and BCT removes the least, although the interval for the gap between BCT and AttCT includes zero.

Figure 2. Every method cuts the trained cue, but transfer to new cue types is weaker. Whiskers show 95% intervals and dotted lines the original model (Table 6).

methodtrained-cue flip|knewΔ\Delta [95% CI]held-out (5 cues)Δ\Delta [95% CI]uncued accuracy (Δ\Delta)
Gemma-2-2B-it
original0.490 [0.39, 0.59]–0.546 [0.50, 0.59]–0.454 [0.39, 0.52]
BCT0.116 ± 0.024-0.37 [-0.48, -0.27]0.421 ± 0.018-0.12 [-0.17, -0.08]0.406 ± 0.045 (-0.05)
ACT0.241 ± 0.012-0.25 [-0.34, -0.16]0.479 ± 0.004-0.07 [-0.10, -0.04]0.468 ± 0.005 (+0.01)
AttCT0.257 ± 0.010-0.23 [-0.32, -0.15]0.486 ± 0.003-0.06 [-0.08, -0.03]0.444 ± 0.005 (-0.01)
RMCT0.091 ± 0.015-0.40 [-0.50, -0.30]0.282 ± 0.043-0.26 [-0.31, -0.22]0.458 ± 0.028 (+0.00)
Llama-3.1-8B-Instruct
original0.555 [0.46, 0.64]–0.421 [0.39, 0.46]–0.551 [0.49, 0.62]
BCT0.350 ± 0.050-0.20 [-0.29, -0.12]0.421 ± 0.018+0.00 [-0.04, 0.04]0.532 ± 0.009 (-0.02)
ACT0.236 ± 0.029-0.32 [-0.41, -0.23]0.326 ± 0.007-0.10 [-0.12, -0.07]0.548 ± 0.005 (+0.00)
AttCT0.274 ± 0.014-0.28 [-0.37, -0.19]0.348 ± 0.003-0.07 [-0.10, -0.05]0.540 ± 0.011 (-0.01)
RMCT0.042 ± 0.019-0.51 [-0.60, -0.42]0.188 ± 0.042-0.23 [-0.27, -0.20]0.590 ± 0.018 (+0.04)

Table 6. All four methods reduce following of the trained cue, and transfer depends on the method and the model. Test set, generated answers. Harmful-cue following (flip∣knewflip|knew) for the trained cue at its original positions and averaged over the five held-out cue types, and uncued accuracy. Seed mean ± SD. Δ\Delta is the method minus the original model, with its 95% question-cluster interval (no interval for accuracy). The cells of the original model show their own interval.

Transfer to the five held-out cue types is weaker. RMCT transfers best on both models. We had predicted that BCT would transfer best on Gemma, but it does not, and on Llama it barely changes following of held-out cues. On Gemma, much of RMCT’s advantage comes from a preference for one answer letter. It answers D on 0.54 of uncued questions, and a model that leans toward one letter can follow cues that name other letters less often. As a control, we shift the original model’s answer-letter frequencies to roughly match those of RMCT. Against this shifted model, RMCT’s drop on held-out cues is close to that of BCT (exploratory, Table 8). RMCT keeps a clear lead on Llama under this control. Training costs little uncued accuracy on the test set (at most 0.05 for every method), but Gemma BCT loses accuracy on the AttCT authors’ MMLU evaluation (seed 42, Appendix § 12).

methodtrained: method / controlbeyond priorheld-out: method / controlbeyond priormodal-letter share
Gemma-2-2B-it
BCT0.115 / 0.4390.324 ± 0.0250.423 / 0.5870.164 ± 0.0230.31 (D/C/C)
ACT0.238 / 0.4710.233 ± 0.0200.488 / 0.5970.109 ± 0.0030.27 (C/C/C)
AttCT0.251 / 0.4750.224 ± 0.0120.516 / 0.6030.086 ± 0.0080.27 (C/C/C)
RMCT0.098 / 0.3360.237 ± 0.0200.303 / 0.4920.189 ± 0.0120.54 (D/D/D)
RMCT+anchor†0.142 / 0.4620.320 ± 0.0530.392 / 0.5920.200 ± 0.0500.30 (A/C/A)
Llama-3.1-8B-Instruct
BCT0.350 / 0.5270.177 ± 0.0490.420 / 0.4240.003 ± 0.0160.30 (B/B/B)
ACT0.236 / 0.5160.280 ± 0.0370.349 / 0.4190.069 ± 0.0020.32 (B/B/B)
AttCT0.274 / 0.5390.265 ± 0.0170.366 / 0.4200.054 ± 0.0050.34 (B/B/B)
RMCT0.042 / 0.4540.412 ± 0.0130.200 / 0.3930.194 ± 0.0340.35 (D/D/D)

Table 8. Letter-prior control (exploratory, first-position readout, with the gate-failing Gemma checkpoints on the v7.1 readout). The table gives the flip∣knewflip|knew of the method and of the original model shifted to the uncued letter distribution of the method, on the trained cue and the held-out mean. It also gives the drop beyond the prior (control – method, seed mean ± SD) and the share of uncued answers on the most frequent letter of the method (seed mean, with the letter per seed). † marks an exploratory variant.

The fix is not selective

Does a model that resists wrong cues still follow correct ones? We test this in two ways.

First, we hold the question fixed, with the cue before the options (exploratory). On questions that the model answers wrongly without a cue, we give either a cue that names the correct answer or a cue that names another wrong option. Figure 3 shows how often the model gives the cued letter before and after training. Before CT, the model follows each cue on more than half of these questions. After CT, it follows both far less often. The green and red arrows are about equally long, so the two cues lose about the same influence. If anything, the correct cue loses more in absolute terms. Across the four methods, the correct cue’s drop minus the wrong cue’s drop ranges from −0.02 to +0.07 (Table 12).

Figure 3. A cue that names the correct answer loses its effect as much as a wrong one. Arrows run from the original model (circle) to the model after CT, on questions that both models answer wrongly without a cue (cue before the options). A selective fix would keep the green arrows short.

methodcorrect cuewrong cuedrop difference [95%]log-odds gapchange [95%]
Gemma-2-2B-it
BCT0.62→0.060.62 \to0.060.57→0.050.57 \to0.05+0.043+0.043 [-0.028, 0.118]+0.22→+0.16+0.22 \to+0.16[-0.66, 0.55]
ACT0.66→0.090.66 \to0.090.58→0.040.58 \to0.04+0.028+0.028 [-0.033, 0.091]+0.33→+0.82+0.33 \to+0.82[-0.02, 1.05]
AttCT0.66→0.120.66 \to0.120.59→0.080.59 \to0.08+0.035+0.035 [-0.025, 0.097]+0.33→+0.48+0.33 \to+0.48[-0.26, 0.56]
RMCT0.61→0.110.61 \to0.110.56→0.040.56 \to0.04−0.022-0.022 [-0.094, 0.054]+0.21→+1.15+0.21 \to+1.15[0.33, 1.72]
gold-gated BCT0.61→0.210.61 \to0.210.58→0.110.58 \to0.11−0.074-0.074 [-0.153, 0.013]+0.13→+0.85+0.13 \to+0.85[0.18, 1.29]
Llama-3.1-8B-Instruct
BCT0.63→0.240.63 \to0.240.57→0.170.57 \to0.17−0.010-0.010 [-0.097, 0.076]+0.27→+0.47+0.27 \to+0.47[-0.23, 0.64]
ACT0.66→0.180.66 \to0.180.57→0.150.57 \to0.15+0.057+0.057 [-0.009, 0.118]+0.39→+0.27+0.39 \to+0.27[-0.44, 0.20]
AttCT0.66→0.210.66 \to0.210.56→0.170.56 \to0.17+0.066+0.066 [0.001, 0.130]+0.43→+0.23+0.43 \to+0.23[-0.49, 0.10]
RMCT0.58→0.050.58 \to0.050.56→0.050.56 \to0.05+0.019+0.019 [-0.063, 0.099]+0.09→+0.07+0.09 \to+0.07[-0.85, 0.89]
gold-gated BCT0.63→0.370.63 \to0.370.55→0.250.55 \to0.25−0.032-0.032 [-0.118, 0.046]+0.34→+0.54+0.34 \to+0.54[-0.16, 0.58]

Table 12. Same questions, cue before the options. The table gives the following of a cue that names the correct answer and of a cue that names a wrong option on the same questions, original →\to trained (seed means). The questions are those that both the original and the trained model answer wrongly without a cue, and the option that the wrong cue names is neither model’s uncued answer. The last columns give the difference of the two drops and the change in their log-odds gap, with 95% question-bootstrap intervals. Exploratory.

Second, we compare the helpful and harmful drops pooled over positions (exploratory, Table 9). For a selective fix, the helpful drop would be at most half of the harmful drop. Instead, the helpful drop is 1.10–1.32 times the harmful drop on Gemma and 1.24–1.36 times on Llama. (Llama BCT has a ratio of 3.55 because it barely lowers harmful-cue following.) Other scales agree. Before the options, where the fix is strong, the ratio is at least 0.82 whether we measure absolute drops, proportional drops or log-odds shifts (Tables 10 and 11). Two cases differ. After the options, where the fix is weak, the helpful drop is mostly the smaller one in absolute and proportional terms. In the same-question test, Gemma RMCT keeps little cue following, and on the log-odds scale this remainder favors correct cues more than before (Appendix § 11.§ 2). Our prediction that every method lowers helpful-cue following in every seed holds on both models.

Why would CT suppress correct cues? Each CT objective is minimized by ignoring the cue, and the training cue names a wrong option in 80% of the training questions. Even when the cue names the correct answer, the BCT target (the model’s own uncued answer) often disagrees with the cue (on 144 of the 239 such questions for Gemma and 114 for Llama). What is new is the size of this cost, and that all four methods share it.

The fix depends on the cue position

Training places every cue before the question. Does the fix still hold when the cue appears elsewhere? We place the same cues at four positions and measure the share of harmful-cue following that each method removes (Figure 4, Table 9). The fix works well before the answer options and fades after them. Before the options, the reductions are large for every method, except for Llama BCT at the pre position. At the end of the prompt, ACT and AttCT remove less than a tenth of harmful-cue following on both models, and Llama BCT even makes it worse than in the original model.

Figure 4. The fix fades for cues placed after the answer options. Whiskers show 95% intervals. Training puts every cue at the front (shaded). On Gemma, the values of RMCT after the options come mostly from its preference for answering D (§ 4.3).

Llama RMCT is the exception, because its fix depends much less on the position. Its reduction at the front exceeds its reduction just after the options by only +0.227 (95% [0.193, 0.260]), so our prediction that the reduction of every method falls by at least 0.30 there fails. Gemma RMCT also looks less position-dependent, but this rests mostly on its preference for D (§ 4.1). A model that answers D whatever the cue says looks as if it resists every cue that names another letter. We therefore add a stricter count. Every wrong answer counts as following the cue, and we leave out the questions whose correct answer is the letter that the checkpoint most often gives without a cue. By this count, Gemma RMCT removes only 0.18 just after the options and 0.10 at the end, no more than BCT (0.36 and 0.14, exploratory, Appendix § 11.§ 3).

Are cues after the options simply stronger to begin with? To check, we compare only combinations of wording and position that the original model follows at similar rates before and after the options (exploratory, Table 13). ACT on Llama still removes 0.82 of harmful-cue following before the options and only 0.17 after them. Before the options, following falls about as much for novel wordings as for wordings from the training family. So position, more than strength or wording, seems to decide how well the fix works.

methodmatched: before / afterafter coef.end, non-correctfrontprepostend
Gemma-2-2B-it
BCT0.93 / 0.46-0.570.130.95 / 0.950.90 / 0.870.40 / 0.410.19 / 0.17
ACT0.86 / 0.25-0.660.080.92 / 0.920.80 / 0.780.11 / 0.150.06 / 0.12
AttCT0.83 / 0.18-0.720.020.90 / 0.930.79 / 0.780.04 / 0.100.00 / 0.05
RMCT0.91 / 0.65-0.300.280.92 / 0.930.91 / 0.930.57 / 0.510.41 / 0.35
RMCT+anchor†0.92 / 0.64-0.360.280.94 / 0.960.90 / 0.880.39 / 0.400.29 / 0.32
Llama-3.1-8B-Instruct
BCT0.58 / -0.51-1.11-0.220.66 / 0.770.34 / 0.57-0.17 / -0.13-0.24 / -0.17
ACT0.82 / 0.17-0.640.060.92 / 0.930.73 / 0.670.08 / 0.110.05 / 0.10
AttCT0.80 / 0.14-0.660.040.93 / 0.910.69 / 0.640.06 / 0.090.02 / 0.09
RMCT0.95 / 0.83-0.130.550.97 / 0.980.93 / 0.920.77 / 0.710.65 / 0.57

Table 13. Position checks (seed means, exploratory). Matched is the relative reduction of harmful-cue following in wording ×\times position cells whose original harmful-cue following lies in the range shared by cells before and after the options. After coef. is the after-the-options coefficient of a regression of relative reduction on original harmful-cue following and position over the 24 cells. End, non-correct is the end-position relative reduction when every answer other than the correct one counts as following the cue. The right part gives the relative reduction by position for the three training-family wordings and the three novel wordings. † marks an exploratory variant.

The test set shows the same pattern. Two thirds of its trained-cue items have the cue at a position other than the front (exploratory, Table 14). With the cue first, every method removes 0.80–1.00 of harmful-cue following, while after the options ACT and AttCT remove only 0.14–0.28. For BCT and RMCT the results vary more. Both the lack of selectivity and the dependence on position also hold at the neighboring learning rates. For Llama ACT and AttCT, they also hold on the AttCT authors’ own data (exploratory, seed 42, Table 15).

methodfront (trained)prepostfront − post [95%]
\multicolumn{3}{c}{relative reduction of trained-cue flip|knew}
Gemma-2-2B-it
original flip|knew (nn)0.303 (33)0.467 (30)0.686 (35)–
BCT0.820.980.60+0.22 [-0.077, 0.475]
ACT0.800.780.25+0.55 [0.204, 0.843]
AttCT0.830.820.14+0.69 [0.405, 0.947]
RMCT1.000.960.69+0.31 [0.172, 0.486]
RMCT+anchor†0.920.870.55+0.37 [0.151, 0.562]
Llama-3.1-8B-Instruct
original flip|knew (nn)0.405 (42)0.618 (34)0.651 (43)–
BCT0.820.49-0.03+0.98 [0.796, 1.215]
ACT0.830.770.28+0.55 [0.285, 0.774]
AttCT0.880.620.20+0.68 [0.430, 0.900]
RMCT0.960.950.88+0.08 [-0.038, 0.192]

Table 14. The trained cue of the test set (Table 6) split by the cue position of each source row (post hoc, exploratory). The table gives the original flip|knew per position with the number of known items in parentheses, the relative reduction of flip|knew per position (seed means), and the front−post gap with a 95% question-bootstrap interval (B = 2000, with the same resampled questions for the original model and every seed). † marks an exploratory variant.

checkpoint (seed 42)harmful drophelpful dropratio abs.prop.log-oddsrel. red. frontprepostend
Gemma
BCT, selected LR0.3820.4461.170.931.160.970.900.550.29
BCT, LR 3.16e-50.1270.2311.821.422.040.870.72-0.10-0.05
BCT, LR 3.16e-40.3500.4491.280.991.280.950.920.610.25
ACT, selected LR0.2490.3041.220.981.350.920.780.130.09
ACT, LR 3.16e-60.2100.2441.160.931.320.790.690.100.08
ACT, LR 3.16e-50.2360.2991.271.021.410.960.780.110.05
AttCT, selected LR0.2050.2731.331.071.500.890.740.050.02
AttCT, LR 5e-60.1890.2411.281.021.460.820.700.050.02
AttCT, LR 5e-50.2080.2771.331.071.500.960.680.070.02
RMCT, selected LR0.4110.4581.120.881.070.930.930.650.51
RMCT, LR 1e-50.4860.6081.250.961.190.960.940.730.62
Llama
BCT, selected LR0.1050.1851.761.131.880.790.580.02-0.05
BCT, LR 1.58e-50.0870.1071.230.791.390.390.380.070.08
BCT, LR 1.58e-40.0510.1202.341.572.610.830.62-0.20-0.29
ACT, selected LR0.1850.2461.330.891.370.930.690.090.06
ACT, LR 1.58e-60.1590.1761.110.741.180.720.500.110.10
ACT, LR 1.58e-50.1870.2471.320.881.360.970.750.080.06
AttCT, selected LR0.1690.2301.360.901.420.930.660.070.05
AttCT, LR 1.58e-60.1520.1721.140.751.220.770.520.090.08
AttCT, LR 1.58e-50.1770.2361.330.891.400.960.690.070.05
RMCT, selected LR0.3800.4671.230.840.960.980.930.730.59
RMCT, LR 3.16e-60.3300.5221.580.961.240.950.920.730.61
RMCT, LR 3.16e-50.3550.5161.450.831.050.950.960.810.57
Llama, authors’ recipe
BCT0.0850.0961.140.741.300.090.310.210.08
ACT0.2110.2921.380.931.370.970.810.140.11
AttCT0.1970.2721.380.941.390.950.730.120.08

Table 15. Robustness of the pattern to the learning rate and the training data (exploratory, seed 42). The table gives the paired drops on the cue battery, the helpful-to-harmful ratio on three scales and the relative reduction of harmful-cue following by position. It covers the selected learning rate, its non-degenerate neighbors, and the Llama checkpoints trained with the authors’ own data and configuration.

The circuit stays in use

Does CT remove the mechanism that carries the cue, or only weaken its effect? If CT had removed the circuit, the errors that remain after training would no longer pass through it. Figure 5a shows that they still do. For every method, patching the circuit heads corrects most of the remaining cue-induced errors (0.598–0.910), much as in the original model. Random heads in the same layers correct at most 0.121 (Table 16). This was our prediction for Llama, and it holds. Before and after CT, most of these errors are on held-out cues.

Figure 5. The circuit still carries the leftover errors, and the fix sits mainly before it. (a) Share of the remaining cue-induced errors that patching the circuit heads corrects. (b, c) We copy one block of attention layers from the trained model into the original model and show the share of the method’s fixes that this mixed model reproduces (our planned readout for Llama). The swap cannot test RMCT. Llama uses one seed.

methodcircuit [Wilson 95%] (recovered / flips)writersreaderscircuit, trained-cue flips (nn)best random set
Gemma-2-2B-it
original0.818 [0.780, 0.850] (390/477)0.710.260.82 (50)0.07
BCT0.843 [0.817, 0.866] (694/823)0.780.260.97 (30)0.06
ACT0.844 [0.821, 0.864] (933/1106)0.750.250.92 (72)0.06
AttCT0.861 [0.840, 0.880] (981/1139)0.750.270.88 (73)0.07
RMCT0.910 [0.885, 0.931] (559/614)0.870.430.90 (29)0.12
Llama-3.1-8B-Instruct
original0.650 [0.603, 0.693] (278/428)0.460.240.68 (66)0.07
BCT0.598 [0.548, 0.647] (222/371)0.520.160.85 (34)0.05
ACT0.678 [0.625, 0.727] (219/323)0.460.230.72 (32)0.07
AttCT0.686 [0.635, 0.734] (232/338)0.480.250.81 (31)0.05
RMCT0.808 [0.748, 0.857] (160/198)0.630.340.80 (5)0.07

Table 16. Patch recovery. Each cell is the share of a method’s remaining cue-induced flips that return to the correct answer when the uncued answer-position activations of a head set are patched in, pooled over seeds (Gemma three seeds, Llama seed 42). The head sets are the circuit (with a Wilson 95% interval), its writer and reader subsets (the Llama writer subset is L17.H24 alone), the circuit on trained-cue flips only, and the best random head set (the maximum over three layer-matched sets and, for Gemma, over seeds). The Gemma circuit column and the subset and trained-cue columns are exploratory.

If the circuit stays in use, where does the fix sit? The layer swap answers this (Figure 5b, Figure 5c, Table 18). For ACT and AttCT, the fix sits mainly in the layers before the circuit. Copying either block before the circuit reproduces 49%–73% of the trained model’s fixes on both models. The layers after the circuit reproduce at most 8%. Gemma BCT behaves the same way. For Llama BCT, however, the layers after the circuit reproduce a quarter of its fixes, so our prediction of under 15% fails, although 15% lies inside the interval. The swap copies only attention weights, and RMCT also trains MLP and output weights, so this test cannot locate its fix (Appendix § 11.§ 4).

methodfixed by trained modelEARLYCARRIERCIRCUITLATED1
Llama-3.1-8B-Instruct
BCT720.71 / 0.51 / 0.380.72 / 0.53 / 0.730.28 / 0.21 / 0.170.25 / 0.19 / 0.16fails (CARRIER)
ACT640.73 / 0.69 / 0.590.70 / 0.66 / 0.580.11 / 0.09 / 0.060.08 / 0.08 / 0.01holds (EARLY)
AttCT570.72 / 0.65 / 0.610.70 / 0.61 / 0.590.14 / 0.11 / 0.060.07 / 0.05 / 0.01holds (EARLY)
Gemma-2-2B-it
BCT95–1110.49 / 0.42 / 0.420.71 / 0.61 / 0.620.29 / 0.25 / 0.260.03 / 0.01 / 0.083/3 seeds
ACT56–620.49 / 0.46 / 0.420.51 / 0.51 / 0.480.38 / 0.35 / 0.280.02 / 0.02 / 0.013/3 seeds
AttCT47–490.60 / 0.59 / 0.500.60 / 0.58 / 0.580.18 / 0.16 / 0.100.00 / 0.00 / 0.013/3 seeds

Table 18. Layer swap. Each cell is the forward mediated fraction when one block of the trained model’s q, k, v and o projections is copied into the original model (Llama EARLY 0–8, CARRIER 9–14, CIRCUIT 15–18 and LATE 19–31, Gemma 0–6, 7–12, 13–18 and 19–25), on items whose cue original model follows. The three readouts are the pre-registered binary readout, the share of the trained model’s own fixes and the share of the fall in the probability of the cued letter. For Llama (seed 42), the table also gives D1 (the largest block is EARLY or CARRIER, and LATE is below 0.15). For Gemma, it gives seed means over three seeds, the range over seeds of the number of fixed items, and the number of seeds that meet the same criterion (exploratory). RMCT cannot be tested this way.

The attention of the Gemma reader heads fits this picture for ACT and AttCT (Table 17). On the items these methods fix, the reader heads attend about a fifth less to the trained cue. On the items where the cue still works, attention falls by 2% or less. Across all labeled trained-cue items (Appendix § 9), it falls by only 11–14% for ACT and AttCT, less than the 30% we predicted. So on the trained cue, ACT and AttCT may fix an item partly by reducing what reaches the reader heads, though this evidence is only correlational.

methodtrained cueheld-out cuesfixedstill followedfixed / gapnn fixed / stillOV depthcentroid
BCT-0.101 ± 0.153-0.21 +0.37 -0.26 -0.06 -0.2514%13%0.2737 / 1319.9 ± 0.4
ACT-0.139 ± 0.016-0.05 -0.05 -0.04 -0.12 -0.0920%2%0.2927 / 2316.9 ± 0.7
AttCT-0.109 ± 0.010+0.00 -0.02 -0.01 -0.05 -0.0120%1%0.2524 / 264.9 ± 0.3
RMCT-0.110 ± 0.046-0.08 -0.05 -0.04 -0.07 -0.1110%5%0.1940 / 1020.4 ± 0.1
RMCT+anchor†-0.128 ± 0.051+0.15 -0.07 +0.01 -0.02 -0.09––––7.6 ± 0.5

Table 17. Early reader heads of Gemma. The left part gives the relative change of their cue-span attention on the labeled items of the original model (seed mean ± SD, with held-out cues in the order post hoc, distractor fact, wrong few-shot, distractor argument and squares). The middle part (exploratory) uses the trained-cue items that the original model follows. It gives the relative drop on the items the method fixes (its cued answer is no longer the cued letter) and on the items it still follows (seed means), the fixed-item drop as a share of the gap of the original model between items it follows and items it resists, and the item counts per seed. The right part gives the OV depth centroid of the LoRA edit over the attention projections (layer index out of 26). † marks an exploratory variant.

Changing the training data

If CT is not selective because its targets ignore whether the cue is right, then targets that take the cue’s correctness into account should help. We test two such targets for BCT, and two changes to what BCT trains on (Figure 6, Table 19).

Figure 6. No change to the training data of BCT makes the fix selective. Cue battery, seed means. (a) shows each BCT variant and the self-training control on Gemma, and (b) compares BCT with position-diverse BCT, which trains with the cue at all four positions.

methodtrained-cueheld-outaccuracyhelpful dropharmful dropCAIrel. red. frontend
Gemma-2-2B-it
BCT0.1160.4210.4060.3840.3041.260.9510.180
response-only BCT0.0450.1930.4440.6520.5651.150.9800.852
Bayes-BCT (r=.5r=.5)0.1610.4600.4120.3080.2341.320.9350.111
Bayes-BCT (r=.8r=.8)0.2020.5040.3980.2760.1911.450.8660.077
gold-gated BCT0.1590.4520.4380.2430.2291.060.8420.037
position-diverse BCT0.0330.2920.3690.6680.5341.250.9020.932
self-training control0.2370.5740.3720.1690.1621.05––
Llama-3.1-8B-Instruct
BCT0.3500.4210.5320.1460.0413.55∗3.55^{*}0.720-0.207
Bayes-BCT (r=.8r=.8)0.4640.4440.4950.036-0.052undef.0.376-0.299
gold-gated BCT0.4120.4360.5050.0940.0195.06∗5.06^{*}0.426-0.127
position-diverse BCT0.2260.3450.5120.3250.2281.420.3830.554
self-training control0.5920.5450.545-0.087-0.186undef.––

Table 19. Changing the training data of BCT. Seed means for the BCT variants of § 4.5 (defined in Appendix § 8). Response-only BCT is run on Gemma only. CAI (content-awareness index) is the helpful drop divided by the harmful drop (* marks a harmful drop below .05, which the registered criterion does not count as selective). Rel. red. is the relative reduction of harmful-cue following at the front and end positions.

Bayes-BCT samples each target from the model’s uncued answer distribution after shifting it toward the cued letter, as if the cue were right with probability rr. Gold-gated BCT uses the correct label. It keeps a target that is right, corrects a wrong target when the cue is right, and drops the rest. Neither becomes selective. Figure 6a shows that both lose less helpful-cue following than BCT, but they also remove less harmful-cue following, so the two drops shrink together. With r=0.8r=0.8 on Gemma, the ratio of Bayes-BCT is no better than that of BCT (1.45 against 1.26), and gold-gated BCT stays well above one half (1.06). We had predicted that on Gemma each would keep at least half of the harmful drop of BCT with at most half of its helpful drop. This prediction fails for both Bayes-BCT (r=0.8r=0.8) and gold-gated BCT. On Llama, neither variant is selective, mostly because, pooled over positions, neither clearly lowers harmful-cue following.

The other two changes alter what BCT trains on. They help with position but leave the fix unselective. Position-diverse BCT places each training cue at one of the four positions, as the BCT recipe of Chua et al. [3] does. Its fix then also works at the end position on both models (Figure 6b), so both of our predictions for it hold. On Llama, however, it halves the fix when the cue comes first, and its helpful drop is the largest of the variants.

Response-only BCT (Irpan et al. [15]) masks the prompt from the loss, drops the instruction data, and its selected learning rate is ten times larger. It is the strongest Gemma method on held-out cues, and it depends much less on position even though every training cue comes first. It is still just as unselective (ratio 1.15), and its MMLU accuracy falls slightly below the 0.55 we predicted (Appendix § 12).

The self-training control of Chua et al. [3] measures how much of the effect of BCT comes from fine-tuning on the model’s own answers alone. It uses the same targets and instruction data without the cue. It increases cue following on Llama. On Gemma, it achieves two thirds of the reduction of BCT for the trained cue on the test set. On the cue battery it reaches about half of the harmful drop of BCT and just under half of its helpful drop (Table 19). So on Gemma, fine-tuning on its own answers explains much of what BCT does.

Relapse after benign fine-tuning

Finally, does the fix survive later benign fine-tuning, which can erode safety training ([24])? We fine-tune each trained model for 125 steps on benign instruction data (Figure 7, Table 20). Cue following rises for every method, but it also rises in the original model, so part of the apparent relapse seems to be a general effect of the fine-tuning. We planned to measure relapse as the share of each method’s improvement that is lost. By this measure, the Gemma methods lose 0.187–0.452 of it. Our prediction that every method loses at least a quarter fails because of BCT (0.187, 95% interval over seeds [−0.020, 0.394]). After we subtract the original model’s own increase, every Gemma method is within seed noise of zero. On Llama, ACT has the largest such value (0.162). It is positive in every seed, but its seed interval includes zero. On the log-odds scale, however, RMCT rises more than ACT (Appendix § 11.§ 6). On held-out cues, cue following rises more for some methods than for the original model (Table 20, without the few-shot correction of Appendix § 10.§ 2).

Figure 7. Benign fine-tuning raises cue following in every model, the original one included. Harmful-cue following on the trained cue, with bands for the SD over seeds (the original model is one run).

methodflip|knew, step 0 → 125relapse fractionseed 95% CIbeyond driftlog-odds risefractionbeyond drift
Gemma-2-2B-it; original model’s drift 0.490 → 0.589, log-odds +0.40 (held-out 0.596 → 0.668)
BCT0.116 → 0.185+0.187 ± 0.083[-0.020, 0.394]-0.079 ± 0.075+0.54 (1.4×)+0.140-0.280 ± 0.103
ACT0.241 → 0.330+0.355 ± 0.042[0.252, 0.458]-0.044 ± 0.059+0.44 (1.1×)+1.093+0.421 ± 0.058
AttCT0.257 → 0.363+0.452 ± 0.070[0.278, 0.626]+0.026 ± 0.088+0.50 (1.2×)+1.369+0.503 ± 0.046
RMCT0.091 → 0.199+0.269 ± 0.109[0.000, 0.539]+0.020 ± 0.106+0.89 (2.2×)+0.272+0.028 ± 0.076
RMCT+anchor†0.143 → 0.241+0.275 ± 0.096[0.037, 0.513]-0.013 ± 0.128+0.67 (1.7×)+0.638+0.284 ± 0.068
Llama-3.1-8B-Instruct; original model’s drift 0.555 → 0.699, log-odds +0.62 (held-out 0.428 → 0.605)
BCT0.350 → 0.492+0.666 ± 0.179[0.222, 1.111]-0.071 ± 0.362+0.59 (0.9×)––
ACT0.236 → 0.434+0.619 ± 0.035[0.531, 0.707]+0.162 ± 0.074+0.91 (1.5×)+2.544+0.293 ± 0.022
AttCT0.274 → 0.427+0.543 ± 0.055[0.406, 0.680]+0.026 ± 0.078+0.68 (1.1×)+2.792+0.073 ± 0.047
RMCT0.042 → 0.192+0.294 ± 0.041[0.192, 0.397]+0.012 ± 0.034+1.76 (2.8×)+1.090+0.308 ± 0.062

Table 20. Relapse under 125 steps of benign LoRA fine-tuning (generated answers, three seeds per method, ± is the seed SD). The relapse fraction is (method at step 125 − method at step 0)/(original at step 0 − method at step 0), and its seed 95% CI is a tt interval over seeds. Beyond original drift subtracts the change of the original model under the same fine-tuning. Log-odds rise is the change in the logit of trained-cue flip∣\midknew from step 0 to 125, with its ratio to that of the original model in parentheses. The drift-adjusted log-odds columns are exploratory. Held-out fractions are given only where the held-out gap of the method is at least .05. The held-out columns use the 16-token answers without the few-shot correction (as does the original held-out rate in the header), and the held-out fraction beyond drift subtracts the held-out change of the original model.

Discussion

Predictions. For our three main findings, the predictions that every method lowers helpful-cue following and that the circuit still carries the remaining errors both hold. Two others fail because of one method each. The fix of Llama RMCT fades after the options by less than we predicted. For Llama BCT, the layers after the circuit reproduce more of the fix than the 15% ceiling we predicted. The clearest failure concerns the Gemma reader heads. Their attention falls by much less than we predicted (§ 4.4), and for BCT on some held-out cues it changes more than we expected. In total, 14 of the 27 scored outcomes hold (Appendix § 7 lists all of them). Several pass or fail within the error bars, and one passes narrowly, since BCT lowers following of held-out cues on Llama by only 0.0005.

One pattern across four methods. Although the four methods act on different parts of the model, their fixes look alike. The account most consistent with our results is that the reader–writer circuit already exists after pretraining ([10]) and that all four methods turn down how strongly it moves the answer. ACT and AttnCT seem to do so partly by changing what reaches the reader heads. Gupta and Gupta [8] report a similar pattern for context grounding, where gains from SFT and DPO largely reuse machinery already in the starting model. The earlier evidence that BCT learns a different solution comes from training curves and linear directions. These do not measure this circuit,

so the two findings need not conflict. The methods differ in how strong the fix is, how far it transfers and, for Llama RMCT, how much it depends on position, but all treat right and wrong cues alike.

What CT evaluations should measure. Our rule for selecting learning rates, like the source evaluations, never rewards following a correct cue. We suggest that CT evaluations also test correct cues and cues at new positions, on the same questions as the wrong cues. They should also state whether “held-out” means new questions or new cue types (Appendix § 12). Objectives that do not reward ignoring a correct cue may keep correct cues effective. Wang et al. [34] train with reinforcement learning on cues that name the correct answer half the time, and reward correct answers under both kinds of cue. Their models keep performance when the cue is right. Preference optimization can teach a model when to accept persuasion (Stengel-Eskin et al., [28]), and in one benchmark, supervised training on the same targets does not (Yang and Yeung, [39]). Of the four methods, RMCT transfers furthest, partly through its better preference on Gemma. It also needs 256 samples per question. No method gives a model that weighs cues by their merit.

Conclusion

Trained on the same data, four consistency-training methods behave alike. Each makes the cue move the answer less whether the cue is right or wrong, and does so mostly when the cue comes before the answer options, as in training. None teaches a model when to trust a cue or removes the circuit that carries it. Supplementary code is available at: https://github.com/prakharg55/bias_ct_eval

Limitations

We note a few limitations.

(i) We study one setting on two models (2B and 8B) trained with LoRA. The models give direct letter answers to multiple-choice questions, and training uses one cue placed first (except in position-diverse BCT). BCT and RMCT were designed for chain-of-thought answers, and our RMCT runs use the authors’ algorithm but not their training setup (Appendix F).

(ii) Each method keeps its authors’ configuration, so the methods also differ in adapters, batch size and number of updates. Our runs also depart from the source recipes in ways listed in Appendix B. Rankings between methods may reflect these choices.

(iii) Learning rates are selected on one seed, and the Llama patching and layer-swap results also use one seed. The self-training control and the BCT variants other than response-only BCT reuse the learning rate of BCT. Several predictions pass or fail within the error bars, some analyses are exploratory and post hoc, and the attention comparison is correlational.

(iv) The benign fine-tuning data share a source file with the instruction data of BCT, and relapse beyond the original model’s own increase has wide intervals.

Ethical considerations

We use only public models, datasets and code, and we involve no human subjects. Fixes that make a model ignore wrong user cues can also make it ignore correct ones, so we suggest testing for this before deployment.

Use of AI assistants. The authors used generative AI tools for experiments, verification of results, and language editing. All experimental designs, analyses, and claims were reviewed and verified by the authors.

Acknowledgments

This material is based in part upon work supported by the German Federal Ministry of Education and Research (BMBF): Tübingen AI Center, FKZ: 01IS18039B; by the Machine Learning Cluster of Excellence, EXC number 2064/1 – Project number 390727645; by Coefficient Giving; by Schmidt Sciences; by the Cooperative AI Foundation; by the Canadian AI Safety Institute Research Program at CIFAR. Resources used in preparing this research project were provided, in part, by the Digital Research Alliance of Canada; the Province of Ontario; the Swiss AI Compute grants; the Government of Canada through CIFAR; and companies sponsoring the Vector Institute. We thank Jiarui Liu, Andrew Kim, and the other key developers of PaperMentor ([19]), which provides mentorship and valuable suggestions in polishing this paper after the first draft by the authors. We further acknowledge the crucial work of AdminBot for administrative support that streamlined our research workflow.

Pre-registration

We appended every prediction to an append-only file before running the experiments it concerns. The supplementary material contains a copy in which only the names of the compute cluster and of a queue are redacted. The timestamps are those written in the file, and we cannot certify them externally. We wrote each block of predictions after the results of earlier blocks were known. P1–P5 followed one-epoch pilot runs at the reference learning rates on seed 42. The 27 Sep block followed the Gemma results of P1–P5, including the position pattern. The 30 Sep block followed the results of the 27 Sep block, and S1–S2 followed the merged results of all other predictions. The earlier sections of the file record a retired design (our own re-implementations of the objectives), whose predictions we do not score. Analyses without a registered prediction are exploratory. They include the selectivity ratios of the four methods and their scales, the same-question test, the position and strength splits, the letter-prior control, the reading split by fixed items, the Gemma patching and layer swap, the head subsets, and the drift-adjusted and log-odds relapse.

Registered predictions

Table 1 quotes each scored prediction as written, with its registration time and outcome. We typeset inequalities and multiplication and keep internal names such as “gen3” and “W4c”, which the file defines. “Log 82.7” points to the project log, whose depth definition is the one in Appendix § 9. The Appendices § 8.§ 4 and § 10.§ 3 give the selection and merge reference values that some predictions quote (for example “BCT: 18%”) are those known when they were written, on the readouts then in use.

coderegistered (EDT)prediction, as writtenoutcome
P126 Sep, 23:47efficacy: every arm cuts trained-cue (suggested_answer test) flip|knew by ≥50%\ge50\% vs untrained.fails
P226 Sep, 23:47transfer: BCT has the lowest mean held-out caving (5 held-out cues) of the four arms.fails
P326 Sep, 23:47reading vs following (wave-3 W2 pattern): early-reader cue-span mass on the trained cue falls by ≥30%\ge30\% for ACT and AttCT in all seeds, while on every held-out cue each arm’s relative change stays within ±25%\pm25\% (seed mean).fails
P426 Sep, 23:47where edits land: per-head OV depth centroid (log 82.7 definition) is lower for AttCT and ACT than for RMCT in all seeds.holds
P526 Sep, 23:47content-blindness: every arm lowers paired helpful-cue uptake (gen3, W4c paired definition) in all seeds.holds
F127 Sep, 14:21Bayes-BCT(0.8): helpful-uptake drop ≤0.5×\le0.5\times BCT’s AND harmful-caving drop ≥0.5×\ge0.5\times BCT’s.fails
F227 Sep, 14:21Gold-gated BCT: the same two conditions.fails
F327 Sep, 14:21Position-diverse BCT: relative reduction of harmful caving at the END position ≥2×\ge2\times BCT’s (BCT: 18%).holds
R127 Sep, 14:21every CT method’s trained-cue relapse fraction at step 125 is ≥0.25\ge0.25 (seed mean).fails
L127 Sep, 14:21every arm lowers trained-cue flip|knew and held-out caving vs untrained (seed means).holds
L227 Sep, 14:21content-blind: every arm’s paired helpful-uptake drop >0>0 in every seed.holds
L327 Sep, 14:21position-bound: for every arm, the relative reduction of harmful caving at post AND at end is ≥30\ge30 points below that at front (seed means).fails
L427 Sep, 14:21circuit intact: for every arm (seed 42), circuit patch recovery ≥0.8×\ge0.8\times the untrained model’s and ≥3×\ge3\times the best random set’s.holds
L527 Sep, 14:21RMCT (anchor 0) is degenerate (Amendment 2) at its published LR 2.86e-4.fails
LF130 Sep, 08:50Bayes-BCT(0.8) is NOT content-aware, and its paired harmful drop is ≤\le Llama BCT’s (.038) (its targets follow wrong hints on 25% of known items, so it trains cue-following).holds
LF230 Sep, 08:50Gold-gated BCT is NOT content-aware (as on gemma, where helpful and harmful drops were equal).holds
LF330 Sep, 08:50Position-diverse BCT: relative reduction of harmful caving at the END position ≥.50\ge.50 (Llama BCT: −.22-.22) AND helpful uptake at the END position falls by ≥.30\ge.30 absolute vs untrained (.858) – the muting generalises, content-blind.holds
LR130 Sep, 08:50ACT and AttCT: trained-cue relapse fraction at step 125 ≥.25\ge.25 (seed means).holds
LR230 Sep, 08:50BCT’s trained-cue relapse fraction is below ACT’s and below AttCT’s.fails
B130 Sep, 08:50content-blind: paired helpful-uptake drop >0>0 in every seed.holds
B230 Sep, 08:50position-bound: relative reduction of harmful caving at front minus that at end ≥30\ge30 points (seed mean).fails
B330 Sep, 08:50less accuracy cost than BCT: MMLU clean accuracy on the AttCT evaluator ≥.755\ge.755 (BCT .512, untrained .596), seed 42.fails
D130 Sep, 08:50for each of BCT, ACT, AttCT the single-scope weight splice with the largest forward mediated fraction is EARLY or CARRIER (below the circuit), and LATE mediates <.15<.15.fails
S11 Oct, 19:11the self-training control’s cut in trained-cue flip|knew vs untrained is less than half of BCT’s cut, on each model.fails / holds
S21 Oct, 19:11its paired helpful-uptake drop (gen battery, W4c paired definition) is less than half of BCT’s, on each model.holds / holds

Table 1. Every scored prediction as registered (2026). P1–P5, F1–F3 and R1 concern Gemma, and B1–B3 concern response-only BCT on Gemma. L1–L5, LF1–LF3, LR1–LR2 and D1 concern Llama. S1–S2 outcomes are given as Gemma/Llama. The 30 Sep 12:07 erratum withdrew the composition numbers of the 30 Sep block, including the 25% in LF1, and the Llama Bayes-BCT data for LF1 were rebuilt on the corrected readout. The P1–P5 section header in the file reads 27 Sep, and an erratum appended one minute later corrects it to 26 Sep. Outcomes are on the corrected, unmerged results.

Amendments

Table 2 lists every later amendment, erratum and correction note in order. Most notes state what had or had not been run when they were written. We appended rule changes before computing in full the outputs they apply to, with exceptions that the notes record. Amendments 1 and 2 were written after the base-grid validation results they apply to. The Llama readout notes (12:07, 12:53) followed its seed-42 repeated test rates, and the 12:53 note also followed the v6 validation results of the Llama sweep. The 23:14 note followed a subsample re-score that had shown changes of at most .02. The 05:13 note accepts an observed discrepancy. P1 and P2 also fail on the superseded summed readout (05:13 note), and the unmerged re-run replaces the other values in the record that use that readout. Appendices B.4 and D.3 give the selection and merge reports that the notes promise. Several smaller side reports that the notes promise are not included, for example the Gemma test rates and cue-battery values on the superseded summed readout and the first-position relapse fractions. Re-scoring the cue battery changed no verdict.

appended (EDT)amendment, erratum or note
27 Sep, 00:27LR rule, Amendment 1 (the header says 00:45, corrected by an erratum). The grid-end test uses the best F1, extensions are sequential and at most two, and an untied published LR yields to the best-F1 LR. RMCT execution v2 uses shared-prefix sampling and a one-epoch run in resumable segments (we verified that the LoRA weights and the Adam step are identical across boundaries).
27 Sep, 02:07Exploratory RMCT+anchor variant declared (anchor weight 0.5, the trainer’s default), after the pre-registered RMCT runs had collapsed to a constant answer at every LR evaluated so far. It is not part of P1–P5.
27 Sep, 02:31LR rule, Amendment 2. A checkpoint that gives one letter on at least 95% of validation prompts is degenerate and excluded. If every grid point is degenerate, the grid extends downward.
30 Sep, 09:01Erratum to D1. RMCT cannot be tested by the layer swap (its adapter trains MLP and unembedding weights), so it is dropped from that analysis.
30 Sep, 11:04Llama relapse runs use micro-batches of 2 because of memory. The loss and gradient are unchanged up to bf16 numerics.
30 Sep, 12:07–13:27Llama readout correction, in four notes. The first-token readout must include the “(L” tokens, with a readout gate of at least .97 agreement with greedy decoding (v6, 12:07, with gate coverage and handling at 12:24). Because v6 fails for Llama BCT, the readout becomes the maximum over letter forms (v7), and Llama LR selection, test rates and relapse move to generated answers (12:53). Exact bf16 ties are broken as greedy decoding breaks them (v7.1, 13:27).
30 Sep, 20:33–1 Oct, 05:13Readout gates, in six notes. A cue-battery gate is declared (20:33). It fails for Gemma BCT, so every Gemma cue-battery checkpoint is re-scored on v7.1 (23:14). The test-set gate is extended to all Gemma checkpoints (31 at the time), and failing checkpoints use generated answers, and v7.1 for their forward-pass instruments including patch recovery (01:45, 01:52). At the first failure (BCT seed 44), its letter-prior control and patch recovery are recomputed against an original-model reference on the same readout (04:13). A regression exception records that one item differs for one checkpoint because of an exact bf16 tie (05:13).
1 Oct, 06:18 and 07:56Gemma relapse re-scored on generated answers, because the first-token readout fails its gate after benign fine-tuning. The exploratory RMCT-anchor relapse is re-run on the same basis.
1 Oct, 11:19Pre-write-up audit. Three checks are declared with their rules, namely adapter-merge fidelity, few-shot cues scored at the test slot, and Gemma LR selection on generated answers.
1 Oct, 19:11Self-training control declared, with S1–S2.
1 Oct, 19:43 and 20:11Merge-fidelity outcome. All seven checkpoints checked by then fail the declared rule (48 of 51 in the end, Appendix D.3). Every adapter result, including the layer swap, is re-run unmerged. The RMCT check in the paper’s regime stays on the authors’ merged pipeline.
1 Oct, 21:26Outcome of the unmerged LR re-selection, recorded before any re-selected test result was read. The selections for Gemma ACT and AttCT change, and every other selection stands.
9 Oct, 00:12 and 00:28Exploratory analyses declared. The cue battery at the neighboring learning rates of every sweep, and on the Llama checkpoints trained with the authors’ recipe (Table 15).

Table 2. Amendments, errata and correction notes after the first scored predictions, in the order appended (2026). Notes on one topic appended within hours of each other share a row. The full texts are in the supplementary pre-registration file.

Verdicts with intervals

Table 3 gives the deciding value and interval for the verdicts that turn on a single estimate, and the other verdicts have no computed interval. Several verdicts are decided inside their intervals. These are P1 (ACT, AttCT), L1 (BCT, held-out), R1, LR2, B3, D1 (BCT, ACT), and S1 and S2 on Gemma (B3 in Table 21). LF3 holds narrowly on both of its conditions. L1 (BCT, held-out) failed under the superseded readout.

newline and the uncued prompt), where the loss of ACT over the longest shared token suffix covers the whole question. The cues use 51 templates of the suggested-answer family. In 6 rows an extraction error cuts the first letter of the cue and adds a line with a stray “A”. 239 of the cues name the correct answer, and we keep them, as in the BCT and AttCT recipes.

The BCT targets are each model’s own temperature-1 answers to the uncued prompts, produced by the generator of the AttCT authors (Gemma 45% correct, Llama 53%), with the Gemma targets re-rendered in its chat template. As in Chua et al. [3], we mix them 1:1 with the rows of the instruction file of the AttCT repository, whose responses come from other models (gold-gated BCT keeps all 1,213 instruction rows).

The validation set is rows 500–600 of the trained cue (LogiQA 500–549). It has 265 items, 11 of which share questions with the test set (distractor argument, LogiQA). The test set is rows 1200–1300 of six cue types, or the last 100 rows where a dump has fewer than 1,250 usable rows (LogiQA and TruthfulQA, whose dumps are shorter, and the distractor argument, whose dumps lack many cued prompts). Besides the trained cue, these are a planted earlier answer (an earlier assistant turn

Experimental details

Data

The shared training set is rows 100–500 of the suggested-answer cue in the test dumps that Chua et al. [3] release for MMLU, HellaSwag, LogiQA and TruthfulQA. The dumps are formatted for chain-of-thought answers and put the cue before the question, before the options, after the options or after the answer instruction. Our loader replaces the reasoning instruction with a direct-answer one, which also removes the cues placed after that instruction, so we drop the pairs whose cued and uncued prompts become identical (24% of rows). This leaves 1,213 questions. We move every training cue before the question (the cue, a

verdictdeciding quantitythresholdvalue95% intervalstatus
P1 (BCT)trained flipknew − 0.5× original (≤ 0 holds)0.00-0.129[-0.196, -0.062]resolved
P1 (ACT)trained flipknew − 0.5× original (≤ 0 holds)0.00-0.004[-0.077, 0.069]unresolved
P1 (AttCT)trained flipknew − 0.5× original (≤ 0 holds)0.00+0.012[-0.058, 0.084]unresolved
P1 (RMCT)trained flipknew − 0.5× original (≤ 0 holds)0.00-0.153[-0.211, -0.093]resolved
P2BCT held-out − lowest other method (< 0 holds)0.00+0.139[0.094, 0.185]resolved
L1 (BCT, trained)trained method − original (< 0 holds)0.00-0.204[-0.293, -0.115]resolved
L1 (BCT, held)held method − original (< 0 holds)0.00-0.0005[-0.040, 0.039]unresolved
L1 (ACT, trained)trained method − original (< 0 holds)0.00-0.318[-0.407, -0.232]resolved
L1 (ACT, held)held method − original (< 0 holds)0.00-0.095[-0.123, -0.068]resolved
L1 (AttCT, trained)trained method − original (< 0 holds)0.00-0.280[-0.368, -0.193]resolved
L1 (AttCT, held)held method − original (< 0 holds)0.00-0.073[-0.100, -0.046]resolved
L1 (RMCT, trained)trained method − original (< 0 holds)0.00-0.513[-0.601, -0.424]resolved
L1 (RMCT, held)held method − original (< 0 holds)0.00-0.233[-0.267, -0.200]resolved
R1 (BCT)trained-cue relapse fraction, generated (≥ .25 holds)0.25+0.187[0.008, 0.361]unresolved
R1 (ACT)trained-cue relapse fraction, generated (≥ .25 holds)0.25+0.355[0.027, 0.731]unresolved
R1 (AttCT)trained-cue relapse fraction, generated (≥ .25 holds)0.25+0.452[0.075, 0.912]unresolved
R1 (RMCT)trained-cue relapse fraction, generated (≥ .25 holds)0.25+0.269[0.123, 0.440]unresolved
LR1 (ACT)trained-cue relapse fraction, generated (≥ .25 holds)0.25+0.619[0.389, 0.948]resolved
LR1 (AttCT)trained-cue relapse fraction, generated (≥ .25 holds)0.25+0.543[0.291, 0.888]resolved
LR2 (BCT vs ACT)BCT fraction − method fraction (< 0 holds)0.00+0.047[-0.406, 0.500]unresolved
LR2 (BCT vs AttCT)BCT fraction − method fraction (< 0 holds)0.00+0.124[-0.342, 0.589]unresolved
D1 (BCT, LATE < .15)LATE mediated fraction (< .15 holds)0.15+0.250[0.148, 0.364]unresolved
D1 (ACT, LATE < .15)LATE mediated fraction (< .15 holds)0.15+0.078[0.016, 0.152]unresolved
D1 (AttCT, LATE < .15)LATE mediated fraction (< .15 holds)0.15+0.070[0.016, 0.146]resolved
S1 (Gemma)control’s cut − 0.5× BCT’s cut in trained-cue flipknew (< 0 holds)0.00+0.066[-0.010, 0.141]unresolved
S2 (Gemma)control’s helpful drop − 0.5× BCT’s (< 0 holds)0.00-0.023[-0.175, 0.130]unresolved
S1 (Llama)control’s cut − 0.5× BCT’s cut in trained-cue flipknew (< 0 holds)0.00-0.139[-0.231, -0.048]resolved
S2 (Llama)control’s helpful drop − 0.5× BCT’s (< 0 holds)0.00-0.160[-0.249, -0.070]resolved
L3 (BCT, front − post)relative reduction front minus post (≥ .30 holds)0.30+0.872[0.797, 0.946]resolved
L3 (BCT, front − end)relative reduction front minus end (≥ .30 holds)0.30+0.927[0.845, 1.013]resolved
L3 (ACT, front − post)relative reduction front minus post (≥ .30 holds)0.30+0.830[0.789, 0.867]resolved
L3 (ACT, front − end)relative reduction front minus end (≥ .30 holds)0.30+0.854[0.812, 0.893]resolved
L3 (AttCT, front − post)relative reduction front minus post (≥ .30 holds)0.30+0.846[0.804, 0.885]resolved
L3 (AttCT, front − end)relative reduction front minus end (≥ .30 holds)0.30+0.866[0.826, 0.904]resolved
L3 (RMCT, front − post)relative reduction front minus post (≥ .30 holds)0.30+0.227[0.193, 0.260]resolved
L3 (RMCT, front − end)relative reduction front minus end (≥ .30 holds)0.30+0.358[0.321, 0.395]resolved

caption: Pre-registered verdicts that turn on a single estimate, with the deciding quantity, threshold, value and 95% interval. The intervals come from a bootstrap over questions for test-set, cue-battery and layer-swap rates, a binomial bootstrap for relapse fractions (Table 20 gives the seed tt interval) and a seed-level tt interval for LR2 and S1–S2. A verdict is unresolved when the interval contains the threshold.

:::

that gives the cued letter), a distractor fact about the cued option, wrong few-shot examples (the test question appears as an example with the cued label), a distractor argument, and spurious few-shot squares (a square marks the labeled answer in each example and the cued option in the test question). It shares no questions with training and has 1703 items (216 for the trained cue) over 465 questions, with the cues at the original positions of the dumps. Our planted-answer (“post hoc”) cue differs from that of Chua et al., because the loader drops their sentence “Don’t anchor on your previous response”. Our cue is therefore a planted earlier commitment.

Training

BCT, ACT and AttCT run through the AttCT repository, unmodified at commit b3c1896, with copies of its configurations. For ACT and AttCT, a dataloader shim feeds the shared prompts. RMCT runs the authors’ trainer verbatim at commit 4392d15 of their fork of cot-transparency. We replace their remote training service [31] with a local implementation of its interface. Around the trainer, we supply our own prompts and chat templates (for Llama the Hugging Face template, which adds a short system block that the authors’ renderer omits). We add a bare-letter fallback to the answer parser, because theirs reads almost no bare-letter answers, and we run the epoch in six segments through the trainer’s resume path. Every method trains a rank-8 LoRA adapter for one epoch.

  • BCT adapts q, k, v and o (α=16\alpha=16). Its loss is cross-entropy over prompt and response tokens, as in the GPT-3.5 runs of the BCT paper, so most loss tokens are question text. It uses batch 2 with 8-step accumulation.

  • ACT adapts q and v (on Gemma inherited from the base configuration of the repository, whose Gemma-2-2B ACT configuration is labeled full fine-tuning). Its loss is the squared residual distance to the uncued activations of the frozen initial model, over the longest shared token suffix and all layers, with weight 10−410^{-4} and batch 1.

  • AttCT adapts q and v. Its loss is the Jensen-

Shannon divergence between per-head attention rows on the uncued content span, over all layers, with batch 1.

  • RMCT adapts attention, MLP and unembedding weights (the default modules of the trainer), with 128 cued and 128 uncued rollouts per question. The reward is r=−(p^−pref)(y−p^)r = -(\hat{p} - p_{\mathrm{ref}})(y - \hat{p}), where y=1y = 1 if a rollout gives the cued letter and p^\hat{p} and prefp_{\mathrm{ref}} are the cued and uncued rates, and advantages are standardized over the parsed cued rollouts of each batch. Unparsed rollouts drop out of the rates, the gradient and the KL term. The loss is the PPO-clipped loss of the trainer (GRPO in the terms of the paper), with KL coefficient 0.05, 16 generated tokens and 4 questions per batch. The trainer skips batches with zero advantage, so the 304 batches give 294 to 303 updates in the final RMCT runs (287 to 297 for RMCT+anchor). The LoRA scale is 4 (α=32\alpha=32), as the Tinker cookbook documents. Tinker does not expose α\alpha, and the RMCT paper states α=16\alpha=16 on attention and MLP.

The ACT and AttCT dataloaders truncate 5–7 prompts longer than 512 tokens. The exploratory RMCT+anchor variant adds the optional anchor reward of the trainer at its code default weight of 0.5, with 128 anchor rollouts, which the RMCT paper does not use. We declared it after every pre-registered Gemma RMCT learning rate evaluated by then had collapsed to a constant answer.

Our runs also depart from the source recipes in other ways. Irpan et al. [15] fine-tune all self-attention parameters with batches of 4 or 8. The AttCT paper states q, k, v, o LoRA targets in its method section and 8-step accumulation in its ablations, unlike the repository configuration we run. The selected BCT learning rates are 100 and 10 times those of the BCT configurations in the repository (1e-6 for the Gemma-3-4B configuration we copy and 5e-6 for Llama). Library versions differ from the authors’ lock file (Appendix § 13).

BCT variants

All variants use the BCT pipeline. Bayes-BCT samples each target from p0(a∣q)L(h∣a)p_0(a \mid q)L(h \mid a), where p0(a∣q)p_0(a \mid q) is the original model’s uncued answer distribution and L(h=X∣a)=rL(h=X \mid a) = r if a=Xa=X and (1−r)/(K−1)(1-r)/(K-1) otherwise, for KK options (an unparsed target is kept). Gold-gated BCT keeps the model’s own targets when they are correct, corrects wrong targets when the cue is right and drops the rest.

Position-diverse BCT places each training cue at a random one of the four positions. Response-only BCT masks the prompt from the loss and drops the instruction mix (Gemma only). The self-training control trains on the same targets and instruction mix with the cue removed. Response-only BCT has its own learning-rate sweep, and the other variants and the control use the rate selected for BCT on the same model.

Learning-rate selection

We sweep learning rates on seed 42 and select them by the F1 score of Irpan et al. [15], the harmonic mean of uncued accuracy and the rate of resisting a wrong cue on validation items. The pre-registered rule has three parts. A best F1 at a grid end extends the grid by one half-decade, at most twice (once for response-only BCT, as its protocol specifies). A tie within 0.005 goes to the reference rate if that rate is among the tied points, and otherwise to the best F1. A checkpoint that gives one letter on at least 95% of uncued validation prompts is degenerate and excluded. We apply this last rule to generated answers on Gemma, with unparsed answers as a category, and to the first-position readout on Llama.

The reference rates (called “published” in the prediction texts) are those of the authors’ configurations. On Gemma, they are 1e-6 for BCT (the Gemma-3-4B configuration we copy) and for ACT (the repository’s Gemma-2-2B configuration, chosen there by the same F1), and 5e-6 for AttCT (the default its configurations inherit). Every Llama method except RMCT also has 5e-6. For RMCT, the reference rate is 2.86e-4, the middle of the three rates its paper pools. Irpan et al. [15] publish no rates.

We first selected rates on merged checkpoints (with the first-position readout on Gemma) and re-applied the same rule to the unmerged sweep after the merge correction (Table 4). Eight of the ten selections are unchanged. Gemma ACT moves from 3.16e-6 to 1e-5 and AttCT from 5e-6 to 1.58e-5, and all reported results for these two methods use the re-selected rates. At the superseded rates, their unmerged test rates are 0.260 (trained cue), 0.451 (held-out) and 0.457 (accuracy) for ACT, and 0.290, 0.485 and 0.448 for AttCT. The draft of the tie rule before Amendment 1 would have kept 3.16e-6 for Gemma ACT, which lies within the 0.005 tie band of 1e-5.

modelmethodstoredre-selectedchangedvalidation F1 by LR (unmerged, generated)
GemmaBCT1e-41e-4no1e-6: 0.468 3.16e-6: 0.469 1e-5: 0.449 3.16e-5: 0.509 1e-4: 0.569 3.16e-4: 0.523
GemmaACT3.16e-61e-5yes1e-6: 0.532 3.16e-6: 0.559 1e-5: 0.563 3.16e-5: 0.557 1e-4: 0.559
GemmaAttCT5e-61.58e-5yes5e-6: 0.548 1.58e-5: 0.553 5e-5: 0.539 1.58e-4: 0.537 5e-4: 0.534
GemmaRMCT3.16e-63.16e-6no3.16e-6: 0.574 1e-5: 0.524 3.16e-5: 0.353{sup}d 1e-4: 0.339{sup}d 2.86e-4: 0.339{sup}d 5e-4: 0.339{sup}d 1e-3: 0.000{sup}d`
GemmaRMCT+anchor{sup}†`3.16e-53.16e-5no1e-5: 0.573 3.16e-5: 0.575 1e-4: 0.022{sup}d 2.86e-4: 0.008{sup}d
Gemmaresponse-only BCT1e-31e-3no1e-5: 0.508 3.16e-5: 0.583 1e-4: 0.583 3.16e-4: 0.598 1e-3: 0.627
LlamaBCT5e-55e-5no1.58e-6: 0.531 5e-6: 0.523 1.58e-5: 0.590 5e-5: 0.613 1.58e-4: 0.572
LlamaACT5e-65e-6no1.58e-6: 0.609 5e-6: 0.628 1.58e-5: 0.621 5e-5: 0.618 1.58e-4: 0.611
LlamaAttCT5e-65e-6no1.58e-6: 0.610 5e-6: 0.626 1.58e-5: 0.619 5e-5: 0.616 1.58e-4: 0.607
LlamaRMCT1e-51e-5no3.16e-6: 0.668 1e-5: 0.735 3.16e-5: 0.665 1e-4: 0.339{sup}d` 2.86e-4: 0.015

Table 4. Stored and re-selected learning rates, with the validation F1 at every grid point (unmerged adapters, generated answers). dd marks a degenerate point (excluded), and † an exploratory variant.

Several selections are weak. The F1 curves of ACT and AttCT vary by about .03 or less across their grids on both models, so these selections are weak preferences. The OV depth centroid of ACT is 10.0 at the superseded rate and 16.9 at the re-selected one (AttCT 5.6 and 4.9). Gemma RMCT is selected at the grid edge after both permitted extensions. On Llama, RMCT at 2.86e-4 stops producing the answer format in our 16-token direct-answer setting (its parse rate is near zero), which the one-letter rule cannot detect (L5).

Scoring and statistics

Answers are greedy 16-token generations parsed to a letter, and an unparsed answer counts as no answer. The two few-shot cues are regenerated with 128 tokens and scored at the test slot (Appendix § 10.§ 2). We evaluate all adapters unmerged, as in training, except in the RMCT check in the paper’s regime. An item is known if the uncued answer is correct. Harmful-cue following (flip|knew) is the share of known items whose cued answer is the cued letter. The held-out rate is the unweighted mean over the five held-out cue types, and we measure uncued accuracy on the trained-cue items.

Test-set intervals are percentile intervals of a paired bootstrap that resamples whole questions within each dataset (B = 10000, and 2000 in the exploratory tables). The original model and every method and seed use the same resample. Relapse fractions use seed tt intervals and a binomial bootstrap, and patch recovery uses Wilson intervals ([37]). Test-set contrasts between methods combine the paired item SE with a seed SE. An independent script re-derives every point estimate in the primary tables from the per-item files.

Instruments

Cue battery

The cue battery has 400 uncued suggested-answer questions (100 per dataset, rows 1200–1300, or the last 100 rows of shorter dumps), and it shares no questions with training. Each question gets six cue wordings (three from the training family and three novel) at the four positions of Figure 8, and each cue names a wrong option XX or the correct option TT. We read the answer at the first position with the v7.1 readout, which Appendix § 10.§ 1 defines and validates against generation. Helpful-cue following is P(answer TT | wrong uncued answer, cue names TT), and harmful-cue following is P(answer XX | correct uncued answer, cue names XX), both averaged over wordings. For the paired drops, helpful-cue following uses the items that both the original model and the checkpoint answer wrongly without a cue, and harmful-cue following uses the items that both answer correctly. The ratio, or content-awareness index (CAI), is the helpful drop divided by the harmful drop, computed where the harmful drop exceeds .02 (Table 19 gives it wherever the harmful drop is positive). A method is content-aware (selective) if its harmful drop is at least .05 and its ratio is at most .5. The relative reduction at a position is 1−1- method/original harmful-cue following at that position, per seed.

Circuit patching

We fixed the circuit heads before any training run reported here. On Gemma they are the readers L18.H6, L14.H2, L13.H6 and L13.H2 and the writers L17.H3, L16.H4 and L15.H7, all in the layer band of the circuit (13–18). Gupta et al. [10] name the first three readers and the writer hub L17.H3, and we added the other three from head-level patching and attention to the cued letter in the original model in that band. On Llama they are the writer hub L17.H24, L18.H28 (the first of the example readers named there) and L15.H8. In that work, L15.H8 and L17.H24 are the two top heads in every leave-one-cue-out fold. That work leaves the role of L15.H8 open. Our Llama subsets group it with L18.H28 as “readers” and leave L17.H24 alone as “writers”. We patch the remaining cue-induced flips of each model, which are the items it answers correctly without the cue and with the cued letter when cued. In the cued run, we replace the answer-position activations of the heads with their values from the uncued run. Recovery is the share of flips whose answer returns to TT. Three random head sets matched to the layers of the circuit (the same for every checkpoint) give the baseline.

Layer swap

The qq, kk, vv and oo projections of the trained model (adapter unmerged) replace those of the original model in one block of layers. The blocks are EARLY 0–6, CARRIER 7–12, CIRCUIT 13–18 and LATE 19–25 on Gemma and 0–8, 9–14, 15–18 and 19–31 on Llama. We use items that the original model answers correctly without the cue and with the cued letter when cued, and that the trained model answers correctly without the cue (at most 10 per cue and dataset). Let hh be harmful-cue following on these items. The forward mediated fraction (1−hspliced)/(1−htrained)(1-h_{\mathrm{spliced}})/(1-h_{\mathrm{trained}}) approximates the share of the trained model’s fixes that the mixed model reproduces. The fixed items need not coincide, so Table 18 also gives the item-level share. It is defined when the trained model fixes at least 20 items. Fractions for single blocks are not additive, because two blocks can each reproduce most of a fix. RMCT cannot be tested this way, because its adapter also trains the MLPs and the unembedding.

Attention reading and weight-edit depth

For every head, we take the attention mass from the final position to the cue span, which is the set of tokens inserted or changed in the cued prompt. The early readers of the Gemma circuit are L13.H6, L13.H2, L14.H2. The relative change is mass/original mass −1-1 on the labeled items of the original model, which are items it knows and on which it either followed the cue or strictly resisted it. For the weight-edit depth, we compute for each attention head the relative change of the OV and QK composites under the LoRA edit (attention projections only, which are qq and vv for ACT and AttCT). The depth centroid is the layer index weighted by the part of these changes above the median.

Letter-prior control

We fit a per-letter additive logit offset to the uncued log-probabilities of the original model, so that its shares of uncued answer letters approximately match those of the method. We score this shifted model with the same flip|knew definitions, with the argmax restricted to the true options of each item. The drop beyond the prior is the control minus the method.

Relapse fine-tuning

Starting from each final checkpoint, with its CT adapter frozen, we train a fresh LoRA (rank 16, α=32\alpha=32, on qq, kk, vv, oo, gate, up and down) with AdamW (10−410^{-4}), batch 16 and 125 steps. The data are 2,000 seeded examples from the AttCT instruction file, disjoint from the training mix of BCT, with the loss on assistant tokens. We re-run the test set with generated answers at step 125. At step 0 the first-position readout must reproduce the evaluation of the method exactly, and the generated answers are those of that evaluation (the original model is re-generated at step 0). The relapse fraction is (method at 125 − method at 0) / (original at 0 − method at 0), and the drift-adjusted fraction subtracts the change of the original model itself from the numerator.

Evaluation corrections

We made three corrections during the study. We declared each in the record before computing the corrected results in full, with the exceptions listed in Appendix § 7. Every reported number uses the corrected procedures, with two exceptions. The held-out relapse columns and the merge compari son use the 16-token answers before the few-shot correction, and the RMCT check in the authors’ regime scores merged weights.

First-token readout and validity gates

Early instruments read the answer as the letter with the largest summed probability over its single-token forms (“L” and “ L”) at the first generated position. Llama often answers “(B)”, and its tokenizer has single tokens “(L”. Ignoring these tokens, the readout disagreed with generation on 6.9% (original), 19.7% (BCT) and 11.6% (ACT) of Llama test prompts, and an apparent rise of the held-out harmful-cue following of BCT on Llama was this artifact. Adding the “(L” forms was not enough for BCT, because the letter with the largest summed probability is not always the letter of the most probable token, which is what greedy decoding writes. On Gemma, the correction moves the trained-cue and held-out rates of the original model from .500 and .602 to .490 and .596.

All test rates therefore use generated answers. The forward-pass instruments of Llama and the cue battery of both models use the maximum over letter-form logits with greedy tie-breaking (“v7.1”), and the other forward-pass instruments of Gemma keep the summed readout. A gate requires each checkpoint to agree with generation on at least .97 of the test prompts. All 24 Llama checkpoints pass (agreement at least 0.99). The 13 cue-battery gates also pass (minimum over positions at least 0.990). We ran these on every 4th item with all cued prompts, for the original models, the seed-42 checkpoints of BCT, ACT, AttCT, RMCT and position-diverse BCT on both models, and response-only BCT on Gemma. We did not gate the cue-battery checkpoints of seeds 43 and 44 or of the remaining variants. Of the 33 Gemma checkpoints, 8 fail on the summed readout (unmerged, lowest agreement 0.956, BCT (seed 44); response-only BCT (seeds 42, 43); Bayes-BCT with r=.5r=.5 (seed 44); Bayes-BCT with r=.8r=.8 (seed 44); gold-gated BCT (seeds 43, 44); position-diverse BCT (seed 43)). Their letter-prior control and patch recovery use the v7.1 readout, against an original-model reference on the same readout. After benign fine-tuning the first-token readout also fails, so we score relapse on generated answers.

Few-shot cues at the test slot

On the wrong few-shot and spurious few-shot squares cues, both models often answer every demonstration in turn, so a parser that reads the first line returns the answer to the first demonstration. We regenerated these prompts with 128 tokens (checking that the first 16 tokens equal the stored ones) and scored them at the test slot by a declared rule. This lowers the held-out mean of the original Gemma from 0.596 to 0.546 and moves the held-out mean of every trained method by at most 0.034. Counting a list output as no answer (U) or dropping items whose list has no test slot (CC) bounds the effect. The bounds are widest for the squares cue on Llama, where list outputs are common (original U 0.065, CC 0.539). The held-out difference of Llama BCT from the original model is near zero under the reported rule and takes either sign across the bounds (U +0.044, CC −0.068). The forward-pass instruments read the first position and cannot apply this rule, so we recomputed them without the two few-shot cues. Drops beyond the letter prior change by at most 0.07 and patch recoveries by at most 0.07 (Llama BCT from 0.598 to 0.669, against 0.650 to 0.628 for the original model), and no conclusion in § 4.4 changes.

Merged versus unmerged adapters

Early instruments merged each LoRA adapter into the bf16 base weights, while training and the authors’ evaluator use it unmerged. Before inspecting any result, we declared that a merged result stands only if the generated test answers of the merged and unmerged checkpoints agree on at least .99 of the 3,406 test prompts and each of the three headline rates differs by at most .02. Only 3 of 51 checkpoints meet this rule (agreement 0.924–0.991, Table 5). Most rate differences are small, but merging weakens RMCT on both models (its merged trained-cue and held-out rates are higher), consistent with its small, spread-out update being rounded away in bf16. On merged weights, the Llama layer swap gives the same D1 verdicts (LATE block BCT 0.25, ACT 0.05, AttCT 0.09). Every result therefore uses unmerged adapters, except the RMCT check in the paper’s regime (Table 22), which the authors’ vLLM pipeline evaluates on merged weights. Because merging weakened RMCT in our setting, its effect there may be understated.

methodagreement (range)trained cueheld-outaccuracyrule met
Gemma-2-2B-it
BCT0.972–0.983-0.010-0.001-0.0030/3
ACT0.975–0.988-0.004+0.003-0.0050/3
AttCT0.987–0.991+0.002-0.002-0.0032/3
RMCT0.924–0.943+0.033+0.059+0.0120/3
RMCT+anchor†0.980–0.988-0.002+0.003-0.0030/3
response-only BCT0.981–0.991-0.010+0.001-0.0051/3
Bayes-BCT (r=.5r=.5)0.977–0.983+0.007+0.002+0.0020/3
Bayes-BCT (r=.8r=.8)0.972–0.986-0.014+0.001-0.0020/3
gold-gated BCT0.976–0.980-0.013-0.001+0.0000/3
position-diverse BCT0.965–0.982-0.001-0.001+0.0080/3
Llama-3.1-8B-Instruct
BCT0.972–0.983+0.010+0.005-0.0020/3
ACT0.971–0.979+0.007+0.003-0.0030/3
AttCT0.979–0.985-0.001-0.002+0.0080/3
RMCT0.948–0.952+0.013+0.025-0.0020/3
Bayes-BCT (r=.8r=.8)0.974–0.980-0.004+0.007+0.0020/3
gold-gated BCT0.977–0.980-0.003-0.003+0.0080/3
position-diverse BCT0.966–0.975+0.016+0.004+0.0050/3

Table 5. Merged (bf16 merge_and_unload) and unmerged adapters, per method over its three seeds. The columns give the range of agreement of the generated test answers, the seed-mean difference (merged − unmerged) in trained-cue flip∣knewflip|knew, held-out flip∣knewflip|knew and uncued accuracy, and the number of seeds that meet the declared rule. The merge check came before the learning-rate re-selection, so the Gemma ACT and AttCT rows are the checkpoints at the superseded rates. Held-out rates use the 16-token answers before the few-shot correction. † marks an exploratory variant.

Additional results

This appendix gives the tables behind Sections 4.1–4.6, in the same order. Analyses outside the pre-registration are marked exploratory.

Efficacy and transfer (§4.1)

Table 6 gives the test-set rates behind Figure 2. The held-out mean hides opposite movements on individual cues (Table 7). BCT raises harmful-cue following on the distractor-argument cue on both models while lowering most of the others. Only RMCT and its anchored variant significantly lower it on the wrong few-shot cue on either model, counting an unparsed answer as not following the cue. In the letter-prior control (Table 8), the drop beyond the letter prior is positive on the trained cue for every method, and on held-out cues it is near zero only for Llama BCT. Gemma RMCT puts about half of its uncued answers on one letter (D in every seed), against at most about a third for every other method, so we read its raw rates next to this control.

methodtrained cuepost hocdistractor factwrong few-shotdistractor arg.squares
Gemma-2-2B-it
original0.4900.7940.2550.5040.7370.440
BCT0.116 (-0.37)0.391 (-0.40)0.086 (-0.17)0.513 (+0.01°)0.823 (+0.09)0.292 (-0.15)
ACT0.241 (-0.25)0.679 (-0.12)0.066 (-0.19)0.543 (+0.04°)0.667 (-0.07)0.438 (+0.00°)
AttCT0.257 (-0.23)0.773 (-0.02°)0.056 (-0.20)0.488 (-0.02°)0.688 (-0.05)0.427 (-0.01°)
RMCT0.091 (-0.40)0.369 (-0.43)0.015 (-0.24)0.326 (-0.18)0.506 (-0.23)0.196 (-0.24)
RMCT+anchor†0.143 (-0.35)0.493 (-0.30)0.042 (-0.21)0.311 (-0.19)0.639 (-0.10)0.268 (-0.17)
Llama-3.1-8B-Instruct
original0.5550.7510.1360.4260.5090.284
BCT0.350 (-0.20)0.580 (-0.17)0.056 (-0.08)0.358 (-0.07°)0.743 (+0.23)0.368 (+0.08°)
ACT0.236 (-0.32)0.618 (-0.13)0.028 (-0.11)0.374 (-0.05°)0.416 (-0.09)0.195 (-0.09)
AttCT0.274 (-0.28)0.638 (-0.11)0.028 (-0.11)0.390 (-0.04°)0.446 (-0.06)0.238 (-0.05°)
RMCT0.042 (-0.51)0.256 (-0.50)0.005 (-0.13)0.253 (-0.17)0.213 (-0.30)0.211 (-0.07°)

Table 7. Per-cue flip∣knewflip|knew on the test set (generated answers, with few-shot cues scored at the test slot). The table gives the original value and, for each method, the seed mean with the change from the original model in parentheses. ° marks a change whose 95% question-cluster bootstrap interval includes zero, and † an exploratory variant.

Selectivity (§4.2)

Table 9 gives the paired drops and their ratio pooled over positions. Table 10 recomputes the ratio on the same item sets as absolute drops, proportional drops (1−1- trained/original rate) and log-odds shifts. The original model follows correct cues more often than wrong ones, so the absolute scale favors a larger helpful drop and the proportional scale a smaller one. Pooled over positions, every method stays far above the ratio of 0.5 that would mark a selective fix on every scale. Table 11 gives the ratios position by position, and Table 12 gives the same-question test. All of these analyses are exploratory. Two exceptions appear. After the options, where the fix is weak, the helpful drop is mostly the smaller one in absolute and proportional terms, and at the end position a few ratios fall below one half (Table 11). The second exception is in the same-question test, with the cue before the options. There, on the log-odds scale, the little cue following that remains favors correct cues more than before for Gemma RMCT and for Gemma gold-gated BCT, an oracle variant (Table 12).

Position (§4.3)

Other explanations. Table 13 tests three other explanations of the position effect. Cue strength. Cues after the options are on average stronger. We therefore select the wording × position cells whose original harmful-cue following lies in a range shared by cells before and after the options (Gemma 0.50–0.63, with only 2 cells after the options and Llama 0.26–0.80). In these cells, the reduction is still much larger before the options than after them for BCT, ACT and AttCT, and larger by less for RMCT. A regression of the reduction on strength and an after-the-options indicator over all 24 cells agrees. A third letter. If we count every answer other than the correct one as following the cue, the reduction of Gemma RMCT falls from

methodhelpful dropharmful dropratio [95% CI]rel. red. frontprepostend
Gemma-2-2B-it (original model’s harmful-cue following, front 0.493, pre 0.463, post 0.746, end 0.878)
BCT0.384 ± 0.0570.304 ± 0.0771.26 [1.08, 1.49]0.9510.8850.4030.180
ACT0.309 ± 0.0040.252 ± 0.0021.23 [1.10, 1.38]0.9200.7910.1270.093
AttCT0.280 ± 0.0060.212 ± 0.0061.32 [1.16, 1.51]0.9120.7840.0640.025
RMCT0.416 ± 0.0480.377 ± 0.0351.10 [0.96, 1.28]0.9240.9180.5420.382
Llama-3.1-8B-Instruct (original model’s harmful-cue following, front 0.381, pre 0.403, post 0.605, end 0.638)
BCT0.146 ± 0.0390.041 ± 0.0553.55 –0.7200.455-0.152-0.207
ACT0.247 ± 0.0030.186 ± 0.0021.33 [1.16, 1.52]0.9250.6980.0950.071
AttCT0.232 ± 0.0030.171 ± 0.0021.36 [1.18, 1.56]0.9200.6650.0740.054
RMCT0.473 ± 0.0480.380 ± 0.0241.24 [1.11, 1.40]0.9730.9230.7470.615

caption: tbl:9: Helpful cues drop together with harmful ones, and the fix fades when the cue moves after the options. Cue battery, pooled over six wordings and four positions. Paired drops in helpful- and harmful-cue following (seed mean ± SD), their ratio with a 95% question-bootstrap interval (“–” where the harmful drop exceeds .02 in fewer than 97.5% of samples), and the relative reduction of harmful-cue following at each position (only front is trained).

:::

methodabsolute [95% CI]proportional [95% CI]log-odds [95% CI]
Gemma-2-2B-it
BCT1.26 [1.08, 1.49]0.98 [0.89, 1.09]1.30 [1.10, 1.54]
ACT1.23 [1.09, 1.37]0.99 [0.93, 1.06]1.35 [1.18, 1.56]
AttCT1.32 [1.15, 1.51]1.06 [0.97, 1.15]1.49 [1.26, 1.75]
RMCT1.10 [0.95, 1.28]0.87 [0.78, 0.97]1.10 [0.93, 1.29]
RMCT+anchor†1.04 [0.94, 1.15]0.83 [0.78, 0.88]1.08 [0.96, 1.22]
Llama-3.1-8B-Instruct
BCT3.55 –2.30 [1.34, 5.92]3.84 [2.11, 11.06]
ACT1.33 [1.17, 1.52]0.89 [0.81, 0.97]1.36 [1.19, 1.58]
AttCT1.36 [1.19, 1.56]0.90 [0.82, 0.99]1.41 [1.23, 1.64]
RMCT1.24 [1.11, 1.40]0.84 [0.79, 0.90]0.97 [0.86, 1.08]

caption: tbl:10: Selectivity on three scales. Each cell is the seed-mean helpful drop divided by the seed-mean harmful drop on the paired item sets, over all positions, with a 95% question-bootstrap interval (B = 2000, and “–” where the harmful drop exceeds .02 in fewer than 97.5% of samples). Exploratory. † marks an exploratory variant.

:::

methodbefore optionsfrontprepostend
helpful / harmful drop: absolute / proportional / log-odds
Gemma-2-2B-it
BCT1.50 / 0.99 / 1.081.49 / 0.98 / 0.951.51 / 1.00 / 1.150.88 / 0.73 / 1.090.86 / 0.76 / 1.95
ACT1.37 / 0.98 / 1.081.41 / 0.99 / 1.071.33 / 0.96 / 1.060.74 / 0.63 / 1.130.29 / 0.26 / 0.98
AttCT1.37 / 0.96 / 1.071.42 / 0.98 / 1.101.29 / 0.92 / 1.030.82 / 0.70 / 1.33– / – / -0.17
RMCT1.34 / 0.92 / 0.901.37 / 0.94 / 0.941.31 / 0.90 / 0.860.98 / 0.82 / 1.130.68 / 0.60 / 1.31
RMCT+anchor†1.39 / 0.95 / 0.961.42 / 0.96 / 0.971.36 / 0.93 / 0.940.70 / 0.59 / 0.940.38 / 0.34 / 1.01
Llama-3.1-8B-Instruct
BCT1.78 / 1.01 / 1.351.62 / 0.94 / 1.122.01 / 1.13 / 1.64– / – / –– / – / –
ACT1.42 / 0.84 / 0.891.53 / 0.93 / 0.851.28 / 0.74 / 0.890.93 / 0.66 / 1.460.58 / 0.43 / 0.99
AttCT1.42 / 0.83 / 0.921.54 / 0.93 / 0.881.25 / 0.72 / 0.911.11 / 0.79 / 1.780.42 / 0.31 / 0.75
RMCT1.54 / 0.93 / 0.821.49 / 0.93 / 0.621.58 / 0.94 / 0.920.97 / 0.70 / 0.891.03 / 0.77 / 1.10

caption: tbl:11: Helpful-to-harmful ratio before the options (front and pre pooled) and at each position, with each cell on three scales (“–” on a scale where the harmful drop is below .02). Seed means on the paired item sets. Exploratory. † marks an exploratory variant.

:::

0.542 to 0.39 just after the options and from 0.382 to 0.28 at the end, and that of the other methods falls less. If we also leave out the questions whose correct answer is each checkpoint’s most frequent uncued letter, Gemma RMCT removes 0.18 and 0.10 (BCT 0.36 and 0.14). Under the same count, Llama RMCT keeps 0.63 and 0.51 (ACT 0.09 and 0.05). Across the three Gemma seeds of RMCT, its share of uncued D answers on the cue battery (0.615, 0.532 and 0.415) and its unadjusted reduction just after the options (0.651, 0.542 and 0.433) fall together. The anchored variant, which has no strong letter preference, keeps 0.38 and 0.30 on Gemma, about as much as BCT just after the options and more at the end. Wording. Before the options, the three novel wordings, which never occur in training, are reduced within about .06 of the three training-family wordings (Llama BCT reduces the novel ones more), and after the options both families fade together. The training family has 51 templates, which limits this contrast, but it suggests that the position of the cue matters more than its phrasing.

Learning rate and training data. Table 15 runs the cue battery on the seed-42 checkpoints one grid step below and above the selected learning rate of each of the four methods (where that point exists and is not degenerate) and on the Llama BCT, ACT and AttCT checkpoints trained with the data and configuration of the AttCT authors (18 checkpoints, declared before the runs in Table 2). At every learning rate, the helpful drop is at least as large as the harmful drop in absolute terms and at least 0.74 times it on every scale, and for BCT, ACT and AttCT the fix stays tied to the positions before the options. ACT and AttCT trained with the authors’ recipe show the same pattern, while BCT trained with the authors’ recipe reduces harmful-cue following by at most about a third at any position.

Original cue positions. Table 14 splits the trained-cue items of the test set by the cue position that each source row already has, an analysis we ran after all other results were known. Of these items, 68 put the cue before the question (the trained position), 68 before the options and 80 after them. With the cue first, every method removes 0.80–1.00 of harmful-cue following. After the options, ACT and AttCT remove only 0.14–0.28 on both models, as on the cue battery. RMCT keeps most of its fix there on both models (without a letter control on Gemma), and BCT keeps it on Gemma only. The interval of the front–post gap excludes zero except for Gemma BCT and Llama RMCT, and the cells are small (30–43 known items per position and model).

Mechanism

Table 16 gives patch recovery for the circuit, its head subsets and the trained-cue flips alone. Table 17 gives the cue-span attention of the early readers on the labeled items of the original model (the quantity that P3 tests), its drop on the trained-cue items that each method fixes and on those it still follows (exploratory), and the depth centroids that P4 tests. For ACT and AttCT, the gap between fixed and still-followed items appears only on the trained cue. Pooled over all six cue types, the attention drops on the two kinds of items are about equal for every method (exploratory). The OV depth centroids of Gemma order AttCT < ACT < RMCT in every seed, so P4 holds. However, the centroid of ACT moves between the two learning rates selected for it (Appendix B.4), and the methods place their adapters on different modules, so we do not treat depth as a property of the objective. Table 18 gives the layer swap on both models under three readouts. These are the pre-registered binary readout, the share of the trained model’s own fixes that the mixed model reproduces, and the share of the fall in the probability of the cued letter. The Gemma layer swap is exploratory, because D1 concerns Llama.

Training-data variants

Table 19 gives the values behind Figure 6 for both models, with test-set rates and uncued accuracy.

Relapse

Table 20 gives the relapse fractions behind § 4.6 on the probability and log-odds scales. Beyond the drift of the original model, the fraction of Llama ACT has the seed 95% interval [−0.021, 0.346] and the item interval [−0.432, 0.513]. On the log-odds scale, RMCT, which starts from the lowest rate, rises two to three times as much as the original model.

Checks against the source papers

Two checks tie our pipeline to the source papers. First, on the evaluator of the AttCT authors (Table 21), the Llama checkpoints trained with the authors’ own data and configuration reproduce the values that the AttCT paper reports (its Table 11) within sampling error. Llama ACT and AttCT trained on our shared data come close to these values on MMLU BRR (0.033 and 0.031 against 0.019 and 0.014), but not on held-out BRR (0.057 and 0.062 against 0.003 and 0.005). According to the authors’ evaluator code, that held-out rate uses

held-out questions with the cue templates of their training data, which our shared data do not contain verbatim. It does not use held-out cue types, so the near-zero values of the AttCT paper and our partial transfer do not conflict. The clean accuracy of Gemma BCT falls from 0.596 to 0.512. BCT loses accuracy at every learning rate in the table that moves the MMLU bias rate (only nominally at 3.16e-4), whereas the selected full-attention BCT of Irpan et al. [15] keeps its MMLU accuracy on Gemma 2 2B (61.6 against 61.0). Second, RMCT in the authors’ own regime (Table 22) reproduces the direction of their reported effect but not its size. At their middle learning rate, our two training seeds reach switch rates of 0.362 (two evaluation passes) and 0.350 (one pass) on the training bias, against 0.141/0.132 in the authors’ two evaluations of their

checkpointMMLU BRR ± SEheld-out BRR ± SEMMLU clean acc.Anthropic rate
Gemma-2-2B-it original0.540 ± 0.0180.331 ± 0.0190.596 [0.565, 0.626]0.848
Gemma-2-2B-it BCT0.145 ± 0.0190.319 ± 0.0190.512 [0.481, 0.543]0.466
Gemma-2-2B-it ACT0.081 ± 0.0170.160 ± 0.0170.602 [0.571, 0.632]0.838
Gemma-2-2B-it AttCT0.131 ± 0.0180.176 ± 0.0180.596 [0.565, 0.626]0.849
Gemma-2-2B-it response-only BCT0.011 ± 0.0160.076 ± 0.0180.536 [0.505, 0.567]0.651
Gemma-2-2B-it BCT sweep, LR 1e-50.543 ± 0.0180.421 ± 0.0190.594 [0.563, 0.624]0.656
Gemma-2-2B-it BCT sweep, LR 3.16e-50.352 ± 0.0200.414 ± 0.0190.535 [0.504, 0.566]0.807
Gemma-2-2B-it BCT sweep, LR 3.16e-40.261 ± 0.0190.306 ± 0.0190.566 [0.535, 0.596]0.730
Llama-3.1-8B-Instruct original0.202 ± 0.018 (0.202)0.183 ± 0.018 (0.183)0.664 [0.634, 0.693] (0.664)0.939 (0.939)
Llama-3.1-8B-Instruct BCT0.087 ± 0.0160.125 ± 0.0170.650 [0.620, 0.679]0.877
Llama-3.1-8B-Instruct ACT0.033 ± 0.0150.057 ± 0.0150.670 [0.640, 0.698]0.907
Llama-3.1-8B-Instruct AttCT0.031 ± 0.0150.062 ± 0.0150.667 [0.637, 0.696]0.909
Llama-3.1-8B-Instruct BCT, authors’ recipe§0.180 ± 0.017 (0.189)0.120 ± 0.017 (0.121)0.660 [0.630, 0.689] (0.664)0.928 (0.926)
Llama-3.1-8B-Instruct ACT, authors’ recipe§0.019 ± 0.014 (0.019)0.003 ± 0.014 (0.002)0.674 [0.644, 0.702] (0.669)0.883 (0.880)
Llama-3.1-8B-Instruct AttCT, authors’ recipe§0.014 ± 0.014 (0.016)0.005 ± 0.014 (0.005)0.668 [0.638, 0.696] (0.680)0.889 (0.902)

caption: Table 21: The evaluator of the AttCT authors on seed-42 final checkpoints (adapter unmerged, their code). The columns give the MMLU bias rate (BRR = P(cued answer | cued) − P(cued answer | clean), n = 1000) with its unpaired SE, the held-out BRR (n = 951), clean MMLU accuracy (Wilson 95%) and the Anthropic model-written sycophancy rate (n = 999). Gemma ACT and AttCT use the re-selected learning rates (Appendix § 8.§ 4), and the Gemma BCT sweep rows are three other learning rates of the BCT grid. § marks the authors’ own data and configuration run through their pipeline. Values in parentheses are those that the AttCT paper reports for its Llama checkpoints.

:::

LRevalstraining-bias switch rateheld-out switch rateauthors’ logs (distractor argument)authors’ logs (held-out)
base20.650 [0.567, 0.724]0.419 [0.383, 0.456]0.650–
1e-420.497 [0.417, 0.576]0.414 [0.378, 0.450]0.325/0.3460.438/0.414
2.86e-420.362 [0.284, 0.449]0.375 [0.337, 0.414]0.141/0.1320.320/0.322

caption: Table 22: RMCT in the authors’ own regime (their trainer and paper configuration, chain-of-thought answers, evaluated with their protocol on 100 questions per pass). The table gives the switch rate toward the bias on the training bias and pooled over five held-out biases, for final-step checkpoints pooled over two evaluation passes. Next to these are the values recomputed from the evaluation logs that the authors released (one training run, two passes). The authors’ base value comes from their 400-question logs, and their held-out value for the second pass pools four biases, while ours pools five. A second training run at 2.86e-4 (seed 43) gives 0.350 and 0.309 at its final step. We evaluate with the authors’ vLLM pipeline on bf16-merged checkpoints, which weakened RMCT in our setting (Appendix § 10.§ 3). The direction reproduces, but the size does not at 2.86e-4.

:::

run. This regime uses their prompts, renderer and parser. Possible causes of the gap are the variance of a 16-step run, scoring merged weights, and a difference between Tinker’s internal LoRA step size, which we cannot inspect, and that of our local implementation.

Artifacts, licenses and compute

Models. We use Gemma-2-2B-it (Gemma Terms of Use) and Llama-3.1-8B-Instruct (Llama 3.1 Community License) unmodified as base models for LoRA training, within their license terms and use policies. Any released adapters will carry the required notices.

Data. Questions come from the multiple-choice dumps released with Chua et al. [3] (cot-transparency, MIT license), built on MMLU (MIT), HellaSwag (MIT per its dataset card), LogiQA (its paper states that it is freely available, and its repository has no license file) and TruthfulQA (Apache-2.0). The evaluator of the AttCT authors also uses the model-written sycophancy evaluations of Anthropic (CC BY 4.0; Perez et al., [22]), and the RMCT check in the paper’s regime uses the multiple-choice questions of Humanity’s Last Exam (MIT; Center for AI Safety et al., [2]), as the authors’ pipeline does. These are English benchmark questions without personal information. The benign fine-tuning examples are 2,000 rows of the instruction file distributed with the AttCT repository, disjoint from the rows in the training mix of BCT. That file pairs instruction prompts with model-written responses, and the repository labels it “Alpaca/GPT-4T” without stating a license.

Code. BCT, ACT and AttCT run from the public AttCT repository (no license file) at a fixed commit. RMCT runs the authors’ trainer from their public fork of cot-transparency (MIT). We use the AttCT code unmodified and do not redistribute it, and our configurations are given as changes to its files. Our code, configurations and the redacted pre-registration file are in the supplementary material, and per-item results will be released for research use.

Compute and software. All training and evaluation ran on single NVIDIA A100 GPUs (80 GB, a few jobs on 40 GB), for about 210 GPU-hours, including the retired pilot design. We used PyTorch 2.9.0, Transformers 4.57.6, PEFT 0.19.1, vLLM 0.11.2, Tinker 0.19.0 and tinker-cookbook 0.4.1.

References

  1. [1]David Demitri Africa and Arathi Mani. 2026. Consistency training can entrench misalignment. In Proceedings of the 43rd International Conference on Machine Learning, volume 306 of Proceedings of Machine Learning Research, pages 715–753. PMLR.
  2. [2]Center for AI Safety, Scale AI, and HLE Contributors Consortium. 2026. A benchmark of expert-level academic questions to assess AI capabilities. Nature, 649:1139–1146.
  3. [3]James Chua, Edward Rees, Hunar Batra, Samuel R. Bowman, Julian Michael, Ethan Perez, and Miles Turpin. 2025. Bias-augmented consistency training reduces biased reasoning in chain-of-thought. Preprint, arXiv:2403.05518.arxiv.org/abs/2403.05518
  4. [4]Sukrati Gautam, Neil Shah, Arav Dhoot, Bryan Maruyama, Caroline Wei, Rohan Kapoor, Robert Sidey, Prakhar Gupta, Zi Cheng Huang, and David Demitri Africa. 2026. Consistency training along the transformer stack. Preprint, arXiv:2606.05817.arxiv.org/abs/2606.05817
  5. [5]Gemma Team. 2024. Gemma 2: Improving open language models at a practical size. Preprint, arXiv:2408.00118.
  6. [6]Rifo Ahmad Genadi, Munachiso Samuel Nwadike, Nur-daulet Mukhituly, Tatsuya Hiraoka, Hilal AlQaubeh, and Kentaro Inui. 2026. Syncopancy hides linearly in the attention heads. In Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pages 6896–6912, Rabat, Morocco. Association for Computational Linguistics.
  7. [7]Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al- Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenев, Arthur Hinsvark, and 542 others. 2024. The llama 3 herd of models. Preprint, arXiv:2407.21783.arxiv.org/abs/2407.21783
  8. [8]Prakhar Gupta and Vaibhav Gupta. 2026. Context-grounding gains are mediated by pre-existing machinery: Auditing grpo, sft, and dpo. Preprint, arXiv:2609.00925.arxiv.org/abs/2609.00925
  9. [9]Prakhar Gupta, Terry Jingchen Zhang, Florent Draye, Bernhard Schölkopf, and Zhijing Jin. 2026a. How does alignment tuning shape representations of sycophancy and related cue-induced biases in llms? Preprint, arXiv:2607.18114.arxiv.org/abs/2607.18114
  10. [10]Prakhar Gupta, Terry Jingchen Zhang, and Zhijing Jin. 2026b. Many biases, one circuit: Post-training tunes a pretrained cue-routing mechanism. Preprint, arXiv:2610.22937.arxiv.org/abs/2610.22937
  11. [11]Andy Han, Kristina Fujimoto, Avidan Shah, Kiet Nguyen, Kai Xu, Chen Yueh-Han, Ilia Sucholutsky, and Rico Angell. 2026. On-policy consistency training improves llm safety with minimal capability degradation. Preprint, arXiv:2605.21834.arxiv.org/abs/2605.21834
  12. [12]Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021. Measuring massive multitask language understanding. In International Conference on Learning Representations.
  13. [13]Edward J Hu, yelong shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations.arxiv.org/abs/2106.09685
  14. [14]Sohaib Imran, Prakhar Gupta, Jannes Elstner, and David Demitri Africa. 2026. Consistency training while mitigating obfuscation via rate matching. Preprint, arXiv:2606.02211.arxiv.org/abs/2606.02211
  15. [15]Alex Irpan, Alexander Matt Turner, Mark Kurzeja, David K. Elson, and Rohin Shah. 2025. Consistency training helps stop sycophancy and jailbreaks. Preprint, arXiv:2510.27062.arxiv.org/abs/2510.27062
  16. [16]Andrew Lee, Xiaoyan Bai, Itamar Pres, Martin Wattenberg, Jonathan K Kummerfeld, and Rada Mihalcea. 2024. A mechanistic understanding of alignment algorithms: A case study on dpo and toxicity. In International Conference on Machine Learning, pages 26361–26378. PMLR.arxiv.org/abs/2401.01967
  17. [17]Stephanie Lin, Jacob Hilton, and Owain Evans. 2022. TruthfulQA: Measuring how models mimic human falsehoods. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3214–3252, Dublin, Ireland. Association for Computational Linguistics.arxiv.org/abs/2109.07958
  18. [18]Jian Liu, Leyang Cui, Hanmeng Liu, Dandan Huang, Yile Wang, and Yue Zhang. 2020. Logiqa: A challenge dataset for machine reading comprehension with logical reasoning. In Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI-20, pages 3622–3628. International Joint Conferences on Artificial Intelligence Organization. Main track.arxiv.org/abs/2007.08124
  19. [19]Jiarui Liu, Terry Jingchen Zhang, Ryan Faulkner, Xuanqiang Angelo Huang, Vilém Zouhar, Dominik Glandorf, Isabel Dahlgren, Rishit Dagli, Yuen Chen, Felix Leeb, Van Q. Truong, Punya Syon Pandey, Yves Bicker, Suvajit Majumder, Wenyuan Jiang, Zeju Qiu, Sankalan Pal Chowdhury, Minimaya Sachan, Bernhard Schölkopf, and 2 others. 2026. PaperMentor: A human-centered multi-agent writing tutor for AI research papers in overleaf. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations), pages 397–407, San Diego, California, United States. Association for Computational Linguistics.DOI
  20. [20]Huanhuan Ma, Henry Peng Zou, Chengze Li, Enze Ma, Yunyue Su, and Philip S. Yu. 2026. Sycophancy suppression can impair rational updating: Anti-sycophancy should preserve the ability to update. Preprint, arXiv:2608.26511.arxiv.org/abs/2608.26511
  21. [21]Manav Pandey. 2026. Llms know they’re wrong and agree anyway: The shared sycophancy-lying circuit. Preprint, arXiv:2604.19117.arxiv.org/abs/2604.19117
  22. [22]Ethan Perez, Sam Ringer, Kamile Lukošiute, Karina Nguyen, Edwin Chen, Scott Heiner, Craig Pettit, Catherine Olsson, Sandipan Kundu, Saurav Kadavath, Andy Jones, Anna Chen, Benjamin Mann, Brian Israel, Bryan Seethor, Cameron McKinnon, Christopher Olah, Da Yan, Daniela Amodei, and 44 others. 2023. Discovering language model behaviors with model-written evaluations. In Findings of the Association for Computational Linguistics: ACL 2023, pages 13387–13434, Toronto, Canada. Association for Computational Linguistics.arxiv.org/abs/2212.09251
  23. [23]Long Phan, Devin Kim, Alexander Pan, Alice Blair, Adam Khoja, and Dan Hendrycks. 2026. Reducing political manipulation with consistency training. Preprint, arXiv:2605.22771.
  24. [24]Xiangyu Qi, Yi Zeng, Tinghao Xie, Pin-Yu Chen, Ruoxi Jia, Prateek Mittal, and Peter Henderson. 2024. Fine-tuning aligned language models compromises safety, even when users do not intend to! In The Twelfth International Conference on Learning Representations.arxiv.org/abs/2310.03693
  25. [25]Avidan Shah, Jannik Brinkmann, and Rico Angell. 2026. Mitigating adaptive attacks against reasoning models with activation consistency training. Preprint, arXiv:2605.28467.arxiv.org/abs/2605.28467
  26. [26]Mrinank Sharma, Meg Tong, Tomasz Korbak, David Duvenaud, Amanda Askell, Samuel R. Bowman, Newton Cheng, Esin Durmus, Zac Hatfield-Dodds, Scott R. Johnston, Shauna Kravec, Timothy Maxwell, Sam McCandlish, Kamal Ndousse, Oliver Rausch, Nicholas Schiefer, Da Yan, Miranda Zhang, and Ethan Perez. 2025. Towards understanding sycophancy in language models. Preprint, arXiv:2310.13548.arxiv.org/abs/2310.13548
  27. [27]Debu Sinha. 2026. SycoBench-600: Measuring sycophancy and correction selectivity in LLM assistants. In Findings of the Association for Computational Linguistics: ACL 2026, pages 35278–35284, San Diego, California, United States. Association for Computational Linguistics.
  28. [28]Elias Stengel-Eskin, Peter Hase, and Mohit Bansal. 2025. Teaching models to balance resisting and accepting persuasion. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 8108–8122, Albuquerque, New Mexico. Association for Computational Linguistics.arxiv.org/abs/2410.14596
  29. [29]Xiangkun Sun, Lingkai Kong, Aoqi Zhang, Liang Zeng, and Tonghan Wang. 2026. How llms are persuaded: A few attention heads, rerouted. Preprint, arXiv:2605.09314.arxiv.org/abs/2605.09314
  30. [30]Haoran Tang and Rajiv Khanna. 2026. Unlearning deceptive behaviors in llms with contrastive forget sets. Preprint, arXiv:2609.38909.arxiv.org/abs/2609.38909
  31. [31]Thinking Machines Lab. 2025. Tinker: a training api for researchers and developers. https://thinkingmachines.ai/tinker. Software development kit and tinker-cookbook, https://github.com/thinking-machines-lab/tinker-cookbook.
  32. [32]Miles Turpin, Julian Michael, Ethan Perez, and Samuel R. Bowman. 2023. Language models don’t always say what they think: Unfaithful explanations in chain-of-thought prompting. In Thirty-seventh Conference on Neural Information Processing Systems.arxiv.org/abs/2305.04388
  33. [33]Emiel van Miltenburg, Chris van der Lee, and Emiel Krahmer. 2021. Preregistering NLP research. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 613–623, Online. Association for Computational Linguistics.arxiv.org/abs/2103.06944
  34. [34]Qian Wang, Xuandong Zhao, Zirui Zhang, Zhanzhi Lou, Nuo Chen, Dawn Song, and Bingsheng He. 2026. Making bias non-predictive: Training robust llm reasoning via reinforcement learning. Preprint, arXiv:2602.01528.arxiv.org/abs/2602.01528
  35. [35]Xinpeng Wang, Bolei Ma, Chengzhi Hu, Leon Weber-Genzel, Paul Röttger, Frauke Kreuter, Dirk Hovy, and Barbara Plank. 2024. “my answer is C”: First-token probabilities do not match text answers in instruction-tuned language models. In Findings of the Association for Computational Linguistics: ACL 2024, pages 7407–7416, Bangkok, Thailand. Association for Computational Linguistics.arxiv.org/abs/2402.14499
  36. [36]Jerry Wei, Da Huang, Yifeng Lu, Denny Zhou, and Quoc V. Le. 2024. Simple synthetic data reduces sycophancy in large language models. Preprint, arXiv:2308.03958.arxiv.org/abs/2308.03958
  37. [37]Edwin B. Wilson. 1927. Probable inference, the law of succession, and statistical inference. Journal of the American Statistical Association, 22(158):209–212.
  38. [38]Sen Yang and Yuen-Hei Yeung. 2026a. Resist and update: Counterfactual report coordinates for incentive-compatible llms. Preprint, arXiv:2607.12985.arxiv.org/abs/2607.12985
  39. [39]Sen Yang and Yuen-Hei Yeung. 2026b. Resist, update, reject: Preference optimization installs a prior-dependent reliability switch. Preprint, arXiv:2609.22359.arxiv.org/abs/2609.22359
  40. [40]Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. 2019. HellaSwag: Can a machine really finish your sentence? In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4791–4800, Florence, Italy. Association for Computational Linguistics.arxiv.org/abs/1905.07830

Paper details

vlnc:W_apv3f5sh
Version
v1 · latest
Published
Added
License
CC BY-SA 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

PaperMentor, AdminBot

View statement
Looking for related papers…
Contents