betaFeedback

WorksComputer Science

Programmable Semantic Interventions: Learning Causal Carriers from Behavioral Programs

Abstract

Causal abstraction explains a frozen network by aligning its activations with the variables of a simpler, interpretable model, and checking that interventions on the two agree. Finding such an alignment currently requires a counterfactual dataset: for every pair of prompts, the answer the network ought to give once the variable has been changed. In practice, those labels are produced by an executable algorithm for the behavior, written before studying the model supposed to implement it. However, the criterion these methods optimize only leverages the answer an intervention should produce. Here, we develop an approach that lets the investigator define that relationship directly: the answers the variable can produce, and which of them an intervention should yield. We call this interface a semantic program and introduce Programmable Semantic Interventions (PSI): a low-rank activation subspace learned so that interchanging it changes the model's answer as the program declares, with the expected answers read from the frozen model itself. On the MIB benchmark PSI performs on par with supervised alignment at matched data and training budget, without curated pairs, and it needs no ground-truth labels; while on natural science questions with gold labels withheld, it tracks the model's own answers better than a subspace trained on gold labels. The declaration also fixes a coordinate system with known semantics, enabling donor-free steering: jointly training PSI for both localization and control realizes 94.8% of held-out requests for answers the investigator chooses, without the need for donor prompts.

Introduction

Mechanistic interpretability sets out to explain the inner workings of a network in terms of the algorithm it implements ([20, 25]). Recently, an increased emphasis has been placed on providing causal claims ([18]), i.e., what part of a network’s activations encodes an abstract, high-level (simpler) variable, and does an intervention move the network’s behavior in a predictable way that the corresponding high-level model says it should ([12])? The latter distinction is essential: reading a property off an activation with a trained probe establishes that the property is present ([3]), while a causal claim asks whether the network actually uses it. Namely, causal abstraction makes the causal claim testable ([9, 12]), and standard benchmarks now exist to evaluate methods against it ([1, 19]). However, testing causal claims at scale currently requires the construction of a counterfactual dataset. For every pair of inputs differing (only) in the value of a putative high-level variable, one must build the answer the network ought to give once that value has been exchanged. Methods in the distributed alignment lineage obtain that answer by executing a high-level algorithm the investigator has specified in advance ([11, 33, 13, 28]). In current practice, the investigator therefore commits to an algorithm for the behavior before studying the model that is supposed to implement it, and the approach reaches only those behaviors for which such an algorithm can be written down, using a curated list of prompt pairs.

The success criterion for identification is interchange intervention accuracy [10], and current methods are directly optimized against it. However, the criterion itself never inspects the high-level algorithm: it merely relies on the answer the algorithm produces under the intervention and compares it with the intervened network’s output. This simple observation points to a significant gap between the amount of required specifications and what is ultimately used. Concretely, consider the problem of localizing the variable encoding the position of the correct option in a four-choice question. A full high-level causal specification, as used in current practice, is an executable algorithm that finds the option matching the question, records its position and outputs the label found there. We propose a different interface – semantic programs – in which the investigator writes only the possible answers and the rule that gives the expected one.

A semantic program lists the answers a variable can produce and states which of them an intervention should yield; the expected answer itself is measured from the frozen model. We call the method built on it Programmable Semantic Interventions (PSI, Ψ\Psi): a low-rank activation subspace, learned so that interchanging it between two prompts makes the first answer as the program prescribes. Moreover, a program written in the model’s own answer space induces a coordinate system whose semantics the investigator knows, such that an arbitrary behavior can be directly requested in it. Coming back to the question-answering example, a semantic program for the same task consists of just two lines: a four-option library of labels, plus the rule that an interchange must replace the base’s label with the source’s. As such, it requires no ground-truth labels, and it can be used to request a reading that no prompt in the data exhibits.

Related work. Work on localizing causal variables has steadily reduced the choices that an investigator makes by hand. Distributed alignment search learns the subspace by gradient descent [11], its boundless variant learns the rank along with it [33], multi-task training adds attribute isolation [13], and a hypernetwork takes over the search across token positions [28]. However, throughout this progression, the requirement of an executable high-level model remains, and each method trains on a curated list of prompt pairs. Ψ\Psi thus follows this lineage by changing where the training labels come from rather than adding more automation machinery. A second line of work encodes a behavior into activations with no donor example. Directions fitted from contrastive prompts steer a model at inference [31, 23, 15] (Zou et al. 2023), and recent methods learn the write operator [2, 24] or amortize it over a space of requests [26]. Localizing a variable and controlling it are known to be distinct capabilities: the directions that read a concept and the directions that change it are geometrically distinct [21, 7, 35], steering vectors are brittle across prompts [30], and a subspace can move an output through a pathway unrelated to the feature it names [16]. We thus validate both capabilities in § 3 and refer to Appendix § 7.§ 1 for a broader overview of the relevant literature.

Contributions. We introduce the semantic program as the interface that a causal-variable method requests from the investigator, and Ψ\Psi, which learns a carrier from it. Four results follow: (i) on the causal-variable localization tasks, Ψ\Psi reaches the baseline’s quality without a counterfactual dataset, and performs on par with it at matched data and training budget; (ii) ablating the supervision, we measure the value of a counterfactual dataset and find that a method requiring no pair list recovers most of the performance by benefiting from a larger donor pool size which provides observations the pair list cannot reach; (iii) because the labels are read from the model, Ψ\Psi needs no ground truth: on natural science questions with gold labels withheld, it tracks the model’s own answers, wrong ones included, better than a subspace trained on gold labels; (iv) finally, because the answer list is fixed in advance, the same carrier realizes readings that the investigator requests, with no source prompt and including readings that no prompt in the data exhibits, surpassing what an interchange from a real donor achieves at the same site.

Methods

Problem setup. We consider a frozen subject ϕ\phi, an input prompt x\mathbf{x}, and indicate with ϕn(x)\phi^n(\mathbf{x}) the standard auto-regressive greedy rollout of the model predictions for a sentence length nn. We indicate by h=ϕ∣κ(x)h = \phi|_{\kappa}(\mathrm{x}) the hidden vector of activations h∈Rdh \in\mathbb{R}^{d} produced by the model, where κ\kappa is a set of location indices (e.g. the collection of token positions and layers in the residual stream of a transformer network). Causal abstraction [12] formalizes what it means for such a vector to carry a concept: an abstraction A\mathcal{A} maps the low-level states of the network to the states of a high-level model H\mathrm{H}, whose variables are the concepts we wish to reason with, and it is faithful under a commutative condition on interventions—intervening on the low-level model and abstracting the result should agree with abstracting first and then performing the corresponding intervention on H\mathrm{H} [see 12 for the formal exposition]. In our experiments, A:Rd→Rr\mathcal{A} : \mathbb{R}^{d} \to\mathbb{R}^{r} is a linear map from the hidden vector hh to an rr-dimensional high-level representation z=A(h)z = \mathcal{A}(h), and an intervention is performed by replacing part of the hidden vector hb=ϕ∣κ(b)h_{\mathrm{b}} = \phi|_{\kappa}(\mathrm{b}) of a base prompt b with the corresponding part of the hidden vector hs=ϕ∣κ(s)h_{\mathrm{s}} = \phi|_{\kappa}(\mathrm{s}) of a source prompt s, and then measuring the resulting output ϕb←s(b)\phi_{\mathrm{b}\leftarrow\mathrm{s}}(\mathrm{b}).

Measuring abstraction quality. Empirically, one quantifies the faithfulness of a candidate abstraction A\mathcal{A} by measuring the agreement between the two sides of the commutative condition over a distribution of interventions. An intervention is a pair ι=(b,s)\iota= (\mathrm{b}, \mathrm{s}) of a base and a source prompt, drawn from B×S\mathcal{B} \times\mathcal{S}. The interchange intervention accuracy (IIA) [10, 11] has been proposed as one such criterion: it measures how often the intervened low-level model and the high-level model produce the same output:

IIA=Eι∼B×S[1[ϕb←sn(b)=Hb←s(b)]],(1)\mathrm{IIA} = \mathbb{E}_{\iota\sim\mathcal{B}\times\mathcal{S}}\left[ \mathbf{1}\left[\phi_{\mathrm{b}\leftarrow\mathrm{s}}^{n}(\mathrm{b}) = \mathrm{H}_{\mathrm{b}\leftarrow\mathrm{s}}(\mathrm{b})\right]\right], \tag*{(1)}

where, with a slight abuse of notation, Hb←s(b)\mathrm{H}_{\mathrm{b}\leftarrow\mathrm{s}}(\mathrm{b}) denotes the high-level model’s output under the intervention corresponding to b←s\mathrm{b} \leftarrow\mathrm{s}, and the indicator 1[⋅]\mathbf{1}[\cdot] evaluates to 1 when the two outputs agree and 0 otherwise.

Aligning a subspace. Replacing the hidden vector wholesale would test whether the entire activation carries the concept (the Full Vector baseline of the MIB suite, Mueller et al., 2025), which is a rather crude baseline: a single residual stream is expected to carry many concurrent variables, more of them than it has dimensions [6]. A common alternative is to let the abstraction single out a low-dimensional subspace [11] or a sparse set of learned features [5, 8], leaving the rest of the activation untouched. Concretely, in our work we take an orthonormal A∈Rr×dA \in\mathbb{R}^{r\times d} with r≪dr \ll d, write π=ATA\pi= A^{\mathsf{T}}A for the projector onto its row space, and define the intervention that transplants the source’s carrier into the base while preserving the base’s complement:

hb←s=πhs+(I−π)hb.(2)h_{\mathrm{b}\leftarrow\mathrm{s}} = \pi h_{\mathrm{s}} + (I-\pi)h_{\mathrm{b}}. \tag*{(2)}

The abstraction is then simply the readout of that subspace, A(h)=Ah\mathcal{A}(h) = Ah, and identifying the causal variable amounts to finding AA. In our experiments we restrict ourselves to linear maps as non-linear alignments weaken what a successful abstraction certifies [29]. Moreover, a linear carrier keeps the intervention of Eq. (2) a projection, so the edit has a scale we can compare against the displacements interchange interventions naturally produce.

What the IIA criterion requires. One observation about Eq. (1) motivates our interface proposal (Section 2.1). The IIA criterion never inspects the inner workings of HH. It reads a single quantity off the high-level model: the output it produces under the intervention, while remaining oblivious to its variables, its mechanisms, or its causal graph. Existing methods [11, 33, 13, 28] nonetheless specify HH in full, as an executable algorithm, and obtain that output by running it. The result of that computation is then taken as the counterfactual label attached to each training pair. Thus, the full specification of HH is typically taken as a prerequisite for causal investigation. In particular, it requires postulating a particular algorithm the low-level model ϕ\phi is supposed to implement before one has studied the model itself, and it confines the method to tasks for which a full algorithmic specification is available.

Programmable Semantic Interventions

Rather than specifying the high-level model HH fully and computing its output, the investigator declares that output directly, as a rule stated in the subject’s own answer space. This declaration requires two components: an outcome library O\mathcal{O} that specifies which answers the concept pertains to, and a transport rule T\mathcal{T} that indicates how those answers should be transferred under an intervention. We call the resulting pair of objects that make up this declaration a semantic program, P=(O,T)\mathcal{P}=(\mathcal{O},\mathcal{T}). We offer a graphical overview of the method in Figure 1.

Figure 1. Programmable Semantic Interventions. (a) The investigator declares which answers O\mathcal{O} the variable is about: here the two numerical sums a carry does and does not produce, which result in different strings for the base b and the source s. The frozen subject supplies its probability on each of them, read at the cut ℓk\ell_k where the abstraction A\mathcal{A} selects the carrier. (b) A reading is a point in the simplex over that library. Interchanging the source’s carrier should move the base’s reading to τ(b,s)\tau(\mathrm{b},\mathrm{s}), and the distance dd to what the intervened model reads is the training signal; the same target can be named directly as q∗q^{*} and realized by an edit, with no source prompt.

The outcome library. The outcome library is a small, prompt-dependent set of candidate answers, one for each value the variable of interest can take:

O(x)=(o1(x),…,ok(x)).(3)\mathcal{O}(\mathbf{x}) = \left(o_1(\mathbf{x}), \ldots, o_k(\mathbf{x})\right). \tag*{(3)}

To make this concrete, consider a multiple-choice question, with the position of the correct option as the variable of interest (the MCQA task of Mueller et al. [19]). The prompt x\mathbf{x} asks Coconuts are brown. What color are coconuts? and lists four colors labelled X, C, P and A, with brown in second position; the library is the prompt’s own labels in the order it lists them,

O(x)=("X","C","P","A"),\mathcal{O}(\mathbf{x}) = (\text{"X"}, \text{"C"}, \text{"P"}, \text{"A"}),

indexed by the four values of the position. Writing it requires neither the correct answer nor any account of how the model finds it. For an entity-attribute lookup (e.g. the RAVEL task of Huang et al. [13]) the library is the set of admissible attribute values, while for the carry in two-digit addition it is the two sums with and without a carry (Figure 1), and writing it requires solving the task. We highlight two properties of Eq. (3). First, its entries are answers in the subject’s vocabulary: they are the (potentially multi-token) strings the model would have to emit. Second, the library says nothing about which of the library entries is correct. Its role is to fix a (semantic) reference frame, i.e. the set of outputs this concept is about and expected to elicit, and to leave the readings within that frame to the model.

Once a frame is declared, the subject ϕ\phi supplies its own coordinates within it. Writing ρi(x)=log⁡pϕ(oi(x)∣x)\rho_i(\mathbf{x})=\log p_\phi(o_i(\mathbf{x})\mid\mathbf{x}) for the log-probability the model assigns to the ii-th library entry, we define the following two quantities:

Z(x)=∑i=1keρi(x),qi(x)=eρi(x)Z(x).(4)Z(\mathbf{x}) = \sum_{i=1}^{k} e^{\rho_i(\mathbf{x})}, \qquad q_i(\mathbf{x}) = \frac{e^{\rho_i(\mathbf{x})}}{Z(\mathbf{x})}. \tag*{(4)}

The declared mass Z(x)∈[0,1]Z(\mathbf{x})\in[0,1] (the entries are distinct and none is a prefix of another, so they name disjoint continuations) answers how much of the model’s probability the frame captures at all. Z(x)Z(\mathbf{x}) thus quantifies the coverage of the declared library. The reading q(x)=[q1(x),…,qk(x)]\mathbf{q}(\mathbf{x})=[q_1(\mathbf{x}),\ldots,q_k(\mathbf{x})] is how the captured mass divides between the declared alternatives, and it is the vector we treat as the model’s opinion about the variable. Continuing the multiple-choice example, we subject that places p=0.70p=0.70 on "C" and p=0.05p=0.05 on each of the other three labels has Z(x)=0.85Z(\mathbf{x})=0.85 and q(x)=(0.06,0.82,0.06,0.06)\mathbf{q}(\mathbf{x})=(0.06,0.82,0.06,0.06): most of its mass lies inside the frame, and within the frame it is confident the correct option is the second. Both numbers are thus measured from the frozen model and require no further investigator input.

The transport rule. The second half of the program states how a reading q(x)q(x) should change when the variable is intervened upon. We represent this transformation as a map:

τ=T(q(s),q(b),γ),(5)\tau= \mathcal{T}\left(q(\mathrm{s}), q(\mathrm{b}), \gamma\right), \tag*{(5)}

which takes the source’s reading q(s)q(\mathrm{s}), the base’s reading q(b)q(\mathrm{b}), and an optional per-example gate γ\gamma, and returns the reading the base should exhibit once the source’s value of the variable has been written into it. The source reading q(s)q(\mathrm{s}) can be measured by a declared query for the variable or a donor prompt (see Appendix § 7.§ 3).

The simplest and most common rule is the full swap, τ=q(s)\tau= q(\mathrm{s}). Return to the multiple-choice example, and let the base b be the coconut question (second position) and the source s a question whose correct option is the fourth, with labels K, B, Q and D. If A really is the subspace that carries the position, then transplanting the source’s carrier into the base should make the base behave as though the correct option were the fourth: it should answer A rather than C. Note what makes this well posed: the two prompts have different libraries, O(b)=("X","C","P","A")\mathcal{O}(\mathrm{b}) = (\text{"X"}, \text{"C"}, \text{"P"}, \text{"A"}) against O(s)=("K","B","Q","D")\mathcal{O}(\mathrm{s}) = (\text{"K"}, \text{"B"}, \text{"Q"}, \text{"D"}), but both are indexed by the same variable, so a reading transports even though a token does not.

However, not every variable is decisive in every context. Consider an entity-attribute task, where the prompt selects which attribute is being asked about the introduced entity. In such a task, interventions should alter the model’s behavior only when the base prompt queries the sought attribute (e.g. in the RAVEL task, intervening on the entity’s continent should change the answer to a continent question while leaving a language question untouched). Such cases can be easily covered by a gated rule, τ=γ(b)q(s)+(1−γ(b))q(b)\tau= \gamma(\mathrm{b})q(\mathrm{s}) + (1-\gamma(\mathrm{b}))q(\mathrm{b}), where γ(b)∈{0,1}\gamma(\mathrm{b}) \in\{0,1\} records whether the intervened variable is the operative one for that prompt; the target is the source’s reading when it is, and the base’s own reading, i.e. a no-op, when it is not. More generally, T\mathcal{T} may be given as a table specifying, for each pair of source and base values, the reading that should result. We refer to Appendix § 7.§ 3 for a complete description of the programs used in our experiments.

Semantic programs are label-free. A semantic program P\mathcal{P} is written before any training and consists of Eqs. (3) and (5): a family of candidate answers and a rule for moving between readings of them. It contains no correct answer, no counterfactually annotated example pair, and no executable high-level model. Everything numerical in the target of Eq. (5) is measured from the frozen subject through Eq. (4). Training on a program therefore needs no ground-truth label at any stage: base and source are paired at random, and every target is read from the subject, including on prompts it answers wrongly. This lets Ψ\Psi localize a variable on tasks for which no gold answer is available (§ 3.2). Appendix Table 2 accounts for what each approach asks of the investigator, and § 3 measures what each input is worth.

Learning the alignment from the program

A semantic program P\mathcal{P} determines a training signal without any further input. Given a collection of base prompts B\mathcal{B}, we can construct candidate base-source interventions ι=(b,s)\iota= (\mathrm{b}, \mathrm{s}) by pairing via a random derangement within a group of prompts the program declares comparable. The transport rule of Eq. (5) then produces a target reading τ(b,s)\tau(\mathrm{b},\mathrm{s}), while the interchange of Eq. (2) produces an activation hb←sh_{\mathrm{b}\leftarrow\mathrm{s}} whose library log-probabilities ρ(hb←s)\rho(h_{\mathrm{b}\leftarrow\mathrm{s}}) we can measure, by completing the subject’s forward pass from the edited activation as we write ρ(h)\rho(h), q(h)q(h) and Z(h)Z(h) for the quantities of Eq. (4) measured this way. Learning the subspace AA is then a matter of making the second agree with the first:

Ldonor(A)=Eb,s d(ρ(hb←s),τ(b,s)).(6)\mathcal{L}_{\mathrm{donor}}(A) = \mathbb{E}_{\mathrm{b},\mathrm{s}}\, d\left(\rho(h_{\mathrm{b}\leftarrow\mathrm{s}}), \tau(\mathrm{b},\mathrm{s})\right). \tag*{(6)}

(6) is a differentiable relaxation of Eq. (1): the indicator that asks whether two outputs match exactly is replaced by a divergence dd that measures how far apart two readings are, and the expectation is taken over sampled prompt pairs. One natural choice for dd is the cross-entropy of the target against the model’s log-probabilities [11]. Because the ρi\rho_i are log-probabilities over the whole vocabulary rather than over the library, that quantity decomposes as

−∑iτiρi=H(τ)+KL⁡(τ∥q(hb←s))⏟the reading−log⁡Z(hb←s)⏟the mass,(7)-\sum_i \tau_i \rho_i = H(\tau) + \underbrace{\operatorname{KL}(\tau\parallel q(h_{b\leftarrow s}))}_{\text{the reading}} - \underbrace{\log Z(h_{b\leftarrow s})}_{\text{the mass}}, \tag*{(7)}

where HH is the entropy of the target and KL⁡\operatorname{KL} the Kullback–Leibler divergence. The first group asks the intervened model to produce the requested reading; the second asks it to place as much probability as possible inside the library. The CONDITIONAL form drops the mass term and scores the reading alone; the REFERENCE form targets the mass Z(b)Z(\mathrm{b}) the unintervened prompt already shows, so an intervention is asked to preserve it; the FULL form keeps Eq. (7) as written. We used the FULL form for the Arithmetic localization experiments and CONDITIONAL for the rest (Table 1), while Appendix § 7.5 measures the impact each choice has on control costs.

Table 1: Localizing causal variables on MIB. Interchange intervention accuracy (%, higher is better): the mean over the depth profile, with the best layer in parentheses. Ψ\Psi uses the rank the DAS baseline uses at each cell. Bold is the best method in a column, mean and best layer separately, and the gray rows are nulls that do not compete for it. Baselines from Mueller et al. [19].

ARC (Easy)
Methodgemma-2 2b OAnswerO_{\mathrm{Answer}}gemma-2 2b XOrderX_{\mathrm{Order}}llama-3.1 8b OAnswerO_{\mathrm{Answer}}llama-3.1 8b XOrderX_{\mathrm{Order}}
Full Vector63 (100)43 (74)68 (100)47 (72)
Random18 (27)33 (33)17 (28)33 (33)
DBM82 (99)63 (80)85 (100)69 (82)
+ PCA78 (98)64 (81)84 (100)72 (83)
+ SAE70 (89)54 (70)74 (94)55 (67)
DAS88 (94)76 (88)88 (99)74 (84)
Ψ\Psi (ours)95 (98)71 (89)97 (100)80 (92)
Arithmetic (+)†
Methodgemma-2 2b XCarryX_{\mathrm{Carry}}llama-3.1 8b XCarryX_{\mathrm{Carry}}
Full Vector29 (35)35 (45)
Random26 (26)26 (26)
DBM33 (43)47 (58)
+ PCA32 (44)37 (56)
+ SAE32 (44)38 (55)
DAS31 (35)54 (65)
Ψ\Psi (ours)38 (46)52 (64)
MCQA
Methodgemma-2 2b OAnswerO_{\mathrm{Answer}}gemma-2 2b XOrderX_{\mathrm{Order}}llama-3.1 8b OAnswerO_{\mathrm{Answer}}llama-3.1 8b XOrderX_{\mathrm{Order}}qwen-2.5 0.5b OAnswerO_{\mathrm{Answer}}qwen-2.5 0.5b XOrderX_{\mathrm{Order}}
Full Vector61 (100)44 (77)77 (100)46 (68)35 (99)49 (99)
Random19 (28)33 (33)16 (31)33 (33)7 (18)33 (33)
DBM84 (99)63 (84)86 (100)66 (73)46 (94)60 (99)
+ PCA57 (96)52 (81)65 (99)53 (74)22 (76)54 (100)
+ SAE73 (90)51 (65)80 (99)58 (65)——
DAS95 (97)77 (93)94 (100)77 (91)86 (95)78 (100)
Ψ\Psi (ours)94 (99)76 (92)98 (100)80 (91)92 (98)75 (100)

Table 1. Localizing causal variables on MIB. Interchange intervention accuracy (%, higher is better): the mean over the depth profile, with the best layer in parentheses. Ψ\Psi uses the rank the DAS baseline uses at each cell. Bold is the best method in a column, mean and best layer separately, and the gray rows are nulls that do not compete for it. Baselines from Mueller et al. [19].

Enabling control via the same interface

A trained subspace that minimizes Eq. (6) supports interchange interventions, which realize the reading of whichever source prompt is supplied. But one crucial advantage of the reference frame of Eq. (3) is that it represents a coordinate system with known semantics. Such a coordinate system can thus be used to naturally formulate any representable reading target q⋆q^\star, while the transport rule T\mathcal{T} can then be used to request such a reading from the model when exposed to a base prompt b\mathrm{b}, i.e. construct τ⋆=T(q⋆,q(b),γ)\tau^\star= \mathcal{T}(q^\star, q(\mathrm{b}), \gamma). In particular, we can attach a controller:

ζ=control⁡(τ⋆,q(b),Ahb),(8)\zeta= \operatorname{control}(\tau^\star, q(\mathrm{b}), Ah_{\mathrm{b}}), \tag*{(8)}

which maps a requested reading τ⋆\tau^\star, the subject’s current reading q(b)q(\mathrm{b}), and the base’s carrier coordinates AhbAh_{\mathrm{b}} to an edit ζ∈Rr\zeta\in\mathbb{R}^r expressed in those same carrier coordinates. The steered activation is then hb←ζ=hb+ATζh_{\mathrm{b}\leftarrow\zeta} = h_{\mathrm{b}} + A^{\mathsf{T}}\zeta: by construction the edit lives entirely inside the identified subspace and leaves the complement (I−π)hb(I-\pi)h_{\mathrm{b}} untouched, exactly as the interchange of Eq. (2) does.

The simplest instance, and the one we use in our experiments, is a linear gain driven by the residual between the requested and the current reading. Writing lo⁡(p)∈Rk−1\operatorname{lo}(p) \in\mathbb{R}^{k-1} for the log-odds of a reading against its last entry, lo⁡i(p)=log⁡pi−log⁡pk\operatorname{lo}_i(p) = \log p_i - \log p_k, the gain controller is:

ζ=G[lo⁡(τ⋆)−lo⁡(q(b))],G∈Rr×(k−1),(9)\zeta= G\left[\operatorname{lo}(\tau^\star)-\operatorname{lo}(q(\mathrm{b}))\right], \qquad G \in\mathbb{R}^{r\times(k-1)}, \tag*{(9)}

so that GG is the only learned object and the edit vanishes whenever the request coincides with what the model is already doing. We clip the edit to a fixed budget, ζ←ζmin⁡(1,c/∥ζ∥)\zeta\leftarrow\zeta\min(1,c/\lVert\zeta\rVert), with cc set to a high quantile of the displacements that interchange interventions themselves produce at that site (Appendix § 7.15).

The controller is trained jointly with the subspace, against requests drawn half from the readings other prompts exhibit and half uniformly from the probability simplex, so that it is exercised on targets no prompt in the dataset elicits. With dd the same divergence as before, the complete objective is

L(A,G)=Eb,sd(ρ(hb←s),τ(b,s))⏟donor: the subspace carries the variable+λEb,τ⋆d(ρ(hb←ζ),τ⋆(b,q⋆))⏟control: the subspace is writable.(10)\mathcal{L}(A,G) = \mathbb{E}_{\mathrm{b},\mathrm{s}}\underbrace{d\left(\rho(h_{\mathrm{b}\leftarrow\mathrm{s}}),\tau(\mathrm{b},\mathrm{s})\right)}_{\text{donor: the subspace carries the variable}} + \lambda\mathbb{E}_{\mathrm{b},\tau^\star}\underbrace{d\left(\rho(h_{\mathrm{b}\leftarrow\zeta}),\tau^\star(\mathrm{b},q^\star)\right)}_{\text{control: the subspace is writable}}. \tag*{(10)}

The two terms share the abstraction AA, which is what makes the training joint: the donor term asks the subspace to behave like the variable under transplanation, and the control term asks the same subspace to be reachable by a learned edit. At inference, the controller of Eq. (8) accepts any distribution over the declared library, including readings q⋆q^\star no prompt in the dataset elicited, such as an even split between two answers, with no source prompt and no retraining. This is a capability methods trained on the interchange of Eq. (2) ([11, 33, 13, 28]) do not offer directly, since their interchange operator requires a donor. The controller is thus a steering method, which writes only inside a subspace that has first passed the interchange test. In Section 3.3 we therefore equip their frozen carriers with a controller of our own and measure what that recovers.

Results

Protocol. We evaluate on the causal-variable tasks of Mueller et al. [19], with the benchmark’s subjects gemma-2 2b, llama-3.1 8b and qwen-2.5 0.5b, and with gpt2-small for IOI. At each site we set the carrier rank to the subspace dimension the official DAS baseline uses there, so the two are compared at equal capacity, and we follow the benchmark’s site-selection protocol: at each layer we take the best of the benchmark’s fixed token positions, then report the mean of that depth profile together with its best layer. We also include a random subspace of the same rank at the same sites as a floor. Localization runs use λ=0\lambda= 0 in Eq. (10), while the control experiments of § 3.3 use λ=1\lambda= 1 and focus on gemma-2 2b at two sites. Reported localization numbers are single-seed, following Mueller et al. [19] (Appendix § 7.14); at the one site (MCQA XOrderX_{\mathrm{Order}}, gemma-2 2b layer 17, last token) where we repeated the experiment three times, the spread was ±0.9\pm0.9 points, comparable to one evaluation pair (0.8 pt).

Localizing causal variables

Table 1 reports interchange intervention accuracy against the benchmark’s published baselines. DAS is the closest method to our approach: it also fits a rotation of the residual stream at a fixed rank, but it is given an executable high-level model and a curated list of base–source pairs whose counterfactual answer is known in advance. Ψ\Psi is given neither, and at its best layer it matches or exceeds DAS at all but two cells, trailing by about a point in both. These margins also reflect data use and training budget: the published baselines were trained on the curated pairs only and with fewer updates, and at matched data and budget Ψ\Psi and DAS perform on par (§ 3.1).

On Arithmetic1, the depth mean puts Ψ\Psi ahead on gemma-2 2b (38 against 31) and close behind DAS on llama-3.1 8b (52 against 54). On RAVEL (Appendix A.12, Table 8), Ψ\Psi trails DAS in all measured models and variables and leads the other published methods in all but one cell.

Supervision ablation. A counterfactual dataset supplies two things: a curated pairing of each base prompt with a source, and the oracle answer that pair should produce. We focused on the MCQA task and the gemma-2 2b model, and varied the two ingredients independently at one site (layer 17), holding rank, optimizer and budget fixed, and restricting the unpaired arms to the 108 observations the benchmark’s pair list uses as bases so that all four cells see the same prompts. Holding the pair list’s own rows, its base multiplicities and its 544 optimizer updates, and permuting only which source accompanies which base, costs 0.07 points (89.94 against 89.87) when the target is the subject’s own reading, and 1.34 (91.40 against 90.06) when it is the oracle answer. The coupling a counterfactual dataset supplies is therefore worth little on its own. What it does supply is a donor pool: drawing donors from the same rows instead of from the pair list’s distinct sources costs 5.63 points (84.24). A derangement needs no pair list and can therefore draw on all 383 eligible observations, where the curated arms are confined to the 108 the pair list uses as bases; doing so reaches 90.84 ±\pm 0.9 against 91.40 ±\pm 0.5 for the fully supervised cell. We confirmed this observation by training both Ψ\Psi and DAS with matched data (same donor pool size) and training budget, and observed that the two methods perform on par (Appendix A.8). Additionally, we removed the correctness filter, so that no gold label enters training or checkpoint choice (Appendix A.7): this costs Ψ\Psi 0.9 points on gemma-2 2b (90.24 to 89.34), and the model’s distribution remains the best of the training targets we compared, on both models and with or without the filter. We conclude that the loss of pairing supervision is compensated by a larger donor pool, and that replacing the gold-label target with the model’s own distribution results in better subspace identification when the model answers incorrectly (Section 3.2).

Transfer across answer encodings. The MCQA program applies unchanged to any labelling of the options, since its library is the prompt’s own labels by position (Section 2.1). A carrier trained on one labelling can therefore be tested on another without retraining: we froze the carriers trained at the site above and re-encoded the options of donor and recipient independently (Appendix A.10). With lowercase labels on both sides, every learned carrier reaches 100.0%. Digit labels show an asymmetry: a digit-labelled donor moves a recipient with the original labels to the donor’s position on 92.4–95.8% of pairs, while a digit-labelled recipient follows the donor on only 45.3–49.3%, although the model answers digit-labelled questions correctly on 95.0% of prompts. The carrier therefore holds the position in a form that does not depend on the donor’s labels, and digit-labelled prompts do not rely on it alone to produce their answer; Ψ\Psi, one-hot targets and DAS at the same budget behave alike.

Supervision on natural questions

We then considered the natural four-choice science questions from ARC (Easy), where the four option labels are again an easy library with simple transport rules, but answering requires knowledge that no short algorithm provides, i.e. explicitly writing down a high-level causal model HH is conceptually hard. We explored four different training targets: the model’s distribution over the four labels (Ψ\Psi); a control that keeps the model’s top option and its probability and spreads the remainder evenly over the other three; a one-hot label on the model’s top option, which amounts to DAS with labels read from the model; and a one-hot gold label. Notably, only the last configuration makes use of gold labels, while the others rely on the model’s own answers. We train on all 2241 training questions, of which gemma-2_2b answers 79.7% correctly, at layer 17, last_token, rank 16, with donors drawn from other questions, and test on 589 pairs of held-out questions and measure how closely the swapped recipient reproduces the donor’s own distribution and answer (Figure 3; Appendix § 7.11).

Figure 3. ARC science questions with different targets. Natural ARC questions, gemma-2_2b, layer 17, last_token, rank 16; 2241 unfiltered training questions. (a) Top-option probability of the unedited model on training questions it answers correctly (bars) and wrongly (line); dashed lines are the means. (b) KL from the donor’s reading to the edited recipient’s, on 589 held-out pairs. Rows give the training target: the model’s distribution (soft, Ψ\Psi); its top option and probability, with the rest spread evenly (confidence); one-hot on its top option (argmax); one-hot on the gold label (gold). (c) Greedy agreement of the edited recipient with the donor’s own answer, on all pairs (filled) and on the 111 pairs whose donor the model answers wrongly (hollow). Markers are three seeds (mean and SD).

The model’s distribution is far from one-hot even when its answer is correct (Figure 3a): on those questions, its top probability over the four labels averages 0.78, and only 28.7% of all questions exceed 0.9. A one-hot target therefore differs meaningfully from the model’s state even where the model is right, which is why we train on the model’s distribution and on its top option as separate targets. When trained on the model’s distribution (Ψ\Psi), the swapped recipient gives the donor’s answer on 94.8% of pairs, and its distribution is closer to the donor’s (KL 0.020) than with one-hot model labels (0.117) or gold labels (0.090; Figure 3b). The source of the target matters most where the model is wrong (Figure 3c). On the 111 pairs whose donor question the model answers wrongly, the recipient takes the donor’s answer on 85.9% of pairs with Ψ\Psi, 79.3% with one-hot model labels and 72.1% with gold labels. We argue that a gold label on such a question describes the task, while an interchange asks about the model’s state, so the subspace a gold label trains carries that state less well. The confidence control reaches 84.7%: most of the benefit comes from keeping the model’s confidence, and the rest of its distribution adds some fidelity in KL. Lastly, on the benchmark’s own score, which compares the output with the gold counterfactual answer on correctness-filtered pairs, the gold-trained subspace is ahead (71.45 against 70.03), as expected for a score that rewards the correct answer.

Controlling a variable without a donor

Correct localization does not by itself guarantee control [7], and a carrier trained only to transport readings between prompts need not be writable to a reading that no prompt exhibits. We test this by training the carrier and the controller jointly, through the full objective of Eq. (10), at two sites in gemma-2_2b: MCQA XOrderX_{\mathrm{Order}} at layer 17, where the declared library has four entries, and Arithmetic XCarryX_{\mathrm{Carry}} at layer 21, where it has two. A reading qq encodes a probability distribution over the declared library O\mathcal{O}, and geometrically it is a point in the corresponding probability simplex.

Its corners are the requests with an unambiguous right answer, i.e. conditions for which the model should answer as though the variable took value vv, while a generic request q∗q^{*} is a point in the interior of the simplex. Note that for such a request there is typically no prompt in the data that exhibits that reading, and hence no donor that could provide it. During training, half the requests are readings other prompts exhibit and half are uniform draws from the simplex; we evaluate on held-out corner requests, where compliant steering behavior is well-defined. We report what the subject emits under unrestricted decoding (Figure 4a,Figure 4b): requests are realized 94.8% of the time at the MCQA site and 63.7 ± 2.6% at the arithmetic one, above the donor interchange at both, which on arithmetic reaches 50.8%, at the 50% chance floor. Appendix § 7.§ 4 compares the two decision rules. The same controller inside a rank-matched DAS, principal or random subspace reaches 81.7%, 83.9% and 34.2%. Accuracy, however, only asks which answer wins, which fails to capture the nuance of a generic request q∗q^{*}. In panel (c) we thus report the total variation distance to the requested reading on the interior requests, which no donor exhibits, where Ψ\Psi reaches 0.10 against 0.18 for an interchange and 0.44 for no edit.

Figure 4. A declared reading can be requested. Rows are shared across panels. The first three train Ψ\Psi jointly under each mass mode; Ψ\Psi frozen (trained with λ=0\lambda= 0), PCA, DAS and random attach the same controller to a frozen carrier of the same rank; donor interchanges, inside the frozen Ψ\Psi carrier, the training prompt whose reading is nearest the request; none applies no edit. (a, b) Held-out corner requests realized at MCQA XOrderX_{\mathrm{Order}} layer 17 and Arithmetic XCarryX_{\mathrm{Carry}} layer 21 (gemma-2 2b); filled dots are greedy decoding and hollow ones the declared-library rule, on the same recipients. (c) Total variation to the target on interior requests, at the MCQA site. Seed whiskers (three seeds) mostly fall within the dots; the dashed line (search) optimizes each edit separately.

Discussion

A semantic program P\mathcal{P} replaces the executable high-level model H\mathrm{H} that causal-variable methods are supervised by. In our experiments we showed that such an interface can successfully identify a low-rank causal subspace with Ψ\Psi showing competitive performance against available baselines (Table 1), on par with the supervised alternatives at matched data and budget. Our supervision ablation investigation (Figure 2) on the MCQA task corroborated these benchmark findings by revealing how the majority of the performance benefits actually stem from a net increase in the available donor pool size, with explicit counterfactual pairing only contributing modest improvements. Because the labels are read from the model, Ψ\Psi also needs no ground truth: on natural questions, the resulting subspace tracks the model’s own answers, wrong ones included, better than one trained on gold labels (Figure 3), which opens interchange-based localization to tasks where no labels exist. We showed that joint localization and control training yields better control performance (Figure 4), both in terms of explicit control successes and request errors, than than a controller built on top of a frozen localization-only carrier, which is in line with previous observations that explicitly differentiate the two concepts [7].

Figure 2. What a counterfactual dataset buys. MCQA XOrderX_{\mathrm{Order}}, gemma-2 2b, layer 17; mean and standard deviation over three seeds. The two ingredients of counterfactual supervision are varied independently: how base and source are paired (color) and what the intervened model is trained towards (horizontal axis). The oracle answer helps the curated arm and hurts the deranged one, so an oracle label is worth having together with the oracle’s pairing. The star trains the deranged arm on all 383 eligible observations, which a derangement can use because it needs no pair list.

Limitations. First, the semantic programs in this paper were easy to write. Their tasks have small, explicit answer sets, and for a more abstract variable it may be hard to name a library that is both semantically clean and broad enough to hold much of the model’s probability. How much a library saves also depends on the variable: for the carry in addition, writing it requires solving the task. Moreover, targets read from the model are only as good as the model’s distribution over the library: on RAVEL, where this distribution is diffuse, Ψ\Psi trails DAS (Appendix § 7.12). Another limitation is that interchange accuracy by itself is not a guarantee of identification: a subspace that supports the declared transport τ\tau need not be unique; moreover, a subspace can move the output through a pathway that has nothing to do with the variable it is supposed to name ([16]). Finally, gold answers are used to filter the training pool of Table 1, as they are for the published baselines ([19]), although removing the filter costs Ψ\Psi less than a point (§ 3.1).

AI Use Statement

In this work we have used generative AI tools to refine hypotheses, provide feedback on experiments and assist the experiment lifecycle (e.g. submitting cluster jobs, writing scripts to collect results). We have not used AI tools to design the research, write the core Ψ\Psi algorithm, support qualitative and thematic data analysis, interpret results, generate synthetic datasets or assist with translation; the remaining tasks (e.g. theoretical proofs) are not applicable to this work. Additionally, we used generative AI tools to create or modify scientific figures, draft sections of the manuscript, write supporting code (e.g. tests, evaluation scripts), and summarize existing literature and identify gaps. We have reviewed all AI-assisted work, inspecting the code and writing and editing the full manuscript ourselves. We take responsibility for the final content of this work, including text, claims or artifacts produced with the aid of generative AI.

Reproducibility Statement

Appendix A.3 gives the complete semantic program for every task, which together with the objective of § 2.2 and the public MIB dataset ([19]) is everything an implementation needs to reproduce our results.

Appendix

Additional related work

The distributed alignment lineage. DAS learns a rotation whose coordinate block aligns with a high-level variable [11], Boundless DAS learns the subspace boundary as well so the rank need not be fixed by search [33], MDAS optimizes attribute isolation alongside the alignment on RAVEL [13], and HyperDAS replaces the brute-force search over feature locations with a hypernetwork [28]. All four take their training target from an executable high-level model, which is the input a semantic program replaces. HyperDAS in particular automates where to look; the declaration changes what the investigator has to write down, and the two are independent.

Writing a behavior into activations. Representation engineering (Zou et al., 2023) and activation addition [31, 23] add to the residual stream a direction fitted from contrastive prompts, and inference-time intervention does the same along a truth-related direction [15]. Recent work learns the write operator instead of fitting it. CDAS matches the intervened output distribution against the model’s own distribution on concept-eliciting prompts and clamps a rank-1 coordinate at inference [2]; a single causally identified direction acts as a context-sensitivity knob across a model family [17]; LinEAS fits per-layer affine maps against a distributional loss over activation populations [24]; a probe-free low-rank map pushes an activation towards a manifold of desirable content [14]; and UniSteer conditions a flow in activation space on a natural-language description of the wanted behavior [26]. Each of weak supervision, model-relative targets, donor-free inference and amortization over a space of requests is therefore established on its own. What a semantic program adds is a request that is a distribution over a declared answer set indexed by a causal variable’s values, issued to a subspace that interchange interventions have independently validated.

The write operator itself is LoReFT’s, i.e. project into a low-rank subspace, compute a target there, add the residual back [34]. Ours takes the requested reading as an argument, so one training run covers the whole simplex rather than one behavior.

Localization and control are different measurements. Detection directions and steering directions are close to orthogonal [7], and the methods that win at concept detection and at steering are not the same ones [35]. The geometry says why: the directions that read a concept off an activation and the directions that change it are related by the causal inner product and not by the Euclidean one [21]. Steering vectors are in addition variable across inputs and brittle to reasonable changes of prompt [30], and a subspace intervention can move the output through a dormant parallel pathway while the feature it names does nothing [16]. Intervention success as an evaluation criterion is itself established [4]; we use it as a second measurement on one identified carrier rather than as a faithfulness criterion in its own right.

Supervision accounting

The row order runs from the most to the least that a method asks of the investigator before training begins. Every entry is an input, so a shorter row is a cheaper method to apply, and § 3.1 measures what the entries in the DAS row are worth in accuracy.

The semantic program for each task

Everything the investigator writes for a task is below. Each program is an outcome library O\mathcal{O} and a transport rule T\mathcal{T}, fixed before training and shared by every run on that task. Where a rule needs the source’s reading and the source prompt does not supply one, we also give the query that measures it.

MCQA and ARC (Easy). Both are multiple-choice, and both variables use the full-change rule T=q(s)\mathcal{T}=q(s). They differ in what the library names and in how wide it is. For XOrderX_{\mathrm{Order}}, the position of the correct option, the library is the prompt’s own choice labels in the order that prompt lists them, each with its leading space, indexed by option position; an interchange that moves the variable moves the model from one label to another. For OAnswerO_{\mathrm{Answer}}, the answer symbol, the library is the twenty-six leading-space letters ( A, …, Z), identical for every prompt. The two variables therefore do not share a library: one has four entries drawn from the prompt, the other twenty-six fixed ones, which is what lets a single benchmark task carry two variables at different widths.

Table 2. Supervision accounting. Each column is an input the investigator must provide before training begins, and the entries say what that input is: the methods differ in kind, not only in quantity. The DAS lineage {(citep}(Geiger et al., 2024; Wu et al., 2023; Huang et al., 2024)} shares one row because it asks for one thing. PSI replaces the algorithmic specification with a declaration in the answer space and the oracle counterfactual with a measurement; gold answers still enter, as the correctness filter on the training pool that the published baselines also apply. Remaining rows: HyperDAS {(citep}(Sun et al., 2025)}, CDAS {(citep}(Bao et al., 2026)}, LoReFT {(citep}(Wu et al., 2024)}.

MethodHigh-level modelIntervention targetPairingGold labels
DAS, Boundless DAS, MDASalgorithmoracle answercuratedtarget, filter
HyperDASalgorithm, textoracle answercuratedtarget, filter
CDASconcept promptsown distributioncontrast setsset membership
LoReFTnonetask labelnonetraining target
Ψ (ours)library, transportown readingderangementfilter

Arithmetic (+). The variable is the carry out of the units digits, so the library has two entries: the sum with the carry propagated and the sum without it. For 47 + 28 = these are 75 and 65. Both are computed from the operands the prompt states and not from the answer, but writing that library does take knowing what a carry does to a sum. The program supplies the mapping from this variable’s values to the strings they produce, and nothing else about how the subject computes them. The rule is again the full change. The carry is a natural variable to declare: it is the intermediate the benchmark itself targets {(citep}(Mueller et al., 2025)}, transformers trained on addition compute it explicitly {(citep}(Quirke & Barez, 2024)}, and causal mediation analysis has localized where language models route the operands of such sums {(citep}(Stolfo et al., 2023)}.

RAVEL. The library is built once from the benchmark’s entity file: every value any city takes on any of the three attributes, reduced to its first token under the subject’s tokenizer and deduplicated, so a single library of country, continent and language names serves all three variables. Entries are therefore first tokens rather than complete attribute names, and two values sharing a first token are one entry. The rule is gated, T=γ(b) q(s)+(1−γ(b)) q(b)\mathcal{T}=\gamma(\mathbf{b})\,q(\mathbf{s})+(1-\gamma(\mathbf{b}))\,q(\mathbf{b}), with γ(b)=1\gamma(\mathbf{b})=1 when the base prompt queries the attribute being intervened on and γ(b)=0\gamma(\mathbf{b})=0 otherwise. That single bit is the whole of what RAVEL’s isolation requirement asks for: change the answer when the question is about the attribute you moved, leave it alone when it is not.

RAVEL is also the one task where the source’s reading cannot be taken from the source prompt. It has two prompt families: JSON-style continuations such as {"city": "Nagoya", "language": ", which name an attribute, and free text such as Spartak Trnava won 3−1 on aggregate, which names none. The program therefore declares the query that measures the source’s value,

Q: What is the {attribute} of {entity}?
A:

evaluated on the frozen subject with {attribute} the one being intervened on. This is the declared query of Eq. (5): one extra forward pass per source observation, and the only place a program in this paper reaches beyond the prompts it was given.

IOI. We consider the IOI task an extension rather than an instance of the interface, and we separate it because its transport rule is not measured from the subject, due to the particular implementation of this task in the MIB benchmark itself. The benchmark scores the task as a signed logit margin between the indirect object and the subject name, and supplies a linear model of that margin, bias + βtokstok+βposspos\beta_{\mathrm{tok}}s_{\mathrm{tok}} + \beta_{\mathrm{pos}}s_{\mathrm{pos}}, with fixed coefficients and each signal +1 when that variable still matches the base and −1 when it does not. We build the transport table by evaluating that model under the isolated intervention and passing the resulting margin through a sigmoid, which gives the probability the indirect object should carry; the library is the two names. The numbers in an IOI target therefore come from the benchmark’s high-level model and from the gold identity of the base’s indirect object, not from the frozen subject. Every claim in this paper about targets measured from the subject covers MCQA, ARC, arithmetic and RAVEL; it does not cover IOI, and Appendix § 7.12 reports that task on those terms.

Unused supervision. With the exception of IOI above, no program names a correct answer, and none names which library entry a given prompt should elicit. Gold answers enter the other four tasks outside the program, as the filter that keeps only observations the subject already answers correctly, and through the benchmark’s own choice of cut position, which is located by the correct answer symbol. Both are conventions the published baselines also follow, and the experiments of § 3.2 use neither.

Two decision rules

The benchmark’s evaluator decodes from the intervened model over the whole vocabulary and asks whether the expected answer appears. Our training objective works instead on the declared library, so we also audit with the rule the objective optimizes: restricted to the library, does the intervened model prefer the entry the transport rule asked for? Comparing the two tells us whether a score reflects the variable moving or the decode landing elsewhere.

The two agree closely wherever the declared answers are single tokens. They come apart on arithmetic, where an answer spans three tokens: at layer 19 the carrier scores 46.3 under the benchmark’s rule and 71.3 under ours. The carry does move, and the rollout that follows it often lands on a number outside the declared pair. Every localization number in the main text uses the benchmark’s rule; § 3.3 reports both.

The mass term

Equation (7) splits the training signal into the shape of the reading and the probability mass the library holds, and § 2.2 leaves open what to ask of the second term. We measured all three options at one site over three seeds to choose between them. Dropping the term (CONDITIONAL), aiming it at the mass the unintervened prompt already shows (REFERENCE), and maximizing it (FULL) give 90.84 ± 0.9, 91.09 ± 1.0 and 89.02 ± 0.0 on MCQA XOrderX_{\mathrm{Order}}. FULL is the only mode that pays to keep probability on the declared answers, which is what should stop an undeclared token winning a decode, and it scores lowest here, so we use CONDITIONAL for every task except Arithmetic. There the answers span several tokens, and preserving the mass on the digits benefits greedy decoding: at the gemma-2-2b XCarryX_{\mathrm{Carry}} site (layer 20, three seeds) FULL reaches 45.94 ± 0.1 against 45.31 ± 0.2 for CONDITIONAL under the benchmark’s rule, even though it trails under the declared one (68.9 against 73.1). The control experiments show the same ordering under greedy decoding (Figure 4b and below), so the Arithmetic localization runs use FULL.

The same choice is equally important for control, where the two terms of the decomposition pull against each other. At the arithmetic site Ψfull\Psi_{\mathrm{full}} keeps 0.661 of the declared mass after the edit against 0.291 for Ψcond\Psi_{\mathrm{cond}}, and pays 15.6 points of control success under the declared rule for it (77.3% against 93.2 ± 4.3%), and buys it back under free decoding, where the order reverses (75.0% against 63.7 ± 2.6%): the mass CONDITIONAL gives up is mass the undeclared tail takes. Asking the model to keep talking about the declared answers and asking it to change which one it gives are different requests, and the decomposition is where a program states which of the two it wants.

What the control term costs

Carrier cost. The two terms of Eq. (10) share the subspace AA, so it is reasonable to expect them to compete: a subspace shaped to be the writable need not be the best carrier under transplantation. We measured the effect at the MCQA XOrderX_{\mathrm{Order}} site of gemma-2-2b, training three seeds either for localization alone (λ=0\lambda= 0 in Eq. (10)) or jointly (λ=1\lambda= 1). Seed for seed, turning the control term on lowers interchange accuracy by 1.9 points and raises control success by 7.9. The cost is small but consistent, and it is why the benchmark evaluations of § 3.1 use λ=0\lambda= 0 while the control experiments use λ=1\lambda= 1.

The joint carrier is also trained for longer than the frozen ones it is compared against, so we checked whether its advantage is the training budget. Attaching the same controller to donor-only carriers trained for 8, 32 and 64 epochs gives (under the declared rule) 89.6%, 90.4% and 90.2% of requests realized, against 97.5% for the jointly trained carrier on the same recipients (all four with seed 11, whereas Figure 4 reports the three-seed mean). Four to eight times the carrier budget is worth under a point and then saturates, so we attribute the gap to full joint training.

Collateral drift. The control term also competes with the rest of the model’s behavior and altering the model’s preferences within the declared library O\mathcal{O} might come at the expense of the overall probability mass outside the library, i.e. on O⊥\mathcal{O}^{\perp}. Figure 5 plots the measured controlled accuracies for various methods against a measure of collateral control cost, revealing a Pareto-like tradeoff between control efficacy and intervention cost.

Figure 5. Cost against success for the arms of Figure 4, seed 11, the same held-out recipients and the same corner requests, so the two axes are comparable point by point. (a) MCQA XOrderX_{\mathrm{Order}}: collateral drift is the total-variation distance between the undeclared tails of the output distribution before and after the edit, each renormalized, and the dashed line marks what a donor interchange costs at the same site. (b) Arithmetic answers span several tokens, so the undeclared tail is ill defined there and the cost axis is the probability mass the declared library retains.

We quantified the collateral cost differently between the MCQA task (Figure 5a) and the arithmetic task (Figure 5b), given that what can be measured differs by site. MCQA’s declared outcomes are disjoint single tokens, so the undeclared tail is well defined and drift can be read over the whole vocabulary; we thus quantified the cost as the total variation between the tail of the distribution before the edit and the tail of the distribution after the edit. Arithmetic’s answers are three tokens long, and there the available axis is how much probability the declared library still holds after the edit.

Correctness filter

The training pools of the main experiments keep only prompts the model answers correctly. Table 3 trains the same carrier on that filtered pool and on all prompts, with three training targets, a fixed budget of 768 updates and the final checkpoint, so no gold label enters through checkpoint selection. Without the filter, the soft-reading carrier scores 0.9 points lower on gemma-2-2b and 1.8 points lower on qwen-2.5-0.5b under the official evaluator.

Table 3: Correctness filter and training target. MCQA XOrderX_{\mathrm{Order}}, layer 17, last_token, rank 16; 768 updates, final checkpoint, seeds 11–13 (mean ± SD). Filtered: training prompts the model answers correctly (383/440 for gemma-2 2b, 360/440 for qwen-2.5 0.5b); unfiltered: all prompts. On the filtered pool the model’s argmax is the gold label, so the two arms are the same runs. Official: the benchmark’s evaluator (122 eligible pairs). Agreement: edited argmax equals the donor model’s argmax; KL: donor reading to edited reading over the four options; both on the same 150 test pairs for every arm.

PoolTargetOfficial (%) ↑Agreement (%) ↑KL ↓
gemma-2 2b
filteredsoft reading (Ψ)90.24 ± 0.5282.22 ± 0.380.103 ± 0.001
filteredmodel argmax = gold88.55 ± 0.4979.56 ± 1.020.277 ± 0.009
unfilteredsoft reading (Ψ)89.34 ± 0.5282.22 ± 1.390.102 ± 0.004
unfilteredmodel argmax86.44 ± 0.5179.56 ± 0.380.149 ± 0.009
unfilteredgold label87.59 ± 1.4378.22 ± 1.020.280 ± 0.006
qwen-2.5 0.5b
filteredsoft reading (Ψ)99.57 ± 0.7482.00 ± 0.000.194 ± 0.002
filteredmodel argmax = gold96.65 ± 0.6980.00 ± 0.001.030 ± 0.019
unfilteredsoft reading (Ψ)97.78 ± 2.9682.44 ± 1.020.180 ± 0.002
unfilteredmodel argmax96.99 ± 2.8282.89 ± 0.770.298 ± 0.007
unfilteredgold label97.08 ± 1.6079.78 ± 0.771.141 ± 0.044

Table 3.

Matched training budget

Table 4: Matched training budget. MCQA XOrderX_{\mathrm{Order}}, gemma-2 2b, layer 17, last_token, rank 16, seeds 11–13. Every arm takes 768 optimizer updates over 12,256 training examples with the same batch sizes, and is scored at its final checkpoint. The Ψ arms use the 383 correctly answered training prompts with deranged donors; DAS uses the benchmark’s curated pairs (251 after filtering). Official: the benchmark’s evaluator on 122 pairs. KL: donor reading to edited reading on 150 test pairs; not computed for DAS.

TargetMass modeOfficial (%) ↑KL ↓
soft reading (Ψ)CONDITIONAL90.24 ± 0.520.103 ± 0.001
soft reading (Ψ)REFERENCE92.00 ± 1.400.102 ± 0.001
soft reading (Ψ)FULL89.03 ± 0.000.110 ± 0.003
model argmaxCONDITIONAL88.55 ± 0.490.277 ± 0.009
model argmaxREFERENCE89.71 ± 0.950.391 ± 0.007
model argmaxFULL87.64 ± 1.070.195 ± 0.010
DAS (official implementation)–91.70 ± 0.93–

Table 4.

Table 4 compares Ψ with the benchmark’s DAS implementation retrained at the same budget: the same number of optimizer updates, training examples and batch sizes, and the final checkpoint. The Ψ rows cross the three mass modes of § 2.2 with soft and argmax targets. Differences that remain: DAS keeps its own optimizer and full-vocabulary objective, and trains on the benchmark’s curated pairs.

Official score by family

Table 5 splits the official score of the matched-budget arms into the benchmark’s three counterfactual families. Its lower part takes the family in which both the answer’s position and the option symbols differ between base and source, and counts how often the edited output is the recipient’s symbol at the donor’s position and how often it is the donor’s own symbol.

Training targetPosition (%)Symbol (%)Both (%)Macro (%)
soft reading (Ψ\Psi)100.0 ± 0.091.4 ± 0.079.3 ± 1.690.24 ± 0.52
model argmax100.0 ± 0.081.0 ± 1.684.7 ± 3.188.55 ± 0.49
DAS100.0 ± 0.089.5 ± 1.685.6 ± 1.691.70 ± 0.93

Table 5. Official score by counterfactual family. Top: the arms of Table 4 (CONDITIONAL for Ψ\Psi and argmax), official evaluator, by family: only the answer’s position differs between base and source (n=50n = 50), only the option symbols differ (n=35n = 35), both differ (n=37n = 37); macro is their unweighted mean. Bottom: test pairs of the both family whose donor answer symbol differs from the recipient’s symbol at the donor’s answer position (n=44n = 44); share of greedy outputs equal to the recipient’s symbol at the donor’s position, and equal to the donor’s own symbol. Greedy output: first generated field, punctuation stripped, case-sensitive. Learned arms: seeds 11–13; controls: single runs.

ArmRecipient’s symbol at donor’s position (%)Donor’s symbol (%)
soft reading (Ψ\Psi)66.7 ± 3.59.1 ± 0.0
model argmax72.7 ± 2.36.8 ± 0.0
DAS77.3 ± 2.35.3 ± 1.3
full residual copy45.513.6
no edit0.06.8

Table 5.

Transfer across answer encodings

We evaluated frozen carriers, trained with the original option labels, after re-encoding the option labels of donor and recipient independently: lowercase letters, the digits 1–4, and the same digits permuted per prompt. No carrier was retrained, and the expected answer is the donor’s option position written in the recipient’s labels. Figure 6 gives the learned carriers and Table 6 the controls on the same pairs. In a separate test with the donor’s own labels lowercased and an uppercase recipient, no learned carrier outputs the donor’s lowercase symbol (0.0% of 150 pairs); copying the full residual vector outputs the donor’s letter on 44.0% of pairs with an original donor and a digit recipient.

Figure 6. Transfer across answer encodings. MCQA XOrderX_{\mathrm{Order}}, gemma-2 2b, layer 17, last_token, rank 16; carriers trained with the original labels and frozen; seeds 11–13, final checkpoint of 768 updates, mean over seeds. Rows: donor encoding; columns: recipient encoding (A–Z: the original uppercase labels; a–d; 1–4; perm. 1–4: digits permuted per prompt). Each cell is the percentage of the 150 test pairs on which the edited recipient answers with the donor’s option position written in the recipient’s labels. Scoring rule: greedy, first generated field with punctuation stripped, exact case-sensitive label, unweighted mean over the three counterfactual families. The strip above each map is the unedited model’s greedy accuracy on 200 prompts under each recipient encoding. One colour scale, 0–100, for all panels. (a) Ψ\Psi, soft reading; (b) argmax target; (c) the benchmark’s DAS implementation at the same budget.

ControlDonor encodingoriginallowercasedigitsperm. digits
random subspaceoriginal33.333.332.032.0
lowercase33.333.331.332.7
digits33.333.332.032.7
perm. digits33.333.332.032.7
no editoriginal33.333.332.032.7
lowercase33.333.332.032.7
digits33.333.332.032.7
perm. digits33.333.332.032.7
full residual copyoriginal75.362.77.314.7
lowercase100.0100.04.78.7
digits86.079.349.328.0
perm. digits48.744.736.734.7

Table 6. Controls for Figure 6. Same pairs and scoring rule as the figure (greedy, family macro, all 150 pairs); single runs.

Natural ARC

The experiment of Section 3.2 uses the ARC (Easy) four-choice questions as given, on gemma-2 2b at layer 17, last token, rank 16. The 2241 training questions are not filtered for correctness (the model answers 79.7% of them correctly), donors are other training questions, and every arm takes 768 updates and keeps the final checkpoint. Gold labels enter the gold arm only; for the other three they are removed from the training cache and play no part in filtering, targets or checkpoint choice.

The test set excludes the 4 test questions whose text also occurs among the training questions; the remaining 1178 distinct questions form 589 disjoint donor–recipient pairs, 111 of them with a donor the model answers wrongly. Temperatures were fitted per checkpoint on 283 pairs of 566 further validation questions, excluding one whose text overlaps the training or test questions, and applied unchanged to the test pairs. Table 7 also gives the official MIB score of each arm, which is computed on the benchmark’s correctness-filtered counterfactual pairs, a different population from the natural pairs.

TargetKL rawKL fittedTemperatureAgreement (%) allAgreement (%) wrong donorOfficial (%)
soft (Ψ\Psi)0.0200.0191.01–1.0294.8 ± 0.285.9 ± 1.070.03 ± 1.21
confidence0.0280.0280.99–1.0095.2 ± 0.484.7 ± 1.869.95 ± 0.69
argmax0.1170.0631.56–1.5892.9 ± 1.179.3 ± 3.968.27 ± 0.81
gold0.0900.0581.37–1.4292.4 ± 1.172.1 ± 5.671.45 ± 1.36
no edit1.548––25.630.6–

Table 7. Natural ARC by training target. gemma-2 2b, layer 17, last token, rank 16; 2241 unfiltered training questions, 768 updates, final checkpoint, seeds 11–13. KL: donor’s native reading to the edited reading over the four options, on 589 held-out pairs (seed mean); fitted: after one temperature per checkpoint, fitted on 283 separate validation pairs (range over seeds in the next column). Agreement: greedy answer of the edited recipient equals the donor’s own preferred answer, on all pairs and on the 111 pairs whose donor the model answers wrongly. Official: the benchmark’s evaluator on its correctness-filtered counterfactual pairs, a different population from the natural pairs.

Method\multicolumn{3}{c}{gemma-2.2b}\multicolumn{3}{c}{llama-3.1 8b}
AContA_{\mathrm{Cont}}ACtryA_{\mathrm{Ctry}}ALangA_{\mathrm{Lang}}AContA_{\mathrm{Cont}}ACtryA_{\mathrm{Ctry}}ALangA_{\mathrm{Lang}}
Full Vector48 (62)49 (57)45 (56)53 (62)47 (53)47 (57)
Random61 (62)50 (51)55 (55)62 (62)51 (51)56 (57)
DBM66 (71)53 (65)54 (58)68 (80)53 (59)58 (64)
+ PCA63 (70)47 (53)50 (56)62 (74)48 (54)53 (57)
+ SAE64 (72)49 (56)53 (59)64 (72)50 (57)55 (57)
DAS75 (85)57 (67)62 (70)75 (83)58 (64)63 (70)
Ψ\Psi (ours)68 (80)54 (60)59 (64)64 (77)54 (63)60 (67)

Table 8. RAVEL and IOI. Top, RAVEL: conventions as in Table 1, interchange accuracy with the best layer in parentheses, higher better. Bottom, IOI on gpt2-small, whose site is a fixed set of attention heads (three for SPosS_{\mathrm{Pos}}, one for STokS_{\mathrm{Tok}}) rather than a swept layer. The benchmark scores it as a mean-squared error between the causal model’s logit difference and the subject’s, so lower is better there; we add the interchange accuracy of the same runs, which the publication does not report (for Full Vector, the whole-vector interchange at the same heads, from our own audit), and our own run of DAS through the same evaluator, because the published STokS_{\mathrm{Tok}} entry is one we could not reproduce. Accuracy is not bolded where doing nothing already scores highest. A dash marks a number the source does not provide.

MethodSPosS_{\mathrm{Pos}}STokS_{\mathrm{Tok}}SPosS_{\mathrm{Pos}}STokS_{\mathrm{Tok}}
No Change16.195.3933.3100.0
Random5.873.6940.599.4
Full Vector2.452.8277.295.7
DBM2.222.35——
+ PCA2.242.33——
DAS2.202.08——
DAS (reproduced)2.272.9379.098.8
Ψ\Psi (ours)2.293.2680.199.7

Table 8 (PDF p. 22). IOI

RAVEL and IOI

RAVEL. The RAVEL task ([13]) queries a language model about the continent, country and language of a city, and asks a localization method to change one attribute while leaving the others alone. A base prompt queries one attribute at a time, so an intervention on an attribute the prompt is not asking about is supposed to change nothing. We score it with the protocol of the published baselines ([19]): a test set in which half of the pairs query the intervened attribute, two generated tokens and the benchmark’s answer checker. Ψ\Psi trails DAS in all six cells of Table 8 and leads the other published methods in all but one. On this task the model’s reading over the library, which lists the values of all three attributes, is often diffuse: for continent, its top entry is the gold value on 43.8% of prompts.

IOI. This is the one task whose transport rule is supplied by the benchmark’s high-level model rather than measured from the subject, as Appendix § 7.§ 3 sets out, and we report it on those terms. Its subject is gpt2-small. Indirect object identification ([32]) asks a model to complete a sentence such as "When Mary and John went to the store, John gave a drink to" with the name that appeared once rather than twice. MIB derives two variables from it: SPosS_{\mathrm{Pos}}, the position of the repeated subject, and STokS_{\mathrm{Tok}}, its token identity. This task is scored differently from every other one in the benchmark. Rather than asking whether the intervened model produces the expected answer, it measures:

El[(Δϕ(l)−ΔH(l))2],Δ=logit⁡(IO)−logit⁡(S),\mathbb{E}_{l}\left[\left(\Delta_{\phi}(l)-\Delta_{\mathrm{H}}(l)\right)^{2}\right], \qquad\Delta=\operatorname{logit}(\mathrm{IO})-\operatorname{logit}(\mathrm{S}),

i.e., the mean-squared error between the logit difference the intervened subject produces and the one the high-level model H\mathrm{H} prescribes; lower is better. The abstraction site is also fixed: the alignment is carried by designated attention heads previously identified by Wang et al. [32] (three for SPosS_{\mathrm{Pos}} and one for STokS_{\mathrm{Tok}}) rather than by a swept layer.

Table 8 (bottom) reports our results. Under the objective used everywhere else in this paper, Ψ\Psi lags behind the official DAS results and the other trained baselines: it reaches 2.29 on SPosS_{\mathrm{Pos}} and 3.26 on STokS_{\mathrm{Tok}}. We also ran our own reproduction of the DAS method, which reaches 2.27 on SPosS_{\mathrm{Pos}}, close to the official 2.20, but 2.93 on STokS_{\mathrm{Tok}}, substantially higher than the official 2.08.

Two readings of those numbers are worth separating. The first is that the two variables are not equally decisive, and both metrics say so. The MSE has no natural zero, so it is read between its two nulls. On SPosS_{\mathrm{Pos}} doing nothing costs 16.19, a random subspace of the matched rank 5.87, and every trained method between 2.20 and 2.29. On STokS_{\mathrm{Tok}} the same three anchors are 5.39, 3.69 and 2.08 to 3.26: the whole scale there is narrower than the stretch a random draw already covers on SPosS_{\mathrm{Pos}}. Scored as an interchange accuracy on the same audits, doing nothing reaches 33.3% and 100.0% respectively: moving the subject’s position changes which name the model should emit, and moving its token identity does not. On the variable where the answer does move, Ψ\Psi reaches 80.1% against 79.0% for our DAS reproduction, compared with 77.2% for a whole-vector interchange at the same heads and 40.5% for a random subspace. On the other, every method except the whole-vector interchange, doing nothing included, sits within a point of the ceiling, and only the logit difference can be measured at all, which is precisely the reason the benchmark authors opted for the MSE metric on this task.

The second concerns what the objective aims at. A reading over two single-token names expresses the benchmark’s margin exactly, since log⁡(qIO/qS)=logit⁡(IO)−logit⁡(S)\log(q_{\mathrm{IO}}/q_{\mathrm{S}})=\operatorname{logit}(\mathrm{IO})-\operatorname{logit}(\mathrm{S}), and the transport table we build for IOI is precisely such a margin passed through a sigmoid. What differs is the loss and where its numbers come from: we minimize a cross-entropy against that target while the benchmark reports a squared error on the margin itself, so the two weight deviations differently, and the target is supplied by the benchmark’s linear model rather than measured from the subject (Appendix § 7.§ 3). We therefore read the distance between 3.26 and the published 2.08 as the cost of optimizing a different quantity under a supervision the rest of this paper does not use. The distance between the same published entry and our own reproduction of the method that produced it, 2.93, should be read alongside it.

Depthwise localization accuracy

In this section we report for completeness the full depthwise intervention accuracy profiles for the different MIB tasks. The official benchmark scores reported in Table 1 in fact only report two numbers per cell, i.e. mean and maximum. Figure 7 gives the profile they are computed from, which illustrates patterns that the aggregate score hides. On MCQA and ARC the token positions specialize with depth: an early layer is read best at the answer symbol and a late one at the last token, and the envelope is high across most of the network because one position or another carries the variable at each depth. On RAVEL the envelope tracks the random subspace along its whole length.

Figure 7. Depthwise localization accuracy. Interchange accuracy of Ψ\Psi at every layer of every cell, one thin line per token position and the thick line their maximum at each layer. The thick line is what Table 1 summarizes: its average over layers is the depth mean and its highest point is the best layer. The gray line is the random subspace of the same rank, read the same way, wherever we have measured it.

Environment and compute

All experiments are implemented in PyTorch (2.12) and evaluated by the benchmark’s own evaluator at a pinned commit, in its own pinned environment, so our scores and the published baselines are produced by the same code path.

The localization results sweep every layer of each subject against the token positions the benchmark defines for that task, three for MCQA and two for ARC, arithmetic and RAVEL, which is 52 to 96 training runs per cell and 1188 in total. Training is one run per site, for 32 epochs on MCQA, two on ARC and one on arithmetic and RAVEL, matching the budgets under which the baselines were tuned. These ran on H100 cards; the random floors, which need no training, and the control and supervision experiments ran on three RTX A4000 cards. Cumulative compute for the project, including preliminary experiments that do not appear here, amounted to approximately 200 GPU-hours.

Training hyperparameters

Every run in this paper trains the carrier with AdamW (weight decay 10−410^{-4}, gradient norm clipped at 1) and keeps AA orthonormal by taking the QR factor of an unconstrained matrix. The frozen subject runs in half precision, except gpt2-small, which runs in full precision; llama-3.1 8b is the instruction-tuned checkpoint (Meta-Llama-3.1-8B-Instruct) the benchmark specifies. Single-seed results use seed 11 and three-seed results use seeds 11 to 13. Each batch draws its sources by a fresh random derangement among prompts of the same transport group, and the carrier rank is the official DAS rank of each cell (§ 3). Table 9 lists what differs between experiments; every value is read from the frozen configuration of the corresponding runs, which the supplementary material ships.

ExperimentSubjectsEpochsBatchLearning rateMass mode
Localization, every layer and benchmark position (Tables 1 and 8)
MCQAgemma-2_2b, llama-3.1_8b, qwen-2.5_0.5b3216, 8, 323×10−43 \times10^{-4}CONDITIONAL
ARC (Easy)gemma-2_2b, llama-3.1_8b2163×10−43 \times10^{-4}CONDITIONAL
Arithmeticgemma-2_2b, llama-3.1_8b1810−310^{-3}FULL
RAVELgemma-2_2b, llama-3.1_8b132, 1610−310^{-3}CONDITIONAL
IOI, fixed headsgpt2-small23210−310^{-3}CONDITIONAL
Supervision ablation (Figure 2)
MCQA XOrderX_{\mathrm{Order}}, layer 17gemma-2_2b32163×10−43 \times10^{-4}CONDITIONAL
Control (Figure 4)
MCQA XOrderX_{\mathrm{Order}}, layer 17gemma-2_2b32163×10−43 \times10^{-4}all three
Arithmetic XCarryX_{\mathrm{Carry}}, layer 21gemma-2_2b1810−310^{-3}CONDITIONAL, FULL

Table 9. Training settings. Localization runs use λ=0\lambda= 0 in Eq. (10). Control runs use λ=1\lambda= 1 and the gain controller of Eq. (9), trained with learning rate 0.1 against requests drawn half from other prompts’ readings and half uniformly from the simplex, with an edit budget cc (§ 2.3) that every arm at a site shares. It is the 95th percentile of the distance between the activations of randomly paired training prompts, measured over the whole activation at the MCQA site (c=104.4c = 104.4) and inside the frozen Ψ\Psi carrier at the Arithmetic site (c=208.6c = 208.6, against 373.0 over the whole activation). Where a row lists several batch sizes, they follow the order of its subjects.

References

  1. [1]Aryaman Arora, Dan Jurafsky, and Christopher Potts. CausalGym: Benchmarking causal interpretability methods on linguistic tasks. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 14638–14663, Bangkok, Thailand, August 2024. Association for Computational Linguistics. doi: 10.18653/v1/2024.acl-long.785. URL https://aclanthology.org/2024.acl-long.785/.DOI
  2. [2]Yuntai Bao, Xuhong Zhang, Jintao Chen, Ge Su, Yuxiang Cai, Hao Peng, Sun Bing, Haiqin Weng, Liu Yan, and Jianwei Yin. Faithful bi-directional model steering via distribution matching and distributed interchange interventions. In International Conference on Learning Representations, 2026.arxiv.org/abs/2602.05234
  3. [3]Yonatan Belinkov. Probing classifiers: Promises, shortcomings, and advances. Computational Linguistics, 48(1):207–219, 2022. doi: 10.1162/coli_a_00422. URL https://aclanthology.org/2022.coli-1.7/.DOI
  4. [4]Usha Bhalla, Suraj Srinivas, Asma Ghandeharioun, and Himabindu Lakkaraju. Towards unifying interpretability and control: Evaluation via intervention. arXiv preprint arXiv:2411.04430, 2025.arxiv.org/abs/2411.04430
  5. [5]Hoagy Cunningham, Aidan Ewart, Logan Riggs, Robert Huben, and Lee Sharkey. Sparse autoencoders find highly interpretable features in language models. In International Conference on Learning Representations, 2024.arxiv.org/abs/2309.08600
  6. [6]Nelson Elhage, Tristan Hume, Catherine Olsson, Nicholas Schiefer, Tom Henighan, Shauna Kravec, Zac Hatfield-Dodds, Robert Lasenby, Dawn Drain, Carol Chen, et al. Toy models of superposition. arXiv preprint arXiv:2209.10652, 2022.arxiv.org/abs/2209.10652
  7. [7]Cosimo Galeone, Anna Ettorre, Minsu Park, Giuseppe Ettorre, and Daniele Ligorio. Perfect detection, failed control: The geometry of knowing vs. steering in language models. arXiv preprint arXiv:2606.24952, 2026.arxiv.org/abs/2606.24952
  8. [8]Leo Gao, Tom Dupré la Tour, Henk Tillman, Gabriel Goh, Rajan Troll, Alec Radford, Ilya Sutskever, Jan Leike, and Jeffrey Wu. Scaling and evaluating sparse autoencoders. In International Conference on Learning Representations, 2025.arxiv.org/abs/2406.04093
  9. [9]Atticus Geiger, Hanson Lu, Thomas Icard, and Christopher Potts. Causal abstractions of neural networks. In Advances in Neural Information Processing Systems, 2021.arxiv.org/abs/2106.02997
  10. [10]Atticus Geiger, Zhengxuan Wu, Hanson Lu, Josh Rozner, Elisa Kreiss, Thomas Icard, Noah D. Goodman, and Christopher Potts. Inducing causal structure for interpretable neural networks. In International Conference on Machine Learning, 2022.arxiv.org/abs/2112.00826
  11. [11]Atticus Geiger, Zhengxuan Wu, Christopher Potts, Thomas Icard, and Noah D. Goodman. Finding alignments between interpretable causal variables and distributed neural representations. In Causal Learning and Reasoning (CLeaR), 2024.arxiv.org/abs/2303.02536
  12. [12]Atticus Geiger, Duilgúr Ib​​eling, Amir Zur, Maheep Chaudhary, Sonakshi Chauhan, Jing Huang, Aryaman Arora, Zhengxuan Wu, Noah Goodman, Christopher Potts, and Thomas Icard. Causal abstraction: A theoretical foundation for mechanistic interpretability. Journal of Machine Learning Research, 26(83):1–64, 2025. URL https://jmlr.org/papers/volume26/23-0058/23-0058.html.
  13. [13]Jing Huang, Zhengxuan Wu, Christopher Potts, Mor Geva, and Atticus Geiger. RAVEL: Evaluating interpretability methods on disentangling language model representations. In Annual Meeting of the Association for Computational Linguistics, 2024.arxiv.org/abs/2402.17700
  14. [14]Chonghe Jiang, Bao Nguyen, Anthony Man-Chao So, and Viet Anh Nguyen. Probe-free low-rank activation intervention. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pp. 2812–2824, 2025.arxiv.org/abs/2502.04043
  15. [15]Kenneth Li, Oam Patel, Fernanda Viégas, Hanspeter Pfister, and Martin Wattenberg. Inference-time intervention: Eliciting truthful answers from a language model. In Advances in Neural Information Processing Systems, 2023.arxiv.org/abs/2306.03341
  16. [16]Aleksandar Makelov, Georg Lange, Atticus Geiger, and Neel Nanda. Is this the subspace you are looking for? An interpretability illusion for subspace activation patching. In International Conference on Learning Representations, 2024.arxiv.org/abs/2311.17030
  17. [17]Julian Minder, Kevin Du, Niklas Stoehr, Giovanni Monea, Chris Wendler, Robert West, and Ryan Cotterell. Controllable context sensitivity and the knob behind it. In International Conference on Learning Representations, 2025. URL https://openreview.net/forum?id=Igm9bbkzHC.arxiv.org/abs/2411.07404
  18. [18]Aaron Mueller, Jannik Brinkmann, Millicent Li, Samuel Marks, Koyena Pal, Nikhil Prakash, Can Rager, Aruna Sankaranarayanan, Arnab Sen Sharma, Jiuding Sun, Eric Todd, David Bau, and Yonatan Belinkov. The quest for the right mediator: Surveying mechanistic interpretability through the lens of causal mediation analysis. arXiv preprint arXiv:2408.01416, 2024.arxiv.org/abs/2408.01416
  19. [19]Aaron Mueller, Atticus Geiger, Sarah Wiegrefe, Dana Arad, Iván Arcuschin, Adam Belfki, Yik Siu Chan, Jaden Fiotto-Kaufman, Tal Haklay, Michael Hanna, Jing Huang, Rohan Gupta, Yaniv Nikankin, Hadas Orgad, Nikhil Prakash, Anja Reusch, Aruna Sankaranarayanan, Shun Shao, Alessandro Stolfo, Martin Tutek, Amir Zur, David Bau, and Yonatan Belinkov. MIB: A mechanistic interpretability benchmark. In International Conference on Machine Learning, 2025.arxiv.org/abs/2504.13151
  20. [20]Chris Olah, Nick Cammarata, Ludwig Schubert, Gabriel Goh, Michael Petrov, and Shan Carter. Zoom in: An introduction to circuits. Distill, 2020. doi: 10.23915/distill.00024.001. URL https://distill.pub/2020/circuits/zoom-in/.DOI
  21. [21]Kiho Park, Yoong Choe, and Victor Veitch. The linear representation hypothesis and the geometry of large language models. In Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pp. 39643–39666. PMLR, 2024. URL https://proceedings.mlr.press/v235/park24c.html.arxiv.org/abs/2311.03658
  22. [23]Nina Rimsky, Nick Gabrieli, Julian Schulz, Meg Tong, Evan Hubinger, and Alexander Turner. Steering llama 2 via contrastive activation addition. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 15504–15522, Bangkok, Thailand, August 2024. Association for Computational Linguistics. doi: 10.18653/v1/2024.acl-long.828. URL https://aclanthology.org/2024.acl-long.828/.DOI
  23. [24]Pau Rodriguez, Michal Klein, Eleonora Gualdoni, Valentino Maiorca, Arno Blaas, Luca Zappella, Marco Cuturi, and Xavier Suau. LinEAS: End-to-end learning of activation steering with a distributional loss. In Advances in Neural Information Processing Systems, 2025. URL https://arxiv.org/abs/2503.10679.
  24. [25]Lee Sharkey, Bilal Chughtai, Joshua Batson, Jack Lindsey, Jeff Wu, Lucius Bushnaq, Nicholas Goldowsky-Dill, Stefan Heimersheim, Alejandro Ortega, Joseph Bloom, et al. Open problems in mechanistic interpretability. arXiv preprint arXiv:2501.16496, 2025.arxiv.org/abs/2501.16496
  25. [26]Yingdong Shi, Ruiming Zhang, Changming Li, Zhiyu Yang, Kaixing Zhang, Jingyi Yu, and Kan Ren. UniSteer: Text-guided flow matching in activation space for versatile LLM steering. arXiv preprint arXiv:2605.30076, 2026.arxiv.org/abs/2605.30076
  26. [28]Jiuding Sun, Jing Huang, Sidharth Baskaran, Karel D’Oosterlinck, Christopher Potts, Michael Sklar, and Atticus Geiger. HyperDAS: Towards automating mechanistic interpretability with hypernetworks. In International Conference on Learning Representations, 2025.arxiv.org/abs/2503.10894
  27. [29]Denis Sutter, Julian Minder, Thomas Hofmann, and Tiago Pimentel. The non-linear representation dilemma: Is causal abstraction enough for mechanistic interpretability? In Advances in Neural Information Processing Systems, 2025.
  28. [30]Daniel Tan, David Chanin, Aengus Lynch, Brooks Paige, Dimitrios Kanoulas, Adria Garriga-Alonso, and Robert Kirk. Analysing the generalisation and reliability of steering vectors. In Advances in Neural Information Processing Systems, 2024. URL https://proceedings.neurips.cc/paper_files/paper/2024/hash/fb3ad59a84799bfb8d700e56d19c231b-Abstract-Conference.html.
  29. [31]Alexander Matt Turner, Lisa Thiergart, Gavin Leech, David Udell, Juan J Vazquez, Ulisse Mini, and Monte MacDiarmid. Steering language models with activation engineering. arXiv preprint arXiv:2308.10248, 2023.arxiv.org/abs/2308.10248
  30. [32]Kevin Wang, Alexandre Variengien, Arthur Conmy, Buck Shlegeris, and Jacob Steinhardt. Interpretability in the wild: a circuit for indirect object identification in GPT-2 small. In International Conference on Learning Representations, 2023.arxiv.org/abs/2211.00593
  31. [33]Zhengxuan Wu, Atticus Geiger, Thomas Icard, Christopher Potts, and Noah D. Goodman. Interpretability at scale: Identifying causal mechanisms in Alpaca. In Advances in Neural Information Processing Systems, 2023.arxiv.org/abs/2305.08809
  32. [34]Zhengxuan Wu, Aryaman Arora, Zheng Wang, Atticus Geiger, Dan Jurafsky, Christopher D. Manning, and Christopher Potts. REFT: Representation finetuning for language models. In Advances in Neural Information Processing Systems, 2024.arxiv.org/abs/2404.03592
  33. [35]Zhengxuan Wu, Aryaman Arora, Atticus Geiger, Zheng Wang, Jing Huang, Dan Jurafsky, Christopher D Manning, and Christopher Potts. AxBench: Steering LLMs? Even simple baselines outperform sparse autoencoders. In Proceedings of the 42nd International Conference on Machine Learning, volume 267 of Proceedings of Machine Learning Research, pp. 67035–67080. PMLR, 2025. URL https://proceedings.mlr.press/v267/wu25a.html.arxiv.org/abs/2501.17148

Paper details

vlnc:W_bcczapjn
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY-SA 4.0
Usage

views · 2

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

generative AI tools

View statement
Looking for related papers…
Contents