Introduction
A trained language model carries out many computations, such as tracking which brackets are still open, and two research directions describe how. Mechanistic interpretability (MI) draws a wiring diagram: it identifies the components that carry a computation (attention heads and MLPs) and the task quantities they compute and pass on, such as the current nesting depth, which we call variables , and it confirms each assignment by intervening on the network, such as removing a component’s output [28 , 40 , 8 ] . Singular learning theory (SLT) surveys the terrain around the trained parameters: many nearby parameter settings compute the same function, and the local learning coefficient (LLC) λ \lambda λ measures the effective dimension of the trained solution: the volume of parameter settings whose loss lies within ϵ \epsilon ϵ of the minimum shrinks like ϵ λ \epsilon^\lambda ϵ λ , so a model whose d d d parameters are all pinned down by the data has λ = d / 2 \lambda= d/2 λ = d /2 , and every direction the data leave free lowers it [41 , 24 ] . If MI’s diagram is the wiring of a machine, the LLC measures how far its parts can move without changing what it does; it marks stages of learning in small transformers [20 , 38 ] . The two descriptions concern the same computation, so a component that carries more of it should both matter more under intervention and be more tightly constrained: more of the output depends on its parameters, so removing its output changes the prediction more and displacing its parameters raises the loss faster. Establishing this correspondence matters because each description supplies what the other lacks: MI’s accounts are assembled by hand, one task and checkpoint at a time, and depend on the choice of intervention, whereas SLT’s measurements rest on a theory of generalization and can be repeated automatically over every component and checkpoint, but cannot by themselves say which computation a change in geometry reflects. A correspondence would let geometry measured throughout training signal when and where a mechanism forms, which is what monitoring the development of capabilities in large models requires, but three obstacles stand in the way. A pretrained language model offers no independently identified mechanism to check against. The standard estimator draws parameter settings with stochastic-gradient Langevin dynamics (SGLD), and nothing at this scale says how far the draws may move: under standard localization they move each weight by several times its magnitude and change the model’s predictions, while a localization strong enough to hold them where predictions are preserved leaves only the prior (Appendix § 18 ). And near a trained network any such estimate is a function of the Fisher information, which says how steeply the loss rises in each direction, plus corrections, so whether SLT adds anything beyond curvature can only be decided by computing both.
We first make explicit what both descriptions depend on (§ 4 ): the map from a component’s parameters to the model’s next-token predictions on fixed inputs. Writing θ \theta θ for the trained parameters and θ + δ θ \theta+ \delta\theta θ + δ θ for a modified model, both descriptions measure change by the Kullback–Leibler (KL) divergence KL ( p ( y ∣ x ; θ ) ∥ p ( y ∣ x ; θ + δ θ ) ) \operatorname{KL}(p(y \mid x;\theta) \mathbin\Vert p(y \mid x;\theta+ \delta\theta)) KL ( p ( y ∣ x ; θ ) ∥ p ( y ∣ x ; θ + δ θ )) from the trained model’s next-token distribution to the modified one, averaged over inputs x x x . The first derivatives of the predictions with respect to the parameters determine the Fisher matrix, which describes how this divergence rises quadratically, as a parabola, around the trained parameters, and with it the effects of small interventions and, when their rank is constant, the learning coefficient. Beyond the parabola, averaging over perturbations of either sign cancels the odd-order terms, so the first correction is quartic in the size of the perturbation, and we summarize it by a quartic coefficient; and when two components are perturbed together, the divergence contains mixed terms , those that depend on both perturbations at once, which express how changing one component alters the network’s response to the other (§ 3 states these expansions precisely). We then test the correspondence in Pythia-1B at all 154 released training checkpoints (Biderman et al., [5 ] ) on bounded Dyck languages of nested brackets, where the model must predict the next closing bracket. The task is simple by design: testing whether geometry and mechanism agree (in the sense of § 3 ) requires a mechanism known independently of the geometry, and Dyck languages, the canonical model of the nested structure of syntax and code (Chomsky & Schützenberger, [7 ] ; Hewitt et al., [19 ] ), come with a known algorithm, exact values for every intermediate variable, and well-studied transformer solutions (Yao et al., [45 ] ; Strobl et al., [31 ] ), yet Pythia-1B acquires the skill only partially and gradually during pretraining (§ 3 ). It falls short because it is a base model, trained to continue text rather than to complete brackets: at most positions where a closing bracket is scored, an opening bracket is also a legal continuation, 77% of its errors are such openers, and among the four closers it ranks the correct one first at 86% of positions (Appendix § 16 ). SLT estimates λ \lambda λ by treating the parameters near a checkpoint as random: it draws whole parameter settings from a distribution concentrated around the trained values (a localized, tempered posterior), like dropping marbles near the bottom of a valley and recording how high each comes to rest, and averages the loss over the draws; the average grows with the number of directions in which the valley is steep. Our draws change one component at a time and stay inside an explicit region where the predictions are preserved, and evaluating the Fisher approximation on the same draws splits each estimate into its Fisher and quartic parts.
Our contributions (Figure 1 ) are the following. Problem. The first test of whether SLT’s geometry and MI’s causal account agree in a pretrained language model, against a five-stage bracket circuit in Pythia-1B validated by held-out interventions (§ 5.1 ). Solution. A formal correspondence between the terms of the loss expansion (3) and intervention effects (§ 4 ), and a measurement whose draws stay where predictions are preserved, with the Fisher spectrum of all 144 components at all 154 checkpoints and a replication on independent inputs (§ 5.2 ). Results. Fisher-trace changes track causal-effect changes (median partial rank correlation 0.51), largely as a trend shared across the model, while the gain Fisher and the large-temperature count carry what is specific to a component; the quartic part is what a nonlocal sampler registers; and the mixed terms track joint-ablation interactions (Sections 5.3–5.4).
Figure 1. Workflow. The circuit is identified with interventions alone, the local geometry of every component is measured inside the region where predictions are preserved, and the two are compared term by term across the 154 checkpoints.
Related Works
Singular learning theory in trained networks. Singular learning theory describes a trained network by the shape of its loss around the trained parameters. Its central quantity is the local learning coefficient (LLC) λ \lambda λ , the effective number of parameter directions that the data pin down: the volume of parameter settings whose loss lies within ϵ \epsilon ϵ of the trained model’s shrinks like ϵ λ \epsilon^{\lambda} ϵ λ (§ 3 ).
Lau et al. [24 ] made it estimable by drawing parameter settings with SGLD from a localized, tempered posterior [43 , 30 ] . Its trajectory marks learning stages in small transformers [20 , 6 ] , its weight-refined version separates the roles of attention heads in models with known mechanisms [38 ] , and it, with related susceptibility methods (Baker et al., 2026; Gordon et al., 2026; Wang & Murfet, 2026), has been applied to Pythia models up to 6.9B parameters [34 ] and to a 9B reward model (Wang et al., 2026). At this scale, however, no independently identified mechanism has been available, the estimator has had no reference for how far its draws may move the parameters, and the estimate has not been separated from the Fisher information, itself a standard measure of parameter importance [22 , 26 , 25 ] . We supply all three: a circuit validated by interventions, draws confined to where predictions are preserved, and the exact Fisher approximation evaluated on the same draws as the primary control (Equation (5) ).
Circuits and causal interpretation. Mechanistic interpretability explains a behavior by a circuit: components, the variables they compute, and the interventions that confirm each assignment [40 , 8 , 15 ] . What an intervention measures depends on the replacement value, the grouping of components, and the output metric [18 ] , and a role shared by several components can be missed by removing any one, as with backup heads [40 ] . We establish the circuit with held-out interventions on variable subspaces rather than probes alone, compare geometry with joint as well as single ablations, and obtain from the geometry a measure of a role that needs no replacement value.
Problem Formulation and Preliminaries
We ask: for a computation performed by a pretrained language model, do the components that MI finds to carry it, and the points in training at which their causal roles form, coincide with where and when SLT finds the parameters constrained? (Q1) At a checkpoint, does a component’s geometric coefficient rank components as intervention effects do, and (Q2) across checkpoints, do the coefficient and the effect change at the same times? We say that geometry and mechanism agree to the extent that both answers are yes, as measured by the rank correlations of Section 5.3. We now define the terms these questions need.
The Dyck bracket matching task. The bounded Dyck language D y c k ( k , D ) \mathrm{Dyck}(k,D) Dyck ( k , D ) contains the correctly nested strings of k k k bracket types whose depth never exceeds D D D , such as [ ] ( ) [ \ ] \ ( \ ) [ ] ( ) for k = 2 k=2 k = 2 ; wherever the next token is a closing bracket, the correct one is the partner of the most recent unmatched opener. It suits our question for four reasons. It is the canonical hierarchical language: every context-free language is a homomorphic image of a Dyck language intersected with a regular language [7 ] , and the depth bound mirrors the limited nesting of natural-language syntax (Hewitt et al. [19 ] ) and puts the language within reach of fixed-size self-attention networks (Yao et al. [45 ] ), which unbounded Dyck is not (Hahn [17 ] ). It is solved by a known algorithm, a stack, whose variables (the depth, the type on top of the stack, the distance to the most recent unmatched opener) have exact values at every position, giving probes and interventions exact targets. It is a standard test of what recurrent networks and transformers learn and express ([32 , 4 , 31 ] ), with well-characterized solutions that attend to the top of the stack ([45 , 10 ] ) and can be written and learned as interpretable programs ([42 , 13 ] ); yet trained networks can also solve it without stack-like attention (Wen et al. [44 ] ) and carry decodable variables they do not use (Sharma et al. [29 ] ), so its circuit must be established by interventions. Finally, Pythia-1B, trained on the Pile (Gao et al. [14 ] ), which includes source code, reaches a final closing-bracket accuracy of 0.595 for k = 4 k=4 k = 4 and improves across the 154 checkpoints.
Components and the residual stream. A decoder-only transformer keeps one vector per token position, which every layer updates: the heads and the MLP of a layer read the vector, compute an output, and add it back, so the vector entering a layer is the sum of the token embedding and all earlier outputs: the residual stream (Elhage et al. [12 ] ), the shared workspace through which components communicate. A component is one head or one MLP, identified with its parameters (Table 1 ). Pythia-1B has 16 layers of eight heads and one MLP, 144 components; L ℓ H h L_{\ell Hh} L ℓ H h is head h h h of layer ℓ \ell ℓ and L ℓ M L P L_{\ell\mathrm{MLP}} L ℓ MLP its MLP, counting from zero.
Table 1. Protocol of the local-geometry measurements (22,176 component–checkpoint pairs; Appendices D and E).
Variables, circuits, probes, and interventions. A variable s s s is a quantity defined by the task’s algorithm, such as the depth or the type of the most recent unmatched opener, and the algorithm’s operations compute variables from others (“push the opener’s type onto the stack”). The algorithm assigns each variable a value s t s_t s t after the first t t t tokens; whether the network represents or uses it is tested by the probes and interventions below. A circuit hypothesis names a set of components, the variables each reads from and writes to the residual stream, and the operations by which it does so ([40 , 8 , 15 ] ). A probe tests whether a variable is represented: a linear map ω \omega ω fitted to predict s t s_t s t from the residual stream h ℓ , t h_{\ell,t} h ℓ , t at layer ℓ \ell ℓ is scored by its held-out R 2 R^2 R 2 ([1 , 3 ] ). An intervention tests whether the variable is used: it replaces a component’s output by a modified value (zero, a mean over inputs, or a projection onto a subspace) and measures the change in the prediction; replacement by zero, zero-ablation , measures how much the prediction depends on the component, and a large effect of removing a variable’s subspace, against small effects for random subspaces of equal dimension, shows that the component acts through that variable (Figure 2 A).
Figure 2. Two descriptions of the same computation. MI intervenes on the variables that a circuit computes (A); SLT studies the parameter settings that preserve its behavior (B). For K ( u , v ) = ( u + v ) 2 K(u,v)=(u+v)^2 K ( u , v ) = ( u + v ) 2 , cancelling changes leave the output unchanged, and { K < ϵ } \{K<\epsilon\} { K < ϵ } is a strip of area ∝ ϵ 1 / 2 \propto\epsilon^{1/2} ∝ ϵ 1/2 : λ = 1 2 \lambda=\frac{1}{2} λ = 2 1 with two parameters.
The loss geometry of a component. Write x x x for a token prefix and y y y for the next token, so that the model defines p θ ( y ∣ x ) p_\theta(y\mid x) p θ ( y ∣ x ) . Fix a checkpoint θ ⋆ ∈ R d \theta_\star\in\mathbb{R}^d θ ⋆ ∈ R d and a component c c c , whose parameters occupy d c d_c d c of the d d d coordinates. A displacement u ∈ R d c u\in\mathbb{R}^{d_c} u ∈ R d c of those coordinates alone, a d c d_c d c -dimensional vector, gives the model θ ( u ) = θ ⋆ + E c u \theta(u)=\theta_\star+E_cu θ ( u ) = θ ⋆ + E c u , where the d × d c d\times d_c d × d c matrix E c E_c E c places each coordinate of u u u at the component’s position and scales it by the root-mean-square (RMS) of its tensor, so that one unit of u u u moves a weight by the typical “average” magnitude of its tensor. On fixed task inputs, define
K c ( u ) = E x K L ( p θ ⋆ ( ⋅ ∣ x ) ∥ p θ ( u ) ( ⋅ ∣ x ) ) , (1) K_c(u)=\mathbb{E}_x\,\mathrm{KL}\bigl(p_{\theta_\star}(\cdot\mid x)\,\|\,p_{\theta(u)}(\cdot\mid x)\bigr), \tag*{(1)} K c ( u ) = E x KL ( p θ ⋆ ( ⋅ ∣ x ) ∥ p θ ( u ) ( ⋅ ∣ x ) ) , (1)
averaged over the positions at which the task is scored. We use the KL to the checkpoint’s predictions rather than the cross-entropy of the observed tokens because it makes the checkpoint an exact minimum, as SLT’s local analysis assumes, and intervention effects use the same divergence. Its Hessian at u = 0 u=0 u = 0 , the matrix of second derivatives that says how steeply the loss rises in each direction, is the component’s Fisher information matrix F c F_c F c . SLT characterizes the geometry of K c K_c K c by how fast the prior volume of the displacements that keep the loss below ϵ \epsilon ϵ shrinks as ϵ → 0 \epsilon\to0 ϵ → 0 [41 ] : if the loss were quadratic with a Hessian of rank ϱ \varrho ϱ , that volume would shrink like ϵ ϱ / 2 \epsilon^{\varrho/2} ϵ ϱ /2 , and in general it shrinks like a power of ϵ \epsilon ϵ whose exponent is the LLC λ c \lambda_c λ c . For a neighborhood U U U of zero and a smooth positive prior density φ \varphi φ ,
∫ U ∩ { K c ( u ) < ϵ } φ ( u ) d u ≍ ϵ λ c ( log ( 1 / ϵ ) ) m c − 1 ( ϵ → 0 ) , (2) \int_{U\cap\{K_c(u)<\epsilon\}} \varphi(u)\,\mathrm{d}u \asymp\epsilon^{\lambda_c}\left(\log(1/\epsilon)\right)^{m_c-1} \qquad(\epsilon\to0), \tag*{(2)} ∫ U ∩ { K c ( u ) < ϵ } φ ( u ) d u ≍ ϵ λ c ( log ( 1/ ϵ ) ) m c − 1 ( ϵ → 0 ) , (2)
where f ≍ g f \asymp g f ≍ g means that f / g f/g f / g stays between two positive constants, λ c \lambda_c λ c is the LLC of the component (the weight-refined LLC of Wang et al., [38 ] ) and m c m_c m c its multiplicity. A direction along which the loss rises only as u 4 u^4 u 4 contributes 1 / 4 1/4 1/4 to λ c \lambda_c λ c rather than 1 / 2 1/2 1/2 , and an unused direction 0 0 0 (Figure 2 B), so λ c \lambda_c λ c counts the constrained directions weighted by how the loss rises along them.
Beyond the Fisher matrix. For a displacement u = r z u = rz u = r z of scale r r r and Gaussian direction z ∼ N ( 0 , I ) z \sim\mathcal{N}(0,I) z ∼ N ( 0 , I ) , the loss and its average over directions expand as
K c ( r z ) = 1 2 r 2 z T F c z + r 3 K 3 ( z ) + r 4 K 4 ( z ) + O ( r 5 ) , E z K c ( r z ) = r 2 A c + r 4 B c + O ( r 6 ) , (3) K_c(rz) = \tfrac{1}{2}r^2z^\mathsf{T}F_cz+r^3K_3(z)+r^4K_4(z)+O(r^5), \qquad\mathbb{E}_zK_c(rz) = r^2A_c+r^4B_c+O(r^6), \tag*{(3)} K c ( r z ) = 2 1 r 2 z T F c z + r 3 K 3 ( z ) + r 4 K 4 ( z ) + O ( r 5 ) , E z K c ( r z ) = r 2 A c + r 4 B c + O ( r 6 ) , (3)
where the cubic and quartic Taylor terms K 3 K_3 K 3 and K 4 K_4 K 4 also involve second derivatives of the predictions. The first term is a parabola in r r r , whose average is set by the Fisher trace A c = 1 2 tr F c A_c = \tfrac{1}{2}\operatorname{tr}F_c A c = 2 1 tr F c ; since z z z and − z -z − z are equally likely, the odd terms average out, and the first departure from the parabola is the quartic term, with the quartic coefficient B c = E z K 4 ( z ) B_c = \mathbb{E}_zK_4(z) B c = E z K 4 ( z ) . For two components a a a and b b b displaced together, u = ( u a , u b ) u=(u_a,u_b) u = ( u a , u b ) , the loss is not the sum of the two separate losses: the difference K a b ( u a , u b ) − K a b ( u a , 0 ) − K a b ( 0 , u b ) = u a T F a b u b + ⋯ K_{ab}(u_a,u_b)-K_{ab}(u_a,0)-K_{ab}(0,u_b)=u_a^\mathsf{T}F_{ab}u_b+\cdots K ab ( u a , u b ) − K ab ( u a , 0 ) − K ab ( 0 , u b ) = u a T F ab u b + ⋯ collects the mixed terms , those containing both displacements. The leading one is set by the off-diagonal block F a b F_{ab} F ab of the joint Fisher matrix; the higher ones depend also on the joint second derivative of the predictions, which is nonzero when changing one component alters the network’s response to the other.
Estimating the coefficient. Lau et al. [24 ] estimate the LLC by drawing parameter settings from a distribution that favors low loss and stays near θ ⋆ \theta_\star θ ⋆ , a posterior tempered to inverse temperature n β n\beta n β and localized by a Gaussian term of strength γ \gamma γ , and averaging the loss increase over the draws, scaled by n β n\beta n β . For one component and the loss (1), with the neighborhood made explicit, the estimate is
π c ( u ) ∝ 1 U ( u ) e − τ K c ( u ) φ r ( u ) , q c ( τ , r ) = τ E π c K c ( u ) , (4) \pi_c(u) \propto1_U(u)e^{-\tau K_c(u)}\varphi_r(u), \qquad q_c(\tau,r)=\tau\mathbb{E}_{\pi_c}K_c(u), \tag*{(4)} π c ( u ) ∝ 1 U ( u ) e − τ K c ( u ) φ r ( u ) , q c ( τ , r ) = τ E π c K c ( u ) , (4)
with inverse temperature τ \tau τ in place of n β n\beta n β and a Gaussian prior φ r = N ( 0 , r 2 I ) \varphi_r=\mathcal{N}(0,r^2I) φ r = N ( 0 , r 2 I ) whose standard deviation r r r , the radius, sets the localization (γ = 1 / r 2 \gamma=1/r^2 γ = 1/ r 2 ). If the volume law (2) holds where the draws lie, q c → λ c q_c \to\lambda_c q c → λ c as τ → ∞ \tau\to\infty τ → ∞ . For a quadratic loss, with μ i \mu_i μ i the eigenvalues of F c F_c F c ,
q c ( τ , r ) = 1 2 ∑ i = 1 d c τ r 2 μ i 1 + τ r 2 μ i , (5) q_c(\tau,r)=\frac{1}{2}\sum_{i=1}^{d_c}\frac{\tau r^2\mu_i}{1+\tau r^2\mu_i}, \tag*{(5)} q c ( τ , r ) = 2 1 i = 1 ∑ d c 1 + τ r 2 μ i τ r 2 μ i , (5)
half the effective number of well-determined parameters of MacKay [25 ] , the soft rank of τ r 2 F c \tau r^2F_c τ r 2 F c : each direction with curvature above 1 / ( τ r 2 ) 1/(\tau r^2) 1/ ( τ r 2 ) contributes about one half, so at large τ \tau τ the estimate counts constrained directions, whereas q c ≈ 1 2 τ r 2 tr F c q_c\approx\tfrac{1}{2}\tau r^2\operatorname{tr}F_c q c ≈ 2 1 τ r 2 tr F c when all τ r 2 μ i \tau r^2\mu_i τ r 2 μ i are small. Unlike prior SGLD-based estimates ([24 ] ; Wang et al., [38 ] ), we fix U U U explicitly and compare each estimate with that of the quadratic approximation Q c ( u ) = 1 2 u T F c u Q_c(u)=\tfrac{1}{2}u^\mathsf{T}F_cu Q c ( u ) = 2 1 u T F c u on the same draws, isolating what lies beyond curvature.
Methodology: one object, four predictions
The shared quantity: a circuit’s responses. We view the transformer as a directed acyclic graph whose nodes are the token embedding, every head and MLP, and the unembedding, with an edge from w w w to v v v when v v v lies in a later layer (the unembedding counts as the last), because v v v reads the residual stream that w w w writes to. Node v v v outputs o v = f v ( ∑ w ∈ pa ( v ) o w ; θ v ) o_v=f_v\left(\sum_{w\in\operatorname{pa}(v)}o_w;\theta_v\right) o v = f v ( ∑ w ∈ pa ( v ) o w ; θ v ) , with pa ( v ) \operatorname{pa}(v) pa ( v ) its parents and θ v \theta_v θ v its parameters, and a circuit is a subgraph whose nodes carry the variables and operations of a circuit hypothesis (Geiger et al., [15 ] ). Fix inputs x ∼ D x\sim D x ∼ D with scored positions. The circuit’s responses are its predictions p θ ( ⋅ ∣ x ) p_\theta(\cdot\mid x) p θ ( ⋅ ∣ x ) on these inputs as a function of the parameters; both descriptions depend on this response map. MI characterizes the computation that produces the responses, and SLT the parameters that preserve them, through the loss (1), which vanishes exactly where every response is preserved. The two tests act on the same network: patching the output change Δ o c ( x ; u ) \Delta o_c(x;u) Δ o c ( x ; u ) caused by a displacement u u u into the unperturbed network reproduces the perturbed prediction exactly,
p θ ( u ) ( ⋅ ∣ x ) = p θ p a t c h [ Δ o c ( u ) ] ( ⋅ ∣ x ) , (6) p_{\theta(u)}(\mathord{\cdot}\mid x)=p_{\theta}^{\mathrm{patch}[\Delta o_c(u)]}(\mathord{\cdot}\mid x), \tag*{(6)} p θ ( u ) ( ⋅ ∣ x ) = p θ patch [ Δ o c ( u )] ( ⋅ ∣ x ) , (6)
so a parameter perturbation is an activation intervention, and projecting Δ o c \Delta o_c Δ o c onto variable subspaces shows which variable carries a geometric response (Appendix § 22 ).
Why the measurements should agree. Let G c ( u ) G_c(u) G c ( u ) collect the changes in the model’s log-odds (log-ratios of output probabilities) at every input, scored position, and intervention. Since all probabilities are positive, K c K_c K c lies between two multiples of ∥ G c ∥ 2 \lVert G_c\rVert^2 ∥ G c ∥ 2 near u = 0 u=0 u = 0 , so the local geometry of the loss is that of this response map.
Proposition 1 (Shared local constraints). If the Jacobian D G c DG_c D G c has constant rank ϱ c \varrho_c ϱ c near zero, then λ c = ϱ c / 2 \lambda_c=\varrho_c/2 λ c = ϱ c /2 . Moreover, K c ( t u ) = 1 2 t 2 u T F c u + o ( t 2 ) K_c(tu)=\frac{1}{2}t^2u^\mathsf{T}F_cu+o(t^2) K c ( t u ) = 2 1 t 2 u T F c u + o ( t 2 ) with F c = D G c ( 0 ) T Σ D G c ( 0 ) F_c=DG_c(0)^\mathsf{T}\Sigma DG_c(0) F c = D G c ( 0 ) T Σ D G c ( 0 ) , where Σ \Sigma Σ is the Fisher information of the categorical output distribution (the metric under which two nearby output distributions are as far apart as they are distinguishable), so that ker F c = ker D G c ( 0 ) \ker F_c=\ker DG_c(0) ker F c = ker D G c ( 0 ) .
The LLC counts the independent constraints that the responses place on the component, and F c F_c F c measures how strongly each parameter direction moves them (Appendix § 8 ). An intervention moves the same responses. Write Σ c \Sigma_c Σ c for the Fisher metric that the downstream network induces on the component’s output activation o c o_c o c : then F c F_c F c is its pullback through ∂ o c / ∂ u \partial o_c/\partial u ∂ o c / ∂ u , so the Fisher trace A c = 1 2 tr F c = 1 2 ∑ i ∥ ∂ o c / ∂ u i ∥ Σ c 2 A_c=\frac{1}{2}\operatorname{tr}F_c=\frac{1}{2}\sum_i\lVert\partial o_c/\partial u_i\rVert_{\Sigma_c}^2 A c = 2 1 tr F c = 2 1 ∑ i ∥ ∂ o c / ∂ u i ∥ Σ c 2 , and the KL effect of scaling the output by 1 + g 1+g 1 + g is 1 2 g 2 ∥ o c ∥ Σ c 2 \frac{1}{2}g^2\lVert o_c\rVert_{\Sigma_c}^2 2 1 g 2 ∥ o c ∥ Σ c 2 to leading order (Appendix § 21 .§ 3 ). Both are norms under the same metric, which grows as later components read that output, so the two grow together. P1: changes in a component’s Fisher information track changes in its causal effect. As a geometric coefficient is nonnegative and a task-score effect is signed, we measure effects by the KL of the ablated model from the clean one.
Beyond curvature: the quartic coefficient. Write q K q_K q K and q Q q_Q q Q for the estimate (4) computed with K c K_c K c and with its quadratic part Q c Q_c Q c under the same prior. Averaging the expansion (3), with Q c ( z ) = 1 2 z T F c z Q_c(z)=\frac{1}{2}z^\mathsf{T}F_cz Q c ( z ) = 2 1 z T F c z , under the prior gives (Appendix § 9 )
q Q = τ r 2 A c + O ( τ 2 r 4 ) , q K − q Q = τ r 4 B c + O ( r 6 ) . (7) q_Q=\tau r^2A_c+O(\tau^2r^4),\qquad q_K-q_Q=\tau r^4B_c+O(r^6). \tag*{(7)} q Q = τ r 2 A c + O ( τ 2 r 4 ) , q K − q Q = τ r 4 B c + O ( r 6 ) . (7)
The quartic coefficient B c = 1 8 ∑ i , j ∂ i 2 ∂ j 2 K c ( 0 ) B_c=\frac{1}{8}\sum_{i,j}\partial_i^2\partial_j^2K_c(0) B c = 8 1 ∑ i , j ∂ i 2 ∂ j 2 K c ( 0 ) is the average quartic term: positive when the loss rises faster than its quadratic approximation and negative when it saturates, with B c / A c B_c/A_c B c / A c the relative excess per unit r 2 r^2 r 2 . Expanding K c = E z ∼ p θ e G c K_c=\mathbb{E}_{z\sim p_\theta}e^{G_c} K c = E z ∼ p θ e G c shows that K 4 K_4 K 4 depends on the second derivatives of the response map and on the skewness and kurtosis of the output distribution, which all components share and which change as the predictions sharpen, so that components with identical Fisher matrices can differ in B c B_c B c (Appendix § 9 ). P2: B c B_c B c carries what an estimate registers beyond curvature, a trend shared across components rather than a component’s own effect.
Groups and pairs. When several components write the same variable, each ablation can be small while the joint ablation is large, and for a quadratic loss the joint LLC estimate of a group is at most the sum of its members’ estimates, with equality only for uncoupled directions (Equation (17) ). P3: for a redundant group, summed single-component geometry tracks summed single ablations rather than the joint ablation, and the joint estimate lies below the sum. For components a , b a,b a , b with outputs scaled by 1 + g a 1+g_a 1 + g a and 1 + g b 1+g_b 1 + g b , and G ( g a , g b ) G(g_a,g_b) G ( g a , g b ) collecting the log-odds changes, the mixed terms beyond the Fisher block (§ 3 ) are measured by the interaction curvature N a b N_{ab} N ab , the squared Σ \Sigma Σ -norm of the part of the joint response ∂ g a ∂ g b G ( 0 ) \partial_{g_a}\partial_{g_b}G(0) ∂ g a ∂ g b G ( 0 ) that neither component’s own response can produce (Equation (23) ; [11 ] ), and by the mixed quartic coefficient B a b = 1 4 ∂ g a 2 ∂ g b 2 ( K − K l i n ) ( 0 ) B_{ab}=\frac{1}{4}\partial_{g_a}^2\partial_{g_b}^2(K-K_{\mathrm{lin}})(0) B ab = 4 1 ∂ g a 2 ∂ g b 2 ( K − K lin ) ( 0 ) , with K l i n K_{\mathrm{lin}} K lin the loss of a reference model whose logits are linear in ( g a , g b ) (g_a,g_b) ( g a , g b ) (Equation (37) ); both vanish for logits linear in g a g_a g a and g b g_b g b . P4: N a b N_{ab} N ab and B a b B_{ab} B ab track the nonadditive part of joint ablations.
Findings
Following Figure 1 , inputs are Dyck(2, 10) or Dyck(4, 16) strings, written as space-separated brackets after the prefix Bracket s ::. At every position whose next token is a closer (a closing bracket), we score, for the unmodified (clean) model and for intervened ones, closer accuracy (whether the most probable token in the vocabulary is the correct closer) and the contrast S S S , the logit difference between the correct closer and one fixed, randomly drawn incorrect closer [40 ] . Table 1 lists the protocol of the geometric measurements, which share their inputs with the ablations.
A circuit for bracket completion and its development
We identify the circuit with standard MI tools, without geometry. Zero-ablating each of the 144 components, and greedily restoring components into an ablated model until 80% of the clean contrast is recovered (on k = 2 k=2 k = 2 strings), gives the candidates; attention patterns, probes, direct logit attribution (each component’s direct contribution to the output logits), and path patching (replacing a component’s output only on the path to one downstream component) assign roles to 26 circuit components in five stages: L 0. m l p L0.\mathrm{mlp} L 0. mlp and L 0 H 0 L0\mathrm{H0} L 0 H0 –L 7 H 2 L7\mathrm{H2} L 7 H2 encode the token, L 1 L1 L 1 –L 5. m l p L5.\mathrm{mlp} L 5. mlp , L 1 H 1 L1\mathrm{H1} L 1 H1 , L 2 H 1 L2\mathrm{H1} L 2 H1 , and L 2 H 5 L2\mathrm{H5} L 2 H5 update the stack, L 7 H 2 L7\mathrm{H2} L 7 H2 , L 7 H 4 L7\mathrm{H4} L 7 H4 , L 7 H 5 L7\mathrm{H5} L 7 H5 , L 9 H 4 L9\mathrm{H4} L 9 H4 , and L 14 H 1 L14\mathrm{H1} L 14 H1 read the required bracket type, L 11. m l p L11.\mathrm{mlp} L 11. mlp maps it to a closer, and L 13 L13 L 13 –L 15. m l p L15.\mathrm{mlp} L 15. mlp shape the final prediction (Figure 28 , Appendix § 24 ). To test the assignment we fit four variable subspaces on k = 4 k=4 k = 4 strings, TOKEN (token-dependent outputs of L 0. m l p L0.\mathrm{mlp} L 0. mlp ; 11 dimensions), STACK (an additive code of each stack slot’s occupancy and type; 24), POSITION (32), and ANSWER (centered closer unembeddings; 3): projecting the outputs of the 24 writing components onto the subspaces preserves 0.763 ± 0.011 0.763\pm0.011 0.763 ± 0.011 of the clean contrast on held-out strings, and removing STACK from the stack writers lowers held-out closer accuracy from 61.4% to 14.8% (58.4–62.6% for random subspaces of equal rank) and removing ANSWER from the 15 components with direct closer evidence to 39.3% (Appendix § 24 ).
Figure 28. A symbolic algorithm beside its transformer realization. Each row pairs a step of the stack algorithm (σ t \sigma_t σ t is the stack before token x t x_t x t ) with the components that implement it and the causal test behind the assignment (Appendix Q).
Development. Across checkpoints, prefix depth is linearly decodable already at initialization (layer-2 probe R 2 = 0.864 R^2=0.864 R 2 = 0.864 for k = 2 k=2 k = 2 ), whereas the distance to the most recent unmatched opener is learned (R 2 R^2 R 2 from 0.34 to 0.684; Appendix § 23 .§ 4 ), and the eight layer-0 heads act as a redundant group whose joint ablation far exceeds the sum of their single ablations (Appendix § 23 .§ 5 ).
Local geometry of circuit components across training
The measurement stays where predictions are preserved. At r = 0.025 r=0.025 r = 0.025 all 256 draws lie inside U U U for 99.96% of the 5,082 panel pairs, and at r = 0.05 r=0.05 r = 0.05 , 0.1, and 0.2 for 98.3%, 88.5%, and 52%; SGLD chains instead move each weight by several times its RMS, under standard settings too (Appendix § 18 ), so U U U must be imposed on the draws. We reweight the prior draws by e − τ K c e^{-\tau K_c} e − τ K c (importance weighting), so K c K_c K c and Q c Q_c Q c are compared on the same draws.
Inside U U U the loss is quadratic plus quartic. The mean loss agrees with its Fisher approximation to within 0.04% at r = 0.025 r = 0.025 r = 0.025 (median), and B c B_c B c estimated at r = 0.05 r = 0.05 r = 0.05 and r = 0.2 r = 0.2 r = 0.2 agrees with its value at r = 0.1 r = 0.1 r = 0.1 (median ratios 1.000 and 0.998; Appendix § 11 ), so B c B_c B c is a property of the checkpoint, not the radius. Importance weighting passes its diagnostics where τ \tau τ times the spread of K c K_c K c under the prior is below about one; there the posterior stays close to the prior, and q Q = τ r 2 A c q_Q = \tau r^2 A_c q Q = τ r 2 A c and q K − q Q = τ r 4 B c q_K - q_Q = \tau r^4 B_c q K − q Q = τ r 4 B c hold to within 0.4% and 1.1% (medians, r ≤ 0.1 r \le0.1 r ≤ 0.1 ; Appendices § 11 and § 15 ).
The quadratic part at every temperature. Beyond this region the quadratic part remains the spectral sum of Equation (5) and can be evaluated at every τ \tau τ and r r r without drawing. We compute the Fisher spectrum of all 144 components at all 154 checkpoints by stochastic Lanczos quadrature (Appendix § 12 ). In Figure 3 A, q Q / ( τ r 2 A c ) q_Q/(\tau r^2 A_c) q Q / ( τ r 2 A c ) is one for τ r 2 ≲ 10 \tau r^2 \lesssim10 τ r 2 ≲ 10 , departs from one near τ r 2 = 10 2 \tau r^2 = 10^2 τ r 2 = 1 0 2 , and falls to medians of 0.58, 0.28, and 0.04 at τ r 2 = 10 3 \tau r^2 = 10^3 τ r 2 = 1 0 3 , 10 4 10^4 1 0 4 , and 10 6 10^6 1 0 6 . The estimate leaves linearity and grows sublinearly, as ( τ r 2 ) 0.6 (\tau r^2)^{0.6} ( τ r 2 ) 0.6 between 10 4 10^4 1 0 4 and 10 6 10^6 1 0 6 : it counts the directions whose curvature exceeds 1 / ( τ r 2 ) 1/(\tau r^2) 1/ ( τ r 2 ) , and because the spectrum has no gap the count keeps growing with τ \tau τ (Appendix § 12 ). The quartic correction to this count, evaluated on draws from the Gaussian posterior N ( 0 , ( τ F c + I / r 2 ) − 1 ) \mathcal{N}(0,(\tau F_c + I/r^2)^{-1}) N ( 0 , ( τ F c + I / r 2 ) − 1 ) (Appendix § 13 ), equals τ r 4 B c \tau r^4 B_c τ r 4 B c throughout and is 0.032% of q Q q_Q q Q at τ r 2 = 10 4 \tau r^2 = 10^4 τ r 2 = 1 0 4 (90% of the cells below 0.33%); inside U U U the large-temperature estimate is this count.
Figure 3. Local geometry across training. A: median over components of q Q / ( τ r 2 A c ) q_Q/(\tau r^2 A_c) q Q / ( τ r 2 A c ) at the final checkpoint (solid) and at step 8,000 (dashed), with interquartile bands. B–D: A c A_c A c , q Q q_Q q Q at τ r 2 = 10 6 \tau r^2 = 10^6 τ r 2 = 1 0 6 , and zero-ablation KL of all 144 components, circuit in blue.
The Fisher trace and the quartic coefficient develop differently. The Fisher trace of every component falls during training, by a median factor of 9.4, and rises again late (Figure 3 B), whereas B c / A c B_c/A_c B c / A c peaks between steps 128 and 512 and then decays (Figure 11 ). These trajectories are largely shared (median pairwise rank correlation 0.91 for A c A_c A c and 0.93 for B c B_c B c , against 0.47 for the ablation effects), so a component’s geometry is a modulation of a model-wide trend.
Figure 11. Local geometry across training, by circuit stage (33-component panel). Rows are the circuit stages (stages 1–3 of Figure 28 and the answer-writing MLPs of stages 4–5) and the seven controls. Columns: Fisher trace A c A_c A c (log scale), relative quartic coefficient B c / A c B_c/A_c B c / A c at r = 0.1 r = 0.1 r = 0.1 (symmetric logarithmic scale), and the drop in the closer contrast under zero-ablation measured on the same inputs. All 154 checkpoints; step 0 is drawn at the left of the logarithmic axis.
Correspondence between geometry and causal roles
Across training, the Fisher part tracks causal roles (P1). At the final checkpoint (Q1) the Metropolis-chain LLC estimate of all 144 components ranks them as zero-ablation does (Spearman 0.64; Appendix § 19 ). Across training (Q2), for each of the 144 components we correlate the A c A_c A c and ablation-KL trajectories directly (Spearman), after regressing both on clean loss, weight scale, and training step (partial rank correlation), and between adjacent-checkpoint changes given the same covariates (block bootstrap, blocks of 16 checkpoints). The medians are 0.59 (Spearman) and 0.15 (partial) for levels and 0.51 for changes, positive for 97% of the components and with a bootstrap interval above zero for 88% (Figure 4 A–B); the change association is 0.56 for the 26 circuit components and 0.49 for the other 118. Much of this co-movement is shared across the model: pairing each component’s A c A_c A c with the population-median ablation trajectory gives 0.50, and with a random other component’s trajectory 0.26; the component-specific part, the excess over such pairings, is positive for 82% of the components. The gain Fisher F g g = ∥ o c ∥ Σ c 2 F_{gg} = \lVert o_c \rVert_{\Sigma_c}^2 F g g = ∥ o c ∥ Σ c 2 , the squared norm of the component’s output under the downstream Fisher metric of § 4 , tracks the zero-ablation KL (partial 0.98 in levels, 0.97 in changes) and is specific to its component (changes 0.12 against the population-median trajectory), and, once among the covariates, leaves A c A_c A c no association with the ablation (0.02 for levels, 0.07 for changes). Splitting each change of A c A_c A c exactly among the downstream, own, and upstream weights (Shapley values over hybrid checkpoints; Appendix § 14 ) assigns them 0.44, 0.21, and 0.35 of it, and the late rise of A c A_c A c comes from the input (median share 1.08; own weights −0.23).
Figure 4. Correspondence between local geometry and mechanism. A: median partial rank correlation over the 144 components between each geometric quantity and each target, for levels and adjacent-checkpoint changes. B: per-component change associations with the ablation KL (circuit blue, others gray, medians as ticks; independent batch in orange). C: median partial correlation between N a b N_{ab} N ab and Y a b Y_{ab} Y ab for 43 circuit and 43 control pairs on four batches, with 95% block-bootstrap intervals.
The large-temperature estimate counts rather than sums. q Q q_Q q Q at τ r 2 = 10 6 \tau r^2 = 10^6 τ r 2 = 1 0 6 is specific to its component (changes 0.40 against its own ablation, 0.13 against the population median) and tracks ablation changes across the population (0.47 for the 118 non-circuit components) but weakly for the circuit components (0.09; levels − 0.01 -0.01 − 0.01 ), within heads and within MLPs alike (Appendix § 12 ). A circuit component’s causal growth is growth in magnitude along directions that are already constrained, which the trace registers and the count does not (Figure 3 B–D).
The quartic part is what a nonlocal sampler registers (P2). With A c A_c A c among the covariates, B c B_c B c tracks the component’s nonlocal SGLD estimate (partial 0.50; Appendix § 16 ), whereas A c A_c A c and F g g F_{gg} F g g do not (− 0.07 -0.07 − 0.07 , − 0.02 -0.02 − 0.02 ). B c B_c B c is negatively associated with the component’s effect in levels (− 0.24 -0.24 − 0.24 ; changes − 0.07 -0.07 − 0.07 ) and weakly with the opener-distance probe (0.06), whose formation the nonlocal estimate tracks (0.46; Appendix § 18 ) and no local quantity does. The nonlocal estimator registers the growth of the loss beyond its quadratic part, a shared trend that does not mark a component’s role.
The layer-0 heads as a group (P3). The sum of the eight heads’ Fisher traces tracks the sum of their individual ablation effects (adjacent changes 0.71 on the KL) but not their joint ablation (changes 0.03; Table 20 ), and the group’s joint LLC estimate lies 5.8% below the sum of its members’ (Appendix § 19 ), as Equation (17) predicts. An independent input batch reproduces the panel’s trajectories (rank correlation 0.997 for A c A_c A c ) and associations (A c A_c A c changes 0.55 on both; Figure 4 B).
Table 20. Summed single-head geometry against single and joint ablations (P3). Associations across the 148 confirmation checkpoints between the sum of the eight layer-0 heads’ Fisher traces and each target; partial rank correlations given clean loss, weight scale, and training step, adjacent-change partials given the same, and moving-block bootstrap intervals (blocks of 16). The summed trace reaches its minimum at step 6,000 and the joint ablation effect its maximum at step 15,000, which produces the negative level association.
Pairwise geometry tracks circuit interactions
To test P4 we measure 43 circuit pairs (28 among the layer-0 heads; Appendix § 10 ) and 43 matched controls. The target is the nonadditive part of the joint ablation beyond the linearized model, Y a b = ∣ f ( − 1 , − 1 ) − f ( − 1 , 0 ) − f ( 0 , − 1 ) ∣ Y_{ab} = \lvert f(-1,-1) - f(-1,0) - f(0,-1)\rvert Y ab = ∣ f ( − 1 , − 1 ) − f ( − 1 , 0 ) − f ( 0 , − 1 )∣ with f = K − K l i n f = K - K_{\mathrm{lin}} f = K − K lin , which has the network’s Fisher matrix in ( g a , g b ) (g_a,g_b) ( g a , g b ) , so the association lies beyond curvature. Over circuit pairs, the median partial rank correlation of the trajectories, given that Fisher matrix, clean loss, and step, is 0.16, 0.14, and 0.16 on three independent batches, against 0.05 for control pairs (Figure 4 C; B a b B_{ab} B ab : 0.16–0.19, Appendix § 10 ).
Discussion
What the correspondence offers. The correspondence gives SLT’s developmental measurements a mechanistic reading, provided that they stay where predictions are preserved, and says which quantity answers which question: the Fisher trace follows the model-wide course of causal effect, whereas the gain Fisher and the large-temperature count follow a component’s own role and give MI a measure of it that needs no replacement value. Limitations: one model, one synthetic task, one input batch, and fixed coordinates (Appendix § 11 ); on natural bracket contexts, where the circuit was not identified, the final-checkpoint correspondence falls from 0.64 to 0.16 (Appendix § 19 ).
Conclusion
MI and SLT describe one object, a circuit’s responses and the parameters that preserve them; in Pythia-1B their descriptions correspond term by term: Fisher trace to the trend of causal effect, large-temperature count to a component’s own role, quartic part to what a nonlocal sampler registers, pair terms to circuit interactions.
AI Use Statement
Generative AI tools assisted with software implementation, experiment orchestration, literature search, illustration, and manuscript preparation. The authors are responsible for the content, experimental claims, and accompanying software.
Reproducibility Statement
Table 1 lists the protocol of the main measurements. The appendices specify the circuit interventions, sampling objectives, parameter conventions, numerical checks, and derivative estimators. Proofs appear in Appendices § 8 and § 9 ; the developmental geometry protocol is in Appendix § 11 , the direct posterior sampling sweep in Appendix § 15 , the developmental interaction protocol in Appendix § 10 , and further experimental details in Appendix § 16 .
Acknowledgments
This work was partially supported by NSF 2303037, 2312501, 2531008, 2609582, and 2617441; DARPA HR001126CE054; NIH OT2OD038003, U54OD036472, U54DK097771, and U54HG012517; the SRC JUMP 2.0 Center; CDSC Industry Partners; Amazon Research Awards; Amazon; Google; Google Gifts; NEC; Optum; and Snapchat.
The local bridge
Precise local setting
Fix finitely many input–intervention pairs indexed by a a a , with weights w a > 0 w_a>0 w a > 0 summing to one, and a finite output alphabet of size M ≥ 2 M\ge2 M ≥ 2 . This includes a fixed evaluation set; a population extension requires additional uniform regularity and integrability assumptions. Write p a ( u ) = p θ ⋆ a + E c u ( ⋅ ∣ x a ) p_a(u)=p_{\theta^a_\star+E_cu}(\cdot\mid x_a) p a ( u ) = p θ ⋆ a + E c u ( ⋅ ∣ x a ) and assume every coordinate is positive and analytic in a neighborhood of u = 0 u=0 u = 0 . Define log-odds coordinates
r a , j ( u ) = log p a , j ( u ) p a , M ( u ) , j = 1 , … , M − 1 , (8) r_{a,j}(u)=\log\frac{p_{a,j}(u)}{p_{a,M}(u)},\qquad j=1,\ldots,M-1, \tag*{(8)} r a , j ( u ) = log p a , M ( u ) p a , j ( u ) , j = 1 , … , M − 1 , (8)
and stack G c ( u ) = ( r a ( u ) − r a ( 0 ) ) a G_c(u)=(r_a(u)-r_a(0))_a G c ( u ) = ( r a ( u ) − r a ( 0 ) ) a . The weights can equivalently be absorbed into the residual blocks. The nonnegative analytic loss is
K c ( u ) = ∑ a w a D K L ( p a ( 0 ) ∥ p a ( u ) ) . (9) K_c(u)=\sum_a w_aD_{\mathrm{KL}}(p_a(0)\Vert p_a(u)). \tag*{(9)} K c ( u ) = a ∑ w a D KL ( p a ( 0 ) ∥ p a ( u )) . (9)
We localize the integral to a sufficiently small bounded neighborhood with a smooth prior bounded above and below by positive constants. For K c ≡ 0 K_c\equiv0 K c ≡ 0 , its partition integral is constant and we assign exponent λ c = 0 \lambda_c=0 λ c = 0 .
Proof of Proposition 1
For one categorical distribution, set A ( r ) = log ( 1 + ∑ j = 1 M − 1 e r j ) A(r)=\log(1+\sum_{j=1}^{M-1}e^{r_j}) A ( r ) = log ( 1 + ∑ j = 1 M − 1 e r j ) . Its KL divergence from the reference is
A ( r ) − A ( r 0 ) − ∇ A ( r 0 ) T ( r − r 0 ) . (10) A(r)-A(r_0)-\nabla A(r_0)^{\mathsf{T}}(r-r_0). \tag*{(10)} A ( r ) − A ( r 0 ) − ∇ A ( r 0 ) T ( r − r 0 ) . (10)
The Hessian at r 0 r_0 r 0 is W = diag ( p ˉ ) − p ˉ p ˉ T W=\operatorname{diag}(\bar p)-\bar p\bar p^{\mathsf{T}} W = diag ( p ˉ ) − p ˉ p ˉ T , where p ˉ = ( p 1 , … , p M − 1 ) \bar p=(p_1,\ldots,p_{M-1}) p ˉ = ( p 1 , … , p M − 1 ) . Positivity of all M M M probabilities makes W W W positive definite. Continuity, a sufficiently small neighborhood, and finitely many pairs give constants 0 < b < B < ∞ 0<b<B<\infty 0 < b < B < ∞ such that
b ∥ G c ( u ) ∥ 2 ≤ K c ( u ) ≤ B ∥ G c ( u ) ∥ 2 . (11) b\lVert G_c(u)\rVert^2\le K_c(u)\le B\lVert G_c(u)\rVert^2. \tag*{(11)} b ∥ G c ( u ) ∥ 2 ≤ K c ( u ) ≤ B ∥ G c ( u ) ∥ 2 . (11)
If G c G_c G c has constant rank ϱ c \varrho_c ϱ c near zero, the analytic constant-rank theorem gives local source and target coordinates in which it takes the form ( s 1 , … , s ϱ c , 0 , … , 0 ) (s_1,\ldots,s_{\varrho_c},0,\ldots,0) ( s 1 , … , s ϱ c , 0 , … , 0 ) . Coordinate changes have bounded, nonsingular Jacobians locally, so K c ≍ ∑ j = 1 ϱ c s j 2 K_c\asymp\sum_{j=1}^{\varrho_c}s_j^2 K c ≍ ∑ j = 1 ϱ c s j 2 . Its partition integral scales as t − ϱ c / 2 t^{-\varrho_c/2} t − ϱ c /2 : the constrained coordinates give Gaussian integrals and the remaining coordinates contribute bounded factors. Thus λ c = ϱ c / 2 \lambda_c=\varrho_c/2 λ c = ϱ c /2 , with multiplicity one for ϱ c > 0 \varrho_c>0 ϱ c > 0 .
For J a = D r a ( 0 ) J_a=Dr_a(0) J a = D r a ( 0 ) , Taylor expansion gives
F c = ∑ a w a J a T W a J a , K c ( t v ) = 1 2 t 2 v T F c v + o ( t 2 ) . (12) F_c=\sum_a w_aJ_a^{\mathsf{T}}W_aJ_a,\qquad K_c(tv)=\frac{1}{2}t^2v^{\mathsf{T}}F_cv+o(t^2). \tag*{(12)} F c = a ∑ w a J a T W a J a , K c ( t v ) = 2 1 t 2 v T F c v + o ( t 2 ) . (12)
Every W a W_a W a is positive definite, hence v T F c v = 0 v^{\mathsf{T}}F_cv=0 v T F c v = 0 if and only if J a v = 0 J_av=0 J a v = 0 for all a a a , equivalently D G c ( 0 ) v = 0 DG_c(0)v=0 D G c ( 0 ) v = 0 . Thus the relevant differential metric is the categorical Fisher at the reference outputs.
Connection to activation interventions. For the finite transport in Equation (6) , replace every operation affected by the selected parameters with its output in the perturbed computation. All remaining structural functions are fixed. Induction over the directed computation graph then gives identical outputs: each replaced node receives its prescribed perturbed value, and every other node receives the same parent values and applies the same function. The patch must include every relevant occurrence of the component. In the transformer experiments, it covers the complete head or MLP output at every sequence position.
Suppose the component parameters affect the rest of the computation only through its output h c ( x ; u ) h_c(x;u) h c ( x ; u ) , and hold the other component parameters fixed. For a direction v v v , let δ h c ( x ) = D u h c ( x ; 0 ) v \delta h_c(x)=D_uh_c(x;0)v δ h c ( x ) = D u h c ( x ; 0 ) v . Applying the activation intervention h c ( x ) ← h c ( x ; 0 ) + t δ h c ( x ) h_c(x)\leftarrow h_c(x;0)+t\delta h_c(x) h c ( x ) ← h c ( x ; 0 ) + t δ h c ( x ) has the same first-order output derivative as changing the component parameters to t v tv t v . This follows from the chain rule for the fixed downstream computation. The perturbation must include all relevant occurrences of a shared component, and a single v v v must generate the perturbations across inputs. An arbitrary patch, zero-ablation, or mean replacement need not meet these conditions. Vanishing first-order response also does not exclude a higher-order response along a straight parameter line.
Interventional versus observational equivalence. The theorem applies to I = { i d } \mathcal{I} = \{\mathrm{id}\} I = { id } as well as to an intervention-augmented loss. The former has a weaker zero set: two implementations can have identical unperturbed outputs and different responses to internal interventions. The posterior and gain measurements evaluate the identity member at perturbed parameters. The aligned component-gain interventions directly probe the same response map, and the separate MI intervention battery establishes its computational interpretation.
Feature constraints and distribution changes
Suppose a mechanistic hypothesis supplies analytic residuals H j : U → R d j H_j: U \to\mathbb{R}^{d_j} H j : U → R d j with H j ( 0 ) = 0 H_j(0) = 0 H j ( 0 ) = 0 and K c ≍ ∑ j ∥ H j ∥ 2 K_c \asymp\sum_j \lVert H_j\rVert^2 K c ≍ ∑ j ∥ H j ∥ 2 . If their stacked Jacobian has full row rank R = ∑ j d j R = \sum_j d_j R = ∑ j d j at zero, it remains full row rank locally. The same normal-form calculation gives λ c = R / 2 \lambda_c = R/2 λ c = R /2 . This is the precise sense in which independently observable task constraints overlap, additivity need not hold. If a feature is decodable but unused downstream, its residual need not be bounded by the output loss, so the needed local equivalence fails. More generally, if b K 1 ≤ K 2 ≤ B K 1 bK_1 \le K_2 \le BK_1 b K 1 ≤ K 2 ≤ B K 1 near the reference, their local integrability thresholds, and hence real log canonical thresholds (RLCTs), agree. One sufficient condition for distribution invariance is bounded density ratio: if 0 < b ≤ d Q / d P ≤ B < ∞ 0 < b \le dQ/dP \le B < \infty 0 < b ≤ d Q / d P ≤ B < ∞ on the input–intervention pairs, then nonnegativity of pointwise KL gives b K P ≤ K Q ≤ B K P bK_P \le K_Q \le BK_P b K P ≤ K Q ≤ B K P . Equality of RLCTs under this condition does not imply equality of finite-temperature localized estimates. It also does not apply merely because two datasets concern the same formal language.
Counterexamples and finite measurements
Constraint dimension is not effect size. Let a Gaussian output have unit variance and mean s θ s\theta s θ , with θ ⋆ = 1 \theta^\star= 1 θ ⋆ = 1 and s > 0 s > 0 s > 0 . Its reference KL is K ( θ ) = 1 2 s 2 ( θ − 1 ) 2 K(\theta) = \frac{1}{2}s^2(\theta- 1)^2 K ( θ ) = 2 1 s 2 ( θ − 1 ) 2 , giving λ = 1 / 2 \lambda= 1/2 λ = 1/2 for every s s s . Zeroing the parameter has KL effect s 2 / 2 s^2/2 s 2 /2 , which can be arbitrarily small or large. The geometric coefficient quantifies a nonnegative constraint; the sign of a component’s contribution is supplied by the chosen causal readout.
Redundancy and grouping. For K ( u , v ) = ( u + v ) 2 K(u,v) = (u+v)^2 K ( u , v ) = ( u + v ) 2 , the joint coefficient is 1 / 2 1/2 1/2 although the two one-dimensional restricted coefficients each equal 1 / 2 1/2 1/2 . Both parameters constrain the same output direction. A nonlinear example is f ( a , b ) = a + b − a b f(a,b) = a+b-ab f ( a , b ) = a + b − ab at ( a , b ) = ( 1 , 1 ) (a,b) = (1,1) ( a , b ) = ( 1 , 1 ) : setting either parameter to zero leaves f = 1 f = 1 f = 1 , whereas setting both to zero changes f f f to zero. In local coordinates u = a − 1 u = a-1 u = a − 1 , v = b − 1 v = b-1 v = b − 1 , its squared output error is u 2 v 2 u^2v^2 u 2 v 2 . Each individual restriction is identically zero, but the joint partition integral scales as t − 1 / 2 t^{-1/2} t − 1/2 up to logs. This joint geometry and finite joint effects can reveal structure invisible to both marginal quantities. The measured bank can be compared with these examples through its joint and marginal geometric and causal responses.
Why mean replacement can conceal a context interaction. Consider a component with output h a , b ( t , x ) = a m ( t ) + b g ( x ) h_{a,b}(t,x) = a\,m(t) + b\,g(x) h a , b ( t , x ) = a m ( t ) + b g ( x ) and a Gaussian task readout N ( h a , b , 1 ) \mathcal{N}(h_{a,b},1) N ( h a , b , 1 ) , centered at ( a ⋆ , b ⋆ ) (a_\star,b_\star) ( a ⋆ , b ⋆ ) . Here m ( t ) m(t) m ( t ) is a position-dependent write and g ( x ) g(x) g ( x ) is a context feature. Suppose g ( x ) = 0 g(x) = 0 g ( x ) = 0 on a synthetic distribution D s D_s D s , with E D s m ( t ) 2 > 0 \mathbb{E}_{D_s}m(t)^2 > 0 E D s m ( t ) 2 > 0 . Its reference loss is
K D s ( a , b ) = 1 2 E D s [ m ( t ) 2 ] ( a − a ⋆ ) 2 , (13) K_{D_s}(a,b) = \frac{1}{2}\mathbb{E}_{D_s}[m(t)^2](a-a_\star)^2, \tag*{(13)} K D s ( a , b ) = 2 1 E D s [ m ( t ) 2 ] ( a − a ⋆ ) 2 , (13)
so this component has local learning coefficient 1 / 2 1/2 1/2 and a positive LLC estimate. Position-conditioned mean replacement at the reference leaves h a ⋆ , b ⋆ h_{a_\star,b_\star} h a ⋆ , b ⋆ unchanged on D s D_s D s , giving zero causal effect. On a richer distribution D n D_n D n with E [ g ( x ) ∣ t ] = 0 \mathbb{E}[g(x)\mid t] = 0 E [ g ( x ) ∣ t ] = 0 , E [ g ( x ) 2 ] > 0 \mathbb{E}[g(x)^2] > 0 E [ g ( x ) 2 ] > 0 , and b ⋆ ≠ 0 b_\star\ne0 b ⋆ = 0 , the same replacement removes b ⋆ g ( x ) b_\star g(x) b ⋆ g ( x ) and has expected reference KL
E D n K m e a n = 1 2 b ⋆ 2 E D n [ g ( x ) 2 ] > 0. (14) \mathbb{E}_{D_n}K_{\mathrm{mean}} = \frac{1}{2}b_\star^2\mathbb{E}_{D_n}[g(x)^2] > 0. \tag*{(14)} E D n K mean = 2 1 b ⋆ 2 E D n [ g ( x ) 2 ] > 0. (14)
Geometric screening on D s D_s D s identifies the component through its constrained average write; interventions on D n D_n D n reveal its context-dependent contribution. This example explains how the two measurements can guide discovery at different stages of a circuit investigation.
A regular model can have a singular component slice. The residual H ( x , y ) = y + x 2 H(x,y) = y+x^2 H ( x , y ) = y + x 2 has rank one everywhere near the origin, and K ( x , y ) = H ( x , y ) 2 K(x,y) = H(x,y)^2 K ( x , y ) = H ( x , y ) 2 has joint coefficient 1 / 2 1/2 1/2 . Freezing y = 0 y = 0 y = 0 restricts the loss to x 4 x^4 x 4 , whose integral scales as t − 1 / 4 t^{-1/4} t − 1/4 , so the restricted coefficient is 1 / 4 1/4 1/4 despite zero restricted Jacobian rank at the origin. A pointwise rank calculation is therefore insufficient for component-restricted LLCs. The constant-rank assumption in Proposition 1 is essential.
Finite-time sampling response. The all-step average of the SGLD sweep (Appendix I) need not be stationary to have a geometric interpretation in an ideal quadratic model. Let K ( u ) = 1 2 u T H u K(u)=\frac{1}{2}u^{\mathsf T}Hu K ( u ) = 2 1 u T H u , H ⪰ 0 H\succeq0 H ⪰ 0 , start at u 0 = 0 u_0=0 u 0 = 0 , and use the exact loss gradient in the update u s + 1 = [ I − η ( τ H + γ I ) / 2 ] u s + η ξ s u_{s+1}=[I-\eta(\tau H+\gamma I)/2]u_s+\sqrt{\eta}\,\xi_s u s + 1 = [ I − η ( τ H + γ I ) /2 ] u s + η ξ s , with independent standard Gaussian ξ s \xi_s ξ s . For an eigenvalue μ i \mu_i μ i of H H H , set a i = 1 − η ( τ μ i + γ ) / 2 a_i=1-\eta(\tau\mu_i+\gamma)/2 a i = 1 − η ( τ μ i + γ ) /2 and assume ∣ a i ∣ < 1 \lvert a_i\rvert<1 ∣ a i ∣ < 1 . The variance of its coordinate before update s s s is η ∑ j = 0 s − 1 a i 2 j \eta\sum_{j=0}^{s-1}a_i^{2j} η ∑ j = 0 s − 1 a i 2 j , hence for S S S pre-update losses
E [ 1 S ∑ s = 0 S − 1 K ( u s ) ] = τ η 2 ∑ i : μ i > 0 μ i 1 − a i 2 [ 1 − 1 − a i 2 S S ( 1 − a i 2 ) ] . (15) \mathbb{E}\left[\frac{1}{S}\sum_{s=0}^{S-1}K(u_s)\right]=\frac{\tau\eta}{2}\sum_{i:\,\mu_i>0}\frac{\mu_i}{1-a_i^2}\left[1-\frac{1-a_i^{2S}}{S(1-a_i^2)}\right]. \tag*{(15)} E [ S 1 s = 0 ∑ S − 1 K ( u s ) ] = 2 τ η i : μ i > 0 ∑ 1 − a i 2 μ i [ 1 − S ( 1 − a i 2 ) 1 − a i 2 S ] . (15)
For S ≥ 2 S\ge2 S ≥ 2 , each positive-curvature direction contributes positively and null directions contribute zero. When H = F c H=F_c H = F c is the matched-KL quadratic Hessian, this connects finite perturbation response to the task-sensitive subspace in Proposition 1. The weights depend on curvature, step size, and run length. This finite-response calculation assumes an exact quadratic loss, a minimum at the initial point, and no additional minibatch-gradient noise. It provides an explicit link between finite-time sampling and task-sensitive directions under these assumptions.
Finite localization retains curvature scale. In the exactly quadratic case L ( u ) − L ( 0 ) = 1 2 u T H u L(u)-L(0)=\frac{1}{2}u^{\mathsf T}Hu L ( u ) − L ( 0 ) = 2 1 u T H u , with H ⪰ 0 H\succeq0 H ⪰ 0 , let q ( u ) ∝ exp [ − a L ( u ) − 1 2 γ ∥ u ∥ 2 ] q(u)\propto\exp[-aL(u)-\frac{1}{2}\gamma\lVert u\rVert^2] q ( u ) ∝ exp [ − a L ( u ) − 2 1 γ ∥ u ∥ 2 ] , where a = n β > 0 a=n\beta>0 a = n β > 0 and γ > 0 \gamma>0 γ > 0 . If μ i \mu_i μ i are the eigenvalues of H H H , the Gaussian calculation gives
a { E q L − L ( 0 ) } = 1 2 ∑ i a μ i a μ i + γ . (16) a\{\mathbb{E}_qL-L(0)\}=\frac{1}{2}\sum_i\frac{a\mu_i}{a\mu_i+\gamma}. \tag*{(16)} a { E q L − L ( 0 )} = 2 1 i ∑ a μ i + γ a μ i . (16)
It approaches rank ( H ) / 2 \operatorname{rank}(H)/2 rank ( H ) /2 as a → ∞ a\to\infty a → ∞ at fixed γ \gamma γ , but at finite a a a it depends on curvature scale and localization. Shared asymptotic RLCTs can therefore coexist with different finite localized responses, even in this regular example.
Joint component geometry with a factorized prior. The quadratic model also predicts how component measurements combine. Partition the local coordinates into disjoint components and set A = τ r 2 F ⪰ 0 A=\tau r^2F\succeq0 A = τ r 2 F ⪰ 0 in their common prior coordinates. The joint response is q j o i n t = 1 2 tr [ A ( I + A ) − 1 ] q_{\mathrm{joint}}=\frac{1}{2}\operatorname{tr}[A(I+A)^{-1}] q joint = 2 1 tr [ A ( I + A ) − 1 ] ; freezing the other components gives q j = 1 2 tr [ A j j ( I + A j j ) − 1 ] q_j=\frac{1}{2}\operatorname{tr}[A_{jj}(I+A_{jj})^{-1}] q j = 2 1 tr [ A j j ( I + A j j ) − 1 ] . Then
q j o i n t ≤ ∑ j q j . (17) q_{\mathrm{joint}}\le\sum_j q_j. \tag*{(17)} q joint ≤ j ∑ q j . (17)
Indeed, the Schur-complement identity gives ( I + A ) j j − 1 ⪰ ( I + A j j ) − 1 (I+A)^{-1}_{jj}\succeq(I+A_{jj})^{-1} ( I + A ) j j − 1 ⪰ ( I + A j j ) − 1 . Taking traces and using A ( I + A ) − 1 = I − ( I + A ) − 1 A(I+A)^{-1}=I-(I+A)^{-1} A ( I + A ) − 1 = I − ( I + A ) − 1 proves the inequality. Equality holds when the off-diagonal blocks vanish; coupling gives a strict inequality at finite positive temperature and scale. This subadditivity measures coupling of local task-sensitive directions. Joint causal removal can instead have a superadditive effect on the output, as the explicit redundancy example above shows. The two measurements thus characterize different aspects of a shared component group. Our bank experiment uses the product of the eight individual head priors so that the marginal and joint measurements share this chart.
An off-minimum center can give a negative statistic. For L ( u ) = 1 2 ( u − b ) 2 L(u)=\frac{1}{2}(u-b)^2 L ( u ) = 2 1 ( u − b ) 2 centered at u = 0 u=0 u = 0 , the same localized posterior has variance ( a + γ ) − 1 (a+\gamma)^{-1} ( a + γ ) − 1 and mean a b / ( a + γ ) ab/(a+\gamma) ab / ( a + γ ) . Consequently
a { E q L − L ( 0 ) } = a 2 ( a + γ ) − a 2 b 2 ( a + 2 γ ) 2 ( a + γ ) 2 . (18) a\{\mathbb{E}_qL-L(0)\}=\frac{a}{2(a+\gamma)}-\frac{a^2b^2(a+2\gamma)}{2(a+\gamma)^2}. \tag*{(18)} a { E q L − L ( 0 )} = 2 ( a + γ ) a − 2 ( a + γ ) 2 a 2 b 2 ( a + 2 γ ) . (18)
which is negative for sufficiently large ∣ b ∣ \lvert b\rvert ∣ b ∣ . The loss is nonnegative; it is the subtraction of the loss at a nonminimal center that produces the sign. The signed statistic reflects displacement from a nonminimal center; the RLCT remains nonnegative.
The experimental objectives differ. The developmental SGLD loss is
L ( θ ) = 1 B ( T − 1 ) ∑ b = 1 B ∑ t = 1 T − 1 m b t [ − log p θ ( x b , t + 1 ∣ x b , ≤ t ) ] . (19) L(\theta)=\frac{1}{B(T-1)}\sum_{b=1}^{B}\sum_{t=1}^{T-1}m_{bt}\left[-\log p_\theta(x_{b,t+1}\mid x_{b,\le t})\right]. \tag*{(19)} L ( θ ) = B ( T − 1 ) 1 b = 1 ∑ B t = 1 ∑ T − 1 m b t [ − log p θ ( x b , t + 1 ∣ x b , ≤ t ) ] . (19)
where m b t m_{bt} m b t indicates that the target is a closer and p θ p_{\theta} p θ is normalized over the full vocabulary. Non-closer positions contribute zero, but remain in the denominator. Hence closer frequency changes the overall loss scale, which matters to a finite localized statistic such as (16) . On fixed weighted contexts and for a conditional target distribution q q q , cross-entropy excess L q ( θ ) − L q ( θ ⋆ ) L_q(\theta)-L_q(\theta^\star) L q ( θ ) − L q ( θ ⋆ ) equals the reference KL if q = p θ ⋆ q=p_{\theta^\star} q = p θ ⋆ . The developmental objective uses deterministic closer labels, whereas the matched reference loss uses the checkpoint’s full output distribution as its target.
Readout choice also selects different functional relations. Adding a common constant to all closer logits leaves every closer–closer logit difference unchanged, while changing their probability relative to non-bracket tokens and thus changing (19) . Conversely, two models can give the same correct-token probability while allocating the remaining probability differently among distractors. The matched experiment fixes the output discrepancy and compares parameter and activation perturbations under that common readout.
Nonquadratic geometry of circuit responses
This section relates the measured geometric quantities to a common circuit response map. All statements fix the input distribution, output readout, component restriction, and parameter coordinates. For a finite empirical distribution, the expectations below are finite weighted sums over scored positions and output symbols.
The response map and its quadratic approximation
Let p 0 ( y ∣ x ) p_0(y \mid x) p 0 ( y ∣ x ) be the checkpoint distribution and let z ( u ; x ) z(u;x) z ( u ; x ) be the logits after a component-restricted parameter change u u u . Define the centered logit response
G ( u ; x , y ) = z ( u ; x , y ) − z ( 0 ; x , y ) − E Y ∼ p 0 ( ⋅ ∣ x ) [ z ( u ; x , Y ) − z ( 0 ; x , Y ) ] . (20) G(u;x,y)=z(u;x,y)-z(0;x,y)-\mathbb{E}_{Y\sim p_0(\cdot\mid x)}[z(u;x,Y)-z(0;x,Y)]. \tag*{(20)} G ( u ; x , y ) = z ( u ; x , y ) − z ( 0 ; x , y ) − E Y ∼ p 0 ( ⋅ ∣ x ) [ z ( u ; x , Y ) − z ( 0 ; x , Y )] . (20)
Equip these responses with the inner product ⟨ f , g ⟩ 0 = E x ∼ D , Y ∼ p 0 [ f ( x , Y ) g ( x , Y ) ] \langle f,g\rangle_0=\mathbb{E}_{x\sim D,Y\sim p_0}[f(x,Y)g(x,Y)] ⟨ f , g ⟩ 0 = E x ∼ D , Y ∼ p 0 [ f ( x , Y ) g ( x , Y )] . Subtracting the reference-weighted mean removes the arbitrary additive constant in each logit vector. The same expression is obtained by centering log p u − log p 0 \log p_u-\log p_0 log p u − log p 0 , because the softmax normalizer is constant across output symbols.
The predictive discrepancy is
K ( u ) = E D KL ( p 0 ∥ p u ) = E D log E p 0 exp G ( u ) . (21) K(u)=\mathbb{E}_D\operatorname{KL}(p_0\Vert p_u)=\mathbb{E}_D\log\mathbb{E}_{p_0}\exp G(u). \tag*{(21)} K ( u ) = E D KL ( p 0 ∥ p u ) = E D log E p 0 exp G ( u ) . (21)
Consequently K ( 0 ) = 0 K(0)=0 K ( 0 ) = 0 , D K ( 0 ) = 0 DK(0)=0 D K ( 0 ) = 0 , and
D 2 K ( 0 ) [ v , w ] = ⟨ D G ( 0 ) v , D G ( 0 ) w ⟩ 0 . (22) D^2K(0)[v,w]=\langle DG(0)v,DG(0)w\rangle_0. \tag*{(22)} D 2 K ( 0 ) [ v , w ] = ⟨ D G ( 0 ) v , D G ( 0 ) w ⟩ 0 . (22)
The full restricted Fisher matrix is therefore the Gram matrix of the response tangents. This identity includes its off-diagonal entries.
For a fixed pair of component gains ( a , b ) (a,b) ( a , b ) , write J a = ∂ a G ( 0 ) J_a=\partial_aG(0) J a = ∂ a G ( 0 ) , J b = ∂ b G ( 0 ) J_b=\partial_bG(0) J b = ∂ b G ( 0 ) , and H a b = ∂ a ∂ b G ( 0 ) H_{ab}=\partial_a\partial_bG(0) H ab = ∂ a ∂ b G ( 0 ) . Let P T P_T P T be the orthogonal projection onto T = span { J a , J b } T=\operatorname{span}\{J_a,J_b\} T = span { J a , J b } . The interaction curvature is
N a b = ∥ ( I − P T ) H a b ∥ 0 2 . (23) N_{ab}=\left\lVert(I-P_T)H_{ab}\right\rVert_0^2. \tag*{(23)} N ab = ∥ ( I − P T ) H ab ∥ 0 2 . (23)
It measures the part of the joint response that cannot be expressed as a linear combination of the individual responses. Its numerical evaluation uses the 2 × 2 2\times2 2 × 2 Fisher Gram matrix and the inner products of H a b H_{ab} H ab with the two tangents.
Proposition 2 (Interactions beyond the Fisher matrix). For a model with logits z l i n ( a , b ) = z 0 + a ∂ a z ( 0 ) + b ∂ b z ( 0 ) z_{\mathrm{lin}}(a,b)=z_0+a\partial_a z(0)+b\partial_b z(0) z lin ( a , b ) = z 0 + a ∂ a z ( 0 ) + b ∂ b z ( 0 ) , the centered response is exactly a J a + b J b aJ_a+bJ_b a J a + b J b , and N a b = 0 N_{ab}=0 N ab = 0 . Models can have identical reference predictions and identical Fisher matrices but different values of N a b N_{ab} N ab .
Proof. Centering eliminates the softmax normalizer, so a linear logit map has zero mixed derivative. Conversely, add a b h ( x , y ) ab\,h(x,y) ab h ( x , y ) to z l i n z_{\mathrm{lin}} z lin , where h h h is centered and orthogonal to J a , J b J_a,J_b J a , J b . This preserves the reference predictions and both first derivatives, hence the entire Fisher Gram matrix. The normal curvature, the interaction curvature N a b N_{ab} N ab of Section 4, becomes ∥ h ∥ 0 2 \lVert h\rVert_0^2 ∥ h ∥ 0 2 , which can take any nonnegative value. □ \square □
This construction separates nonlinear computation from the nonlinearity of the categorical readout. The linearized-logit control retains the exact softmax and the complete response tangents. Its predictions can exhibit nonlinear KL effects even though its centered mixed response vanishes.
Functional gain coordinates. Output gains specify interventions in units of a component’s own write. An internal reparameterization that preserves each component’s input–output function and commutes with scalar output multiplication preserves the entire gain-response map. Induction along the computation graph then gives the same G ( a , b ) G(a,b) G ( a , b ) , hence the same F F F , N a b N_{ab} N ab , and mixed-volume coefficient (the mixed quartic coefficient B a b B_{ab} B ab of Section 4). This includes inverse changes of basis in a head’s value and output maps. The gain coordinates thus fix the comparison at the level of the component operation.
An exactly solvable circuit interaction. Consider eight output symbols indexed by ( x , y , z ) ∈ { − 1 , 1 } 3 (x,y,z) \in\{-1,1\}^{3} ( x , y , z ) ∈ { − 1 , 1 } 3 , with logits ℓ x y z ( a , b ) = a x + b y + κ a b z \ell_{xyz}(a,b)=ax+by+\kappa abz ℓ x y z ( a , b ) = a x + b y + κab z and uniform reference probabilities at ( a , b ) = ( 0 , 0 ) (a,b)=(0,0) ( a , b ) = ( 0 , 0 ) . The first two terms transmit individual component responses; the third represents their multiplicative interaction. Direct normalization gives
K κ ( a , b ) = log cosh a + log cosh b + log cosh ( κ a b ) . (24) K_\kappa(a,b)=\log\cosh a+\log\cosh b+\log\cosh(\kappa ab). \tag*{(24)} K κ ( a , b ) = log cosh a + log cosh b + log cosh ( κab ) . (24)
Every member of this family has the same restricted Fisher matrix F = I 2 F=I_2 F = I 2 . Its mixed response κ z \kappa z κ z is orthogonal to the individual responses x , y x,y x , y , so N a b = κ 2 N_{ab}=\kappa^2 N ab = κ 2 . The linearized-logit loss is K l i n = log cosh a + log cosh b K_{\mathrm{lin}}=\log\cosh a+\log\cosh b K lin = log cosh a + log cosh b . Consequently B a b = κ 2 / 2 B_{ab}=\kappa^2/2 B ab = κ 2 /2 , while the absolute nonlinear joint zero-ablation interaction is
∣ f ( − 1 , − 1 ) − f ( − 1 , 0 ) − f ( 0 , − 1 ) ∣ = log cosh κ = log cosh N a b , f = K κ − K l i n . (25) \left|f(-1,-1)-f(-1,0)-f(0,-1)\right|=\log\cosh\kappa=\log\cosh\sqrt{N_{ab}},\qquad f=K_\kappa-K_{\mathrm{lin}}. \tag*{(25)} ∣ f ( − 1 , − 1 ) − f ( − 1 , 0 ) − f ( 0 , − 1 ) ∣ = log cosh κ = log cosh N ab , f = K κ − K lin . (25)
Thus local curvature, the mixed loss-volume correction, and the finite causal interaction vary together despite an identical full Fisher matrix. All losses are regular at the origin, with learning coefficient one; the varying geometric information resides in their finite-volume corrections. For every finite κ \kappa κ , the gain radius can be chosen small enough that the entire measurement square has arbitrarily small predictive KL.
Finite corrections to local loss volume
For an untruncated Gaussian prior of covariance r 2 I r^2 I r 2 I , the full quadratic response has the closed form
q Q ( τ , r ) = 1 2 ∑ i τ r 2 μ i 1 + τ r 2 μ i , (26) q_Q(\tau,r)=\frac{1}{2}\sum_i\frac{\tau r^2\mu_i}{1+\tau r^2\mu_i}, \tag*{(26)} q Q ( τ , r ) = 2 1 i ∑ 1 + τ r 2 μ i τ r 2 μ i , (26)
where μ i \mu_i μ i are the restricted Fisher eigenvalues. Comparisons with an additional neighborhood U U U apply the same restriction to the actual and quadratic targets.
Let U U U be a fixed neighborhood containing an open ball around zero. Assume that K K K is analytic on a neighborhood of its closure and that the local integrals are finite. With a Gaussian prior of covariance r 2 I r^2 I r 2 I , define
Z K ( τ , r ) = E u ∼ N ( 0 , r 2 I ) [ 1 U ( u ) e − τ K ( u ) ] , F K = − log Z K , q K = τ ∂ τ F K . (27) Z_K(\tau,r)=\mathbb{E}_{u\sim\mathcal{N}(0,r^2I)}\left[1_U(u)e^{-\tau K(u)}\right],\qquad F_K=-\log Z_K,\qquad q_K=\tau\partial_\tau F_K. \tag*{(27)} Z K ( τ , r ) = E u ∼ N ( 0 , r 2 I ) [ 1 U ( u ) e − τ K ( u ) ] , F K = − log Z K , q K = τ ∂ τ F K . (27)
Normalizing the prior inside U U U changes F K F_K F K by a term independent of τ \tau τ , and leaves q K q_K q K unchanged. The energy fluctuation is
C K = τ 2 Var π K [ K ] , ∂ log τ q K = q K − C K . (28) C_K=\tau^2\operatorname{Var}_{\pi_K}[K],\qquad\partial_{\log\tau}q_K=q_K-C_K. \tag*{(28)} C K = τ 2 Var π K [ K ] , ∂ l o g τ q K = q K − C K . (28)
For a local volume law with exponent λ \lambda λ and multiplicity m m m , the corresponding large-τ \tau τ expansion gives q K = λ − ( m − 1 ) / log τ + o ( 1 / log τ ) q_K=\lambda-(m-1)/\log\tau+o(1/\log\tau) q K = λ − ( m − 1 ) / log τ + o ( 1/ log τ ) . Finite responses additionally contain information about curvature and higher response derivatives.
Write
K ( r z ) = r 2 K 2 ( z ) + r 3 K 3 ( z ) + r 4 K 4 ( z ) + O ( r 5 ) , K 2 ( z ) = 1 2 z T F z . (29) K(rz)=r^2K_2(z)+r^3K_3(z)+r^4K_4(z)+O(r^5),\qquad K_2(z)=\frac{1}{2}z^{\mathsf T}Fz. \tag*{(29)} K ( r z ) = r 2 K 2 ( z ) + r 3 K 3 ( z ) + r 4 K 4 ( z ) + O ( r 5 ) , K 2 ( z ) = 2 1 z T F z . (29)
For a standard Gaussian z z z , set A = 1 2 tr F A=\frac{1}{2}\operatorname{tr}F A = 2 1 tr F and B = E K 4 ( z ) = 1 8 ∑ i , j ∂ i 2 ∂ j 2 K ( 0 ) B=\mathbb{E}K_4(z)=\frac{1}{8}\sum_{i,j}\partial_i^2\partial_j^2K(0) B = E K 4 ( z ) = 8 1 ∑ i , j ∂ i 2 ∂ j 2 K ( 0 ) .
Proposition 3 (The first nonquadratic volume correction). At fixed τ \tau τ , as r → 0 r\to0 r → 0 ,
log Z K = − τ A r 2 + r 4 [ − τ B + τ 2 4 tr ( F 2 ) ] + O ( r 6 ) , (30) \log Z_K=-\tau Ar^2+r^4\left[-\tau B+\frac{\tau^2}{4}\operatorname{tr}(F^2)\right]+O(r^6), \tag*{(30)} log Z K = − τ A r 2 + r 4 [ − τ B + 4 τ 2 tr ( F 2 ) ] + O ( r 6 ) , (30) q K − q Q = τ B r 4 + O ( r 6 ) , Q ( u ) = 1 2 u T F u . (31) q_K-q_Q=\tau Br^4+O(r^6),\qquad Q(u)=\frac{1}{2}u^{\mathsf T}Fu. \tag*{(31)} q K − q Q = τ B r 4 + O ( r 6 ) , Q ( u ) = 2 1 u T F u . (31) The two partition functions use the same neighborhood and prior:
Proof. The Gaussian expectation of every odd homogeneous polynomial vanishes. Thus E K ( r z ) = A r 2 + B r 4 + O ( r 6 ) \mathbb{E}K(rz)=Ar^{2}+Br^{4}+O(r^{6}) E K ( r z ) = A r 2 + B r 4 + O ( r 6 ) and Var K ( r z ) = 1 2 r 4 tr ( F 2 ) + O ( r 6 ) \operatorname{Var}K(rz)=\frac{1}{2}r^{4}\operatorname{tr}(F^{2})+O(r^{6}) Var K ( r z ) = 2 1 r 4 tr ( F 2 ) + O ( r 6 ) . The cumulant expansion of log E e − τ K \log\mathbb{E}e^{-\tau K} log E e − τ K gives the first formula. Differentiating with respect to τ \tau τ and subtracting the quadratic expression gives the second. Because U U U contains a fixed ball, its boundary contributes an exponentially small correction in this small-r r r expansion.
An antithetic radius contrast estimates B B B without subtracting an estimated Fisher trace:
B r = E K ( r z ) − 4 E K ( r z / 2 ) ( 3 / 4 ) r 4 = B + O ( r 2 ) . (32) B_{r}=\frac{\mathbb{E}K(rz)-4\mathbb{E}K(rz/2)}{(3/4)r^{4}}=B+O(r^{2}). \tag*{(32)} B r = ( 3/4 ) r 4 E K ( r z ) − 4 E K ( r z /2 ) = B + O ( r 2 ) . (32)
Using identical directions at both radii cancels every quadratic form sample by sample. For example, K c ( u ) = u 2 / 2 + c u 4 K_{c}(u)=u^{2}/2+cu^{4} K c ( u ) = u 2 /2 + c u 4 has the same Fisher at the origin for every c c c , while its leading finite-response correction is 3 c τ r 4 3c\tau r^{4} 3 c τ r 4 .
The comparison at fixed s = τ r 2 s=\tau r^{2} s = τ r 2 has a different expansion. Let μ s \mu_{s} μ s be the Gaussian measure with density proportional to e − s K 2 ( z ) e^{-sK_{2}(z)} e − s K 2 ( z ) relative to N ( 0 , I ) N(0,I) N ( 0 , I ) . Then
F K − F Q = r 2 [ s E μ s K 4 − s 2 2 E μ s K 3 2 ] + O ( r 4 ) , (33) \mathcal{F}_{K}-\mathcal{F}_{Q}=r^{2}\left[s\mathbb{E}_{\mu_{s}}K_{4}-\frac{s^{2}}{2}\mathbb{E}_{\mu_{s}}K_{3}^{2}\right]+O(r^{4}), \tag*{(33)} F K − F Q = r 2 [ s E μ s K 4 − 2 s 2 E μ s K 3 2 ] + O ( r 4 ) , (33)
and
q K − q Q = r 2 s ∂ s [ s E μ s K 4 − s 2 2 E μ s K 3 2 ] + O ( r 4 ) . (34) q_{K}-q_{Q}=r^{2}s\partial_{s}\left[s\mathbb{E}_{\mu_{s}}K_{4}-\frac{s^{2}}{2}\mathbb{E}_{\mu_{s}}K_{3}^{2}\right]+O(r^{4}). \tag*{(34)} q K − q Q = r 2 s ∂ s [ s E μ s K 4 − 2 s 2 E μ s K 3 2 ] + O ( r 4 ) . (34)
Both cubic and quartic response terms contribute in this regime. Halving r r r while multiplying τ \tau τ by four therefore leaves the Gaussian reference unchanged and reduces the leading nonquadratic correction by a factor of four, provided that boundary effects are negligible.
Circuit coupling in the partition function
For a two-component loss L ( a , b ) L(a,b) L ( a , b ) , define the symmetric rectangular contrast
D r L = 1 4 ∑ s , t ∈ { − 1 , 1 } L ( s r , t r ) − 1 2 ∑ s L ( s r , 0 ) − 1 2 ∑ t L ( 0 , t r ) + L ( 0 , 0 ) . (35) D_{r}L=\frac{1}{4}\sum_{s,t\in\{-1,1\}}L(sr,tr)-\frac{1}{2}\sum_{s}L(sr,0)-\frac{1}{2}\sum_{t}L(0,tr)+L(0,0). \tag*{(35)} D r L = 4 1 s , t ∈ { − 1 , 1 } ∑ L ( sr , t r ) − 2 1 s ∑ L ( sr , 0 ) − 2 1 t ∑ L ( 0 , t r ) + L ( 0 , 0 ) . (35)
Every polynomial of total degree at most three, including the off-diagonal Fisher term, is canceled. Taylor expansion gives
D r K − D r K l i n r 4 = 1 4 ∂ a 2 ∂ b 2 ( K − K l i n ) ( 0 ) + O ( r 2 ) . (36) \frac{D_{r}K-D_{r}K_{\mathrm{lin}}}{r^{4}}=\frac{1}{4}\partial_{a}^{2}\partial_{b}^{2}(K-K_{\mathrm{lin}})(0)+O(r^{2}). \tag*{(36)} r 4 D r K − D r K lin = 4 1 ∂ a 2 ∂ b 2 ( K − K lin ) ( 0 ) + O ( r 2 ) . (36)
Here K l i n K_{\mathrm{lin}} K lin is the KL loss of the exact-softmax linearized-logit control. Denote the limiting coefficient by B a b B_{ab} B ab .
Let F a b \mathcal{F}_{ab} F ab be the free energy with both gains varied, and let F a , F b \mathcal{F}_{a},\mathcal{F}_{b} F a , F b be the corresponding one-gain free energies, using matched Gaussian scales. Define their nonadditive part J K = F a b − F a − F b \mathcal{J}_{K}=\mathcal{F}_{ab}-\mathcal{F}_{a}-\mathcal{F}_{b} J K = F ab − F a − F b . Since K K K and K l i n K_{\mathrm{lin}} K lin have the same Hessian, Proposition 3 gives
J K − J K l i n = τ r 4 B a b + O ( r 6 ) (37) \mathcal{J}_{K}-\mathcal{J}_{K_{\mathrm{lin}}}=\tau r^{4}B_{ab}+O(r^{6}) \tag*{(37)} J K − J K lin = τ r 4 B ab + O ( r 6 ) (37)
at fixed τ \tau τ . Thus the rectangular contrast measures a mixed local-volume correction beyond both the Fisher approximation and the nonlinear categorical readout of a linear response.
The normal curvature in Equation (23) resolves an identifiable part of this higher-order geometry. Expanding Equation (21) , the term a b H a b abH_{ab} ab H ab contributes 1 2 a 2 b 2 ∥ H a b ∥ 2 2 \frac{1}{2}a^{2}b^{2}\lVert H_{ab}\rVert_{2}^{2} 2 1 a 2 b 2 ∥ H ab ∥ 2 2 to the quadratic cumulant. Higher response derivatives and categorical third cumulants contribute additional terms to the complete mixed-volume coefficient. Measuring both quantities separates a positive nonlinear-response contribution from their combined effect on loss volume.
Correspondence with causal interaction tests
A parameter change to a component can be transported exactly into the clean network by patching its complete altered activation tensor. For several components, use their activations from the joint perturbed forward pass; induction along the computation graph then reproduces the same downstream output. Parameter geometry and matched activation interventions consequently evaluate the same response map.
For a scalar circuit readout S ( a , b ) S(a,b) S ( a , b ) , the finite joint response to zero-gain interventions satisfies the exact identity
S ( − 1 , − 1 ) − S ( − 1 , 0 ) − S ( 0 , − 1 ) + S ( 0 , 0 ) = ∫ − 1 0 ∫ − 1 0 ∂ a ∂ b S ( a , b ) d a d b . (38) S(-1,-1)-S(-1,0)-S(0,-1)+S(0,0)=\int_{-1}^{0}\int_{-1}^{0}\partial_a\partial_b S(a,b)\,\mathrm{d}a\,\mathrm{d}b. \tag*{(38)} S ( − 1 , − 1 ) − S ( − 1 , 0 ) − S ( 0 , − 1 ) + S ( 0 , 0 ) = ∫ − 1 0 ∫ − 1 0 ∂ a ∂ b S ( a , b ) d a d b . (38)
For a pairwise logit contrast, ∂ a ∂ b S \partial_a\partial_b S ∂ a ∂ b S is a linear readout of the mixed centered response H a b H_{ab} H ab . If this derivative differs from its value at zero by at most ϵ \epsilon ϵ throughout the integration square, the finite causal interaction differs from the local mixed derivative by at most ϵ \epsilon ϵ . This supplies a sufficient condition for agreement. Changes in the direction or sign of the interaction away from the reference can instead produce cancellation in the finite intervention. A norm of the local response also aggregates output directions that a particular scalar contrast may weight differently.
The KL interaction used in the experiments is governed by a corresponding higher-order condition. Write H ⊥ = ( I − P T ) H a b H_{\perp}=(I-P_T)H_{ab} H ⊥ = ( I − P T ) H ab and decompose
f ( a , b ) = K ( a , b ) − K l i n ( a , b ) = 1 2 a 2 b 2 ∥ H ⊥ ∥ 0 2 + E ( a , b ) . (39) f(a,b)=K(a,b)-K_{\mathrm{lin}}(a,b)=\frac{1}{2}a^2b^2\lVert H_{\perp}\rVert_0^2+E(a,b). \tag*{(39)} f ( a , b ) = K ( a , b ) − K lin ( a , b ) = 2 1 a 2 b 2 ∥ H ⊥ ∥ 0 2 + E ( a , b ) . (39)
Here E E E collects tangent mixed responses, the remaining higher response derivatives, and categorical cumulants beyond the quadratic term. The displayed contribution follows by expanding 1 2 ∥ a J a + b J b + a b H ⊥ ∥ 0 2 \frac{1}{2}\lVert aJ_a+bJ_b+abH_{\perp}\rVert_0^2 2 1 ∥ a J a + b J b + ab H ⊥ ∥ 0 2 and using orthogonality. If ∣ E ∣ ≤ ϵ |E|\leq\epsilon ∣ E ∣ ≤ ϵ at the three nonzero intervention corners, then
∣ f ( − 1 , − 1 ) − f ( − 1 , 0 ) − f ( 0 , − 1 ) ∣ − 1 2 N a b ≤ 3 ϵ . (40) \left|f(-1,-1)-f(-1,0)-f(0,-1)\right|-\frac{1}{2}N_{ab}\leq3\epsilon. \tag*{(40)} ∣ f ( − 1 , − 1 ) − f ( − 1 , 0 ) − f ( 0 , − 1 ) ∣ − 2 1 N ab ≤ 3 ϵ . (40)
This gives a sufficient condition for the specific geometric and causal quantities to agree. Interactions can depart from this relation when the remainder dominates or changes sign across training. Equation (24) gives an exact monotone correspondence even when the quadratic-cumulant approximation to the finite intervention is inaccurate.
Input and intervention choices specify the response being tested. If two input distributions have density ratio bounded above and below by positive constants, their KL losses are locally equivalent and have the same learning coefficient. Finite responses can still differ through their weighting of the same constraints. A change of support can add or remove constraints. Likewise, position-mean replacement preserves a mean component write, whereas a parameter intervention can change that write. These distinctions explain why agreement is tested with matched inputs, output discrepancies, and component interventions.
Developmental interaction measurements
The interaction experiment contains 43 circuit-based pairs and 43 control pairs. The circuit set contains every unordered pair of the eight first-layer heads, together with
( L 0 H 0 , L 7 H 4 ) , ( L 0 H 2 , L 7 H 4 ) , (\mathrm{L0H0,L7H4}),\;(\mathrm{L0H2,L7H4}), ( L0H0 , L7H4 ) , ( L0H2 , L7H4 ) ,
( L 0. m l p , L 2. m l p ) , ( L 1. m l p , L 2. m l p ) , (\mathrm{L0.mlp,L2.mlp}),\;(\mathrm{L1.mlp,L2.mlp}), ( L0.mlp , L2.mlp ) , ( L1.mlp , L2.mlp ) ,
( L 2. m l p , L 3. m l p ) , ( L 3. m l p , L 4. m l p ) , ( L 4. m l p , L 5. m l p ) , (\mathrm{L2.mlp,L3.mlp}),\;(\mathrm{L3.mlp,L4.mlp}),\;(\mathrm{L4.mlp,L5.mlp}), ( L2.mlp , L3.mlp ) , ( L3.mlp , L4.mlp ) , ( L4.mlp , L5.mlp ) ,
( L 2. m l p , L 7 H 4 ) , ( L 3. m l p , L 7 H 4 ) , (\mathrm{L2.mlp,L7H4}),\;(\mathrm{L3.mlp,L7H4}), ( L2.mlp , L7H4 ) , ( L3.mlp , L7H4 ) ,
( L 4. m l p , L 7 H 4 ) , ( L 5. m l p , L 7 H 4 ) , (\mathrm{L4.mlp,L7H4}),\;(\mathrm{L5.mlp,L7H4}), ( L4.mlp , L7H4 ) , ( L5.mlp , L7H4 ) ,
( L 7 H 4 , L 11. m l p ) , ( L 7 H 5 , L 11. m l p ) , (\mathrm{L7H4,L11.mlp}),\;(\mathrm{L7H5,L11.mlp}), ( L7H4 , L11.mlp ) , ( L7H5 , L11.mlp ) ,
( L 11. m l p , L 14 H 1 ) , ( L 7 H 4 , L 14 H 1 ) . (\mathrm{L11.mlp,L14H1}),\;(\mathrm{L7H4,L14H1}). ( L11.mlp , L14H1 ) , ( L7H4 , L14H1 ) .
Control pairs match component types and layer separation, using seed 20260923 and excluding duplicate pairs. Selection follows the mechanistic account; the confirmation outcomes are evaluated for every selected pair.
All 154 released checkpoints are measured. The calibration checkpoints are 0, 512, 2,000, 8,000, 32,000, and 143,000. The other 148 checkpoints define the confirmation set. The primary task is k = 4 k = 4 k = 4 , depth bound 16, sequence length 128, and opening probability 0.45. The original seed-0 batch contains 128 strings and 5,475 closer targets. Independent seeds 20261001, 20261002, and 20261003 each generate 128 strings, with 5,553, 5,482, and 5,421 closer targets. Geometry and interventions use identical examples and masks within each batch. The complete replicated grid contains 4 × 154 × 86 = 52,976 4 \times154 \times86 = 52{,}976 4 × 154 × 86 = 52 , 976 pair–checkpoint–batch measurements.
Parameterization and numerical measurements
For head h h h , gain a a a replaces its output-projection block by ( 1 + a ) W O h (1+a)W^{h}_{O} ( 1 + a ) W O h , preserving the shared attention output bias. For an MLP, the gain multiplies both W o u t W_{\mathrm{out}} W out and b o u t b_{\mathrm{out}} b out . Gain − 1 -1 − 1 therefore implements the corresponding zero-output intervention. Joint gains are applied in the same forward pass, including their downstream effect on other components. The gains are specified in component-write coordinates at each checkpoint.
We load the fixed converted float32 checkpoint values, then promote evaluation arithmetic to float64. The reference KL uses the full vocabulary at closer positions. Separate functional checks use all valid next-token positions. Centered response tangents are computed by symmetric differences at gains 0.0025 and 0.00125, with Richardson extrapolation. The resulting 2 × 2 2 \times2 2 × 2 Gram matrix supplies the full restricted Fisher control. The normal projection uses a relative pseudoinverse cutoff 10 − 8 10^{-8} 1 0 − 8 ; the smallest observed eigenvalue ratio is 0.00329, so neither tangent direction is discarded in any cell.
At each radius r ∈ { 0.025 , 0.05 } r \in\{0.025, 0.05\} r ∈ { 0.025 , 0.05 } , we evaluate the four signed corners ( ± r , ± r ) (\pm r, \pm r) ( ± r , ± r ) and four axis points ( ± r , 0 ) , ( 0 , ± r ) (\pm r, 0), (0, \pm r) ( ± r , 0 ) , ( 0 , ± r ) . Centering the mixed log-probability difference removes the softmax normalizer exactly. Its normal squared norm estimates N a b N_{ab} N ab with second-order radius error. For f = K − K l i n f = K - K_{\mathrm{lin}} f = K − K lin , the mixed-volume estimate is
B a b , r = 1 4 ∑ s , t = ± 1 f ( s r , t r ) − 1 2 ∑ s = ± 1 f ( s r , 0 ) − 1 2 ∑ t = ± 1 f ( 0 , t r ) r 4 . (41) B_{ab,r} = \frac{\frac{1}{4}\sum_{s,t=\pm1} f(sr,tr) - \frac{1}{2}\sum_{s=\pm1} f(sr,0) - \frac{1}{2}\sum_{t=\pm1} f(0,tr)}{r^{4}}. \tag*{(41)} B ab , r = r 4 4 1 ∑ s , t = ± 1 f ( sr , t r ) − 2 1 ∑ s = ± 1 f ( sr , 0 ) − 2 1 ∑ t = ± 1 f ( 0 , t r ) . (41)
Here f ( 0 , 0 ) = 0 f(0,0) = 0 f ( 0 , 0 ) = 0 . The linearized-logit control retains the full individual response tangents and exact categorical normalization. The primary normal-curvature statistic and secondary volume contrast are specified before confirmation analysis.
Every local state is screened for mean KL at most 0.02 nats, absolute cross-entropy change at most 0.1 nats, and absolute full-vocabulary accuracy change at most 0.02. Both closer positions and all valid positions must pass. All 52,976 pair–checkpoint–batch cells pass at the primary radius. Maximum all-position KL across the four batches is 0.01427, 0.01415, 0.01523, and 0.01389 nats. Halving the radius gives normal-curvature rank correlations 0.9987, 0.99986, 0.99951, and 0.99975. For the absolute mixed-volume coefficient the corresponding correlations are 0.99266, 0.99088, 0.99422, and 0.99212.
Repeated-baseline checks require a maximum log-probability error below 10 − 10 10^{-10} 1 0 − 10 ; two later-layer cells that exceeded it with cached prefixes were evaluated from the frozen embedding output instead, with the same parameters, inputs, gains, and arithmetic.
For all 43 prespecified circuit pairs, the dense-path check evaluates 16 equally spaced nonzero magnitudes along each of the eight signed local directions, from the reference to radius 0.05. All 43 × 154 × 4 × 128 = 3,390,464 43 \times154 \times4 \times128 = 3{,}390{,}464 43 × 154 × 4 × 128 = 3 , 390 , 464 states are present and lie inside U U U at closer and at all positions. Maximum all-position KL, loss change, and accuracy change are 0.01353 nats, 0.0642 nats, and 0.01819; the corresponding closer-position maxima are 0.01013, 0.04471, and 0.01587. Midpoints and endpoints reproduce the stencil measurements to within 10 − 9 10^{-9} 1 0 − 9 in KL and cross-entropy. Checkpoint, pair, direction, and gain identifiers verify full coverage of this fixed circuit set. These evaluations provide a finite-grid check of the sampled paths (Figure 5 ).
Figure 5. Locality and radius stability of the interaction measurement. A: empirical distributions (the four batches coincide at this scale) of the largest all-position KL across each pair’s eight local configurations, including all 154 checkpoints. The dashed line marks the fixed 0.02-nat threshold. B: normal-curvature estimates at the two radii over the complete four-batch grid.
Temporal comparisons and dependence
For each pair, we compute a temporal Spearman correlation across the 148 confirmation checkpoints. Partial-rank correlations regress both ranked measurements on an intercept, the smaller and larger Fisher diagonal entries, the absolute cross entry, clean closer loss, and training step. An additional control retains the signed cross entry; all adjusted primary results reported in the main text include it. The reported population summary is the median over pairs, with circuit and comparison strata retained separately. These controls quantify incremental rank association under the specified regression.
Adjacent-change comparisons difference the ordered checkpoint series before ranking. An interval is retained only when both neighboring released checkpoints belong to the confirmation set. Resampling never creates a difference across artificial block boundaries. The controls are differenced in the same way. Checkpoint IDs, rather than file order, align all measurements.
We draw 1,000 shared temporal-block replicates at each block length 8, 16, and 32. Every pair receives the same sampled checkpoint blocks, preserving dependence through shared inputs and components. The 95% percentile intervals in Figure 9 use length 16; the complete results retain all three lengths. A separate deletion analysis removes every pair containing one component, then recomputes the median. These intervals describe stability within the observed training trajectory. The independent input batches replicate the measurement on fresh data from the same task distribution (Table 2 ).
Table 2. Complete temporal summaries at r = 0.05 r = 0.05 r = 0.05 . Each entry is the median within-pair association with the matched nonlinear causal interaction. Adjusted values include the signed cross-Fisher control, clean loss, and time. All pairs have complete trajectories over the confirmation checkpoints.
Figure 9. Pairwise geometry tracks circuit interactions beyond Fisher geometry. Medians over all 43 circuit pairs. A: temporal associations between local normal curvature and the causal interaction. B: associations between adjacent checkpoint changes. C: associations for the absolute mixed local-volume correction. Adjusted estimates control the restricted Fisher terms, clean closer loss, and training step (their differences in B). Bars are 95% shared temporal-block intervals with block length 16. Three independent input batches replicate the original-batch result.
The smallest lower endpoint among all three block lengths and all four batches is 0.030 for the circuit-pair primary adjusted association, 0.009 for adjusted adjacent changes, and 0.015 for the adjusted absolute mixed-volume coefficient. At half radius, the independent-batch median adjusted volume associations are 0.181, 0.154, and 0.180.
A float32 comparison across tasks measures the same 86 pairs on both k = 2 k = 2 k = 2 and k = 4 k = 4 k = 4 ; on k = 2 k = 2 k = 2 , 83 pairs lie inside U U U at closer positions at every confirmation checkpoint. The four-batch float64 experiment is the primary independent-input confirmation and adds the all-position locality checks.
The prespecified signed closer-contrast comparison evaluates the local mixed derivative of the correct-versus-incorrect closer contrast against its joint zero-ablation interaction. Median unadjusted temporal correlations across circuit pairs are 0.453 on the original batch and 0.409, 0.360, and 0.427 on the independent batches. These values retain the signed task readout; the primary KL comparison aggregates the predictive response (Figure 6 ).
Figure 6. Circuit-role examples across all 154 checkpoints. A: first-layer head bank. B: an early MLP pathway. C: retrieval and readout heads. D: an MLP/readout-head pair. Curves show within-trajectory ranks, averaged over four input batches; shading spans the batch values. The statistics use unsmoothed individual-batch measurements. The examples cover distinct circuit roles; Figure 7 and Figure 8 show every measured pair.
Figure 7. Complete circuit-pair trajectories. Each measurement is ranked within its 154-checkpoint trajectory, then averaged across four input batches. Rows include every one of the 43 circuit pairs. Columns are equally spaced released checkpoints, with training-step labels.
Figure 8. Complete control-pair trajectories. The same display for all 43 pairs matched by component type and layer separation.
Robustness of the temporal association. For the 43 circuit pairs, the Fisher-adjusted partial temporal association between N a b N_{ab} N ab and Y a b Y_{ab} Y ab has shared temporal-block bootstrap intervals that remain positive at block lengths 8, 16, and 32 on all three independent batches, and removing every pair incident to any one component leaves a positive median association (lowest value 0.10).
Locality of the dense paths. All 3,390,464 dense path states of the 43 circuit pairs lie inside the region U U U at every checkpoint on every batch (maximum all-position KL 0.0136 nats), as does every primary-radius stencil; halving the gain radius changes the normal curvature by a median 0.25–0.27% across the three batches.
Local geometry across training: protocol
This appendix specifies the local-geometry measurement reported in § 5.2 and summarized in Table 1 . Every quantity is computed from the KL loss (1) of one component at one checkpoint; the estimator, its Fisher control, and the region U U U were fixed before any developmental comparison.
Components, checkpoints, and inputs. The radius–temperature grid covers a panel of 33 components: the 26 circuit components and seven control components sampled uniformly from the others (seed 20260926), namely the circuit components L0.mlp, L0H0–H7, L1–L5.mlp, L1H1, L2H1, L2H5, L7H2, L7H4, L7H5, L9H4, L14H1, L11.mlp, and L13–L15.mlp and the controls L2H3, L6.mlp, L11H0, L12.mlp, L13H7, L14H0, and L14H7; the spectrum measurement of Appendix § 12 covers all 144 components. All 154 released checkpoints are measured. The inputs are the fixed k = 4 k = 4 k = 4 developmental batch: 128 strings of length 128 with 11,334 valid next-token positions, 5,475 of which are closer positions. The reference distribution at every scored position is the checkpoint’s own full-vocabulary next-token distribution.
Parameters and coordinates. A head restriction varies its slices of W Q W_Q W Q , W K W_K W K , W V W_V W V , W O W_O W O , b Q b_Q b Q , b K b_K b K , b V b_V b V (d c = 2,097,920 d_c = 2{,}097{,}920 d c = 2 , 097 , 920 ); an MLP restriction varies W i n W_{\mathrm{in}} W in , b i n b_{\mathrm{in}} b in , W o u t W_{\mathrm{out}} W out , b o u t b_{\mathrm{out}} b out (d c = 33,564,672 d_c = 33{,}564{,}672 d c = 33 , 564 , 672 ). Coordinates rescale each selected tensor by a fixed scale (Equation (49) ). The primary coordinates use the final-checkpoint RMS scales at every checkpoint, so that a coordinate displacement means the same change of weights throughout training. A second set of coordinates rescales by the current checkpoint’s RMS values; it is measured for the seven components of the temperature sweep (Appendix § 15 ) as a coordinate control.
Directions and radii. For each component we draw 128 independent standard Gaussian directions z i z_i z i in these coordinates, shared across all checkpoints and both coordinate choices through a component-indexed seed. Each direction is evaluated with both signs at the twelve amplitudes { r , r / 2 , r / 4 } \{r,\ r/2,\ r/4\} { r , r /2 , r /4 } generated by the eight radii r ∈ { 0.00625 , 0.0125 , 0.025 , 0.05 , 0.1 , 0.15 , 0.2 , 0.4 } r \in\{0.00625, 0.0125, 0.025, 0.05, 0.1, 0.15, 0.2, 0.4\} r ∈ { 0.00625 , 0.0125 , 0.025 , 0.05 , 0.1 , 0.15 , 0.2 , 0.4 } ; identical amplitudes are evaluated once. The endpoint closer KL K ( ± r z i ) K(\pm r z_i) K ( ± r z i ) , its all-position counterpart, cross-entropy, and full-vocabulary accuracy are recorded at every amplitude. The full restricted Fisher quadratic Q ( r z i ) = 1 2 r 2 z i T F c z i Q(rz_i) = \frac{1}{2}r^2 z_i^{\mathsf{T}} F_c z_i Q ( r z i ) = 2 1 r 2 z i T F c z i is evaluated once per direction by an exact Hessian-vector product of the closer reference KL, computed by automatic differentiation through the same forward pass; it is rescaled exactly to every radius. Model arithmetic is float64 after loading the identical float32 checkpoint values. Independent CPU tests verify the Hessian-vector product against fullforward directional derivatives, the exact r 2 r^2 r 2 scaling of Q Q Q , the small-radius agreement of K K K and Q Q Q , and parameter-zero versus activation-zero parity for heads and MLPs.
The region U U U . A sample u = ± r z i \mathbf{u} = \pm r\mathbf{z}_i u = ± r z i lies inside U U U if, at u \mathbf{u} u , u / 2 \mathbf{u}/2 u /2 , and u / 4 \mathbf{u}/4 u /4 , both at closer positions and at all valid positions, K L ≤ 0.02 \mathrm{KL} \le0.02 KL ≤ 0.02 nats, ∣ Δ C E ∣ ≤ 0.1 |\Delta\mathrm{CE}| \le0.1 ∣Δ CE ∣ ≤ 0.1 nats, and ∣ Δ a c c u r a c y ∣ ≤ 0.02 |\Delta\mathrm{accuracy}| \le0.02 ∣Δ accuracy ∣ ≤ 0.02 , and every tensor’s unit-RMS displacement is at most 0.5 0.5 0.5 . A stricter 0.2 0.2 0.2 -displacement mask is retained. The same indicator enters the actual and the quadratic targets, so both LLC estimates are defined on the same region.
Radial statistics. Writing K ˉ i ( r ) = 1 2 [ K ( r z i ) + K ( − r z i ) ] \bar{K}_i(r) = \frac{1}{2}[K(r\mathbf{z}_i) + K(-r\mathbf{z}_i)] K ˉ i ( r ) = 2 1 [ K ( r z i ) + K ( − r z i )] for the average over both signs (antithetic sampling), which cancels every odd-order term, we report at each radius the mean profile E i K ˉ i ( r ) \mathbb{E}_i\bar{K}_i(r) E i K ˉ i ( r ) , the Fisher profile E i Q ( r z i ) \mathbb{E}_iQ(r\mathbf{z}_i) E i Q ( r z i ) , their relative excess, and the quartic coefficient
B ^ i ( r ) = K ˉ i ( r ) − 4 K ˉ i ( r / 2 ) ( 3 / 4 ) r 4 , B ^ ( r ) = E i B ^ i ( r ) , (42) \widehat{B}_i(r) = \frac{\bar{K}_i(r) - 4\bar{K}_i(r/2)}{(3/4)r^4}, \qquad\widehat{B}(r) = \mathbb{E}_i\widehat{B}_i(r), \tag*{(42)} B i ( r ) = ( 3/4 ) r 4 K ˉ i ( r ) − 4 K ˉ i ( r /2 ) , B ( r ) = E i B i ( r ) , (42)
which cancels the quadratic term direction by direction and estimates B = E K 4 ( z ) B = \mathbb{E}K_4(\mathbf{z}) B = E K 4 ( z ) of Proposition 3 with an O ( r 2 ) O(r^2) O ( r 2 ) bias. Agreement of B ^ ( r ) \widehat{B}(r) B ( r ) across radii certifies the quadratic-plus-quartic regime. The quadratic form E i 1 2 z i T F c z i \mathbb{E}_i\frac{1}{2}\mathbf{z}_i^{\mathsf{T}}F_c\mathbf{z}_i E i 2 1 z i T F c z i estimates A = 1 2 tr F c A = \frac{1}{2}\operatorname{tr}F_c A = 2 1 tr F c , the Fisher trace in RMS units. Direction-level standard errors and 1,000 direction bootstrap replicates quantify Monte Carlo uncertainty.
LLC estimates. For every radius and each of the sixteen inverse temperatures
τ ∈ { 100 , 250 , 500 , 10 3 , 2500 , 4000 , 5000 , 10 4 , \tau\in\{100,\ 250,\ 500,\ 10^3,\ 2500,\ 4000,\ 5000,\ 10^4, τ ∈ { 100 , 250 , 500 , 1 0 3 , 2500 , 4000 , 5000 , 1 0 4 ,
1.6 × 10 4 , 2.5 × 10 4 , 4 × 10 4 , 6.4 × 10 4 , 10 5 , 2.5 × 10 5 , 4 × 10 5 , 10 6 } , 1.6 \times10^4,\ 2.5 \times10^4,\ 4 \times10^4,\ 6.4 \times10^4,\ 10^5,\ 2.5 \times10^5,\ 4 \times10^5,\ 10^6\}, 1.6 × 1 0 4 , 2.5 × 1 0 4 , 4 × 1 0 4 , 6.4 × 1 0 4 , 1 0 5 , 2.5 × 1 0 5 , 4 × 1 0 5 , 1 0 6 } ,
the estimates q K ( τ , r ) q_K(\tau,r) q K ( τ , r ) and q Q ( τ , r ) q_Q(\tau,r) q Q ( τ , r ) of Equation (27) are computed by self-normalized importance weighting of the 256 prior samples with weights 1 U ( u ) e − τ K ( u ) \mathbf{1}_U(\mathbf{u})e^{-\tau K(\mathbf{u})} 1 U ( u ) e − τ K ( u ) and 1 U ( u ) e − τ Q ( u ) \mathbf{1}_U(\mathbf{u})e^{-\tau Q(\mathbf{u})} 1 U ( u ) e − τ Q ( u ) . The sign pair is the unit of resampling. A setting passes the diagnostics when the effective sample size over the 128 pairs is at least 100 for both targets and both Pareto tail indices k ^ \hat{k} k ^ are below 0.5 ([36 ] ); other settings are excluded from the developmental summaries. Because a Gaussian prior in millions of dimensions concentrates K K K around its mean, the passing region is τ s d [ K ( r z ) ] ≲ 1 \tau\,\mathrm{sd}[K(r\mathbf{z})] \lesssim1 τ sd [ K ( r z )] ≲ 1 ; in this region q K − q Q = τ r 4 B + O ( r 6 ) q_K-q_Q=\tau r^4B+O(r^6) q K − q Q = τ r 4 B + O ( r 6 ) . Larger inverse temperatures are covered by the direct posterior sampling of Appendix § 15 .
Matched ablation. At every component and checkpoint, the same run measures zero-ablation on the same inputs and positions: the component’s output projection weights (W O W_O W O slice, or W o u t W_{\mathrm{out}} W out and b o u t b_{\mathrm{out}} b out ) are set to zero, holding the shared attention output bias fixed, and the drop in the closer contrast, the KL from the clean model, cross-entropy, and accuracy are recorded. Distractors are one seeded incorrect closer per scored position, identical across components and checkpoints. This intervention is not part of the geometry sample set.
Execution. Each of the 6,160 component–checkpoint–coordinate combinations is evaluated in float64 with batches of eight sequences; the computation before the component’s layer is cached, so the cost of a direction pair scales as 1 + 35 ( 16 − ℓ ) / 16 1+35(16-\ell)/16 1 + 35 ( 16 − ℓ ) /16 with the layer ℓ \ell ℓ . Every run records its clean metrics and coordinate scales, and a cell is assembled only when they agree across its runs.
Grid tables. Table 3 summarizes locality and the quadratic-plus-quartic regime over the complete grid, and Table 4 lists the Fisher trace and quartic coefficient of every component at the six calibration checkpoints. Table 5 and 6 repeat both summaries in current-checkpoint RMS units for the seven components of the temperature sweep (1,078 component–checkpoint pairs). In these units the trajectories have median within-component rank correlations of 0.60 (A c A_c A c ) and 0.49 (B c B_c B c ) with those in final-checkpoint units: dividing out the growth of the weights changes the trajectories in part, and we report the fixed units because a displacement in them means the same change of weights at every checkpoint.
Table 3. Locality and the quadratic-plus-quartic regime across the grid. Over all 5,082 component–checkpoint pairs in fixed final-checkpoint RMS units: the fraction of cells in which every sample at the radius lies inside U U U , the median fraction of samples inside U U U , the median over cells of the maximum closer KL among the 256 samples, the median relative excess of the mean loss over its Fisher approximation, and the median (with interquartile range) of the quartic coefficient at the radius relative to its value at r = 0.1 r = 0.1 r = 0.1 .
Table 4. Fisher trace and quartic coefficient at six checkpoints. A = 1 2 tr F C A = \frac{1}{2}\operatorname{tr} F_C A = 2 1 tr F C is the Fisher trace in RMS units and B B B the quartic coefficient at r = 0.1 r = 0.1 r = 0.1 , both in fixed final-checkpoint RMS units at closer positions. Control components are marked *.
Table 5. Locality and the quadratic-plus-quartic regime in current-checkpoint RMS units. Over the 1,078 component–checkpoint pairs of the seven components of the temperature sweep in current-checkpoint RMS units: the fraction of cells in which every sample at the radius lies inside U U U , the median fraction of samples inside U U U , the median over cells of the maximum closer KL among the 256 samples, the median relative excess of the mean loss over its Fisher approximation, and the median (with interquartile range) of the quartic coefficient at the radius relative to its value at r = 0.1 r=0.1 r = 0.1 .
LLC estimates across radius and temperature. The importance-weighted estimates q K ( τ , r ) q_K(\tau,r) q K ( τ , r ) and q Q ( τ , r ) q_Q(\tau,r) q Q ( τ , r ) pass the diagnostics (effective sample size at least 100 of 128 pairs and Pareto $\hat{k} < 0.5 for both targets) for 55% of the ( pair , r , τ ) (\text{pair}, r, \tau) ( pair , r , τ ) settings, the region τ sd [ K ( r z ) ] ≲ 1 \tau\,\operatorname{sd}[K(rz)] \lesssim1 τ sd [ K ( r z )] ≲ 1 beyond which the prior no longer covers the posterior and Appendix § 15 supplies direct sampling. Within it q Q = τ r 2 A c q_Q = \tau r^2 A_c q Q = τ r 2 A c to within 0.4% and q K − q Q = τ r 4 B c q_K - q_Q = \tau r^4 B_c q K − q Q = τ r 4 B c to within 1.1% (medians at r ≤ 0.1 r \le0.1 r ≤ 0.1 ), as Equation (7) predicts, so the estimates across temperature and radius are generated by A c A_c A c and B c B_c B c : temperature rescales them and radius sets their mixture, from pure curvature at r ≤ 0.025 r \le0.025 r ≤ 0.025 to a quartic share of 2.4% at r = 0.2 r = 0.2 r = 0.2 and 9.5% at r = 0.4 r = 0.4 r = 0.4 . Figure 12 shows the A c A_c A c -adjusted associations of the estimates with the ablation trajectories across the passing region.
Figure 12. A c A_c A c -adjusted associations of the importance-sampled estimates across radius and temperature. Median, over circuit components, of the partial rank correlation (adjusted for A c A_c A c , clean loss, weight scale, and training step) between the ablation-KL trajectory and the trajectory of q K q_K q K (A) or of its quartic part q K − q Q q_K-q_Q q K − q Q (B). Cells are blank where fewer than five components pass the importance-sampling diagnostics at 90% of checkpoints or where the correlation is undefined; numbers give the components included, a set that shrinks as τ \tau τ and r r r grow (at r = 0.4 r=0.4 r = 0.4 only attention heads qualify). Within the passing region the estimates are linear in τ \tau τ (§ 5.2 ), so at fixed r r r the unadjusted rank association of q K q_K q K equals that of A c A_c A c over the same components at every τ \tau τ (0.44 over the circuit components shown here, 0.55 over all 33); what remains after adjusting for A c A_c A c is the quartic term τ r 4 B c \tau r^4B_c τ r 4 B c together with the second-order Fisher term − 1 2 τ 2 r 4 tr F c 2 -\frac{1}{2}\tau^2r^4\operatorname{tr}F_c^2 − 2 1 τ 2 r 4 tr F c 2 , and the variation of A across the grid, down to − 0.4 -0.4 − 0.4 in cells with six components, reflects the growing weight of that term and the shrinking passing set, not a change of regime. The quartic part (B) stays close to the association of B c B_c B c throughout.
Figure 10. Inside U U U the loss is quadratic plus quartic. A: ratio of the mean KL along random directions (both signs) to its Fisher approximation, against radius, for four component–checkpoint pairs. B: fraction of samples inside U U U . C: the quartic coefficient estimated at each radius, relative to its value at r = 0.1 r = 0.1 r = 0.1 , with two direction-level standard errors; its constancy supports Equation (7) .
The Fisher spectrum at every checkpoint
Measurement. For each of the 144 components at each of the 154 checkpoints, in the fixed final-checkpoint RMS coordinates of Appendix § 11 , one run measures on the developmental batch: (i) the causal checks of the grid, zero-ablation (contrast drop and KL), the local gain ablation (g = − 0.05 g = -0.05 g = − 0.05 ), and the gain Fisher F g g = ∂ g 2 K c ( 0 ) F_{gg} = \partial_g^2 K_c(0) F g g = ∂ g 2 K c ( 0 ) by central differences at ϵ = 0.0025 \epsilon= 0.0025 ϵ = 0.0025 and ϵ / 2 \epsilon/2 ϵ /2 ; (ii) the grid’s first 64 directions at r = 0.1 r = 0.1 r = 0.1 , r / 2 r/2 r /2 , and r / 4 r/4 r /4 with both signs, the exact quadratic form 1 2 z i T F c z i \frac{1}{2} z_i^\mathsf{T} F_c z_i 2 1 z i T F c z i and ∥ F c z i ∥ 2 \lVert F_c z_i\rVert^2 ∥ F c z i ∥ 2 by Hessian–vector products, which give A c A_c A c , tr F c 2 \operatorname{tr} F_c^2 tr F c 2 , and B c B_c B c exactly as in the grid (for the 64 panel components the values coincide with the grid’s records for bit); (iii) the Fisher spectrum by stochastic Lanczos quadrature ([33 ] ): four unit-norm Gaussian probes, 32 Lanczos steps each with full reorthogonalization, on exact Hessian–vector products of the closer reference KL in float64. Each probe yields Ritz pairs ( θ v , k , w v , k ) (\theta_{v,k}, w_{v,k}) ( θ v , k , w v , k ) , and any spectral sum is estimated as ∑ i f ( μ i ) ≈ d c 4 ∑ v , k w v , k f ( θ v , k ) \sum_i f(\mu_i) \approx\frac{d_c}{4} \sum_{v,k} w_{v,k} f(\theta_{v,k}) ∑ i f ( μ i ) ≈ 4 d c ∑ v , k w v , k f ( θ v , k ) ; the quadratic estimate at any ( τ , r ) (\tau,r) ( τ , r ) is this sum with
f ( μ ) = 1 2 τ r 2 μ / ( 1 + τ r 2 μ ) f(\mu) = \frac{1}{2}\tau r^2\mu/(1+\tau r^2\mu) f ( μ ) = 2 1 τ r 2 μ / ( 1 + τ r 2 μ ) (Equation (5) ). On a two-layer transformer with a computable spectrum the per-probe quadrature reproduces the exact spectral sums to 10 − 15 10^{-15} 1 0 − 15 , the gain Fisher equals twice the quadratic form of the gain direction to 10 − 7 10^{-7} 1 0 − 7 , and the Hutchinson trace (the mean of z ⊤ F c z z^\top F_c z z ⊤ F c z over Gaussian z z z ; [21 ] ) agrees with the exact trace within 1%; on Pythia-1B the quadrature trace agrees with the 64-direction A c A_c A c within its standard error at every cell. Every cell verifies that its replay of the clean forward pass matches the reference to 10 − 7 10^{-7} 1 0 − 7 , and all 22,176 cells are complete.
Independent inputs. The 33-component panel is measured a second time on an independently generated batch of 128 strings (seed 20261001; 11,441 valid positions, 5,553 closer positions), with new directions and probes, for a replication in which nothing is shared with the original batch but the model. Its 5,078 complete cells (of 5,082) give the values quoted in § 5.3 .
Growth of the count, precision, and component specificity. Table 7 lists the median over components of q Q / ( τ r 2 A c ) q_Q/(\tau r^2A_c) q Q / ( τ r 2 A c ) at the two checkpoints of Figure 3 A. The count grows by a median factor
Table 7. Growth of the quadratic estimate. Median over components of q Q ( τ , r ) / ( τ r 2 A c ) q_Q(\tau,r)/(\tau r^2 A_c) q Q ( τ , r ) / ( τ r 2 A c ) , the ratio of the spectral estimate to its small-temperature limit.
caption: tbl:6 : Fisher trace and quartic coefficient at six checkpoints in current-checkpoint RMS units. A = 1 2 tr F c A = \frac{1}{2}\operatorname{tr} F_c A = 2 1 tr F c is the Fisher trace in RMS units, and B B B the quartic coefficient at r = 0.1 r = 0.1 r = 0.1 , both in current-checkpoint RMS units at closer positions, for the seven components of the temperature sweep.
:::
of 3.9 and 4.0 per decade of τ r 2 \tau r^2 τ r 2 ending at 10 5 10^5 1 0 5 and 10 6 10^6 1 0 6 (exponent 0.6) and by 6.7 and 8.4 ending at 10 7 10^7 1 0 7 and 10 8 10^8 1 0 8 , as the bulk of the spectrum (smallest Ritz values ≈ 10 − 9 \approx10^{-9} ≈ 1 0 − 9 ) comes into range: there is no plateau. The count is precise: across the four probes, q Q ( 10 6 ) q_Q(10^6) q Q ( 1 0 6 ) at the final checkpoint has a relative standard deviation of 0.014 (median over components; 0.029 at the 90th percentile). B c B_c B c averages all 64 directions; 97.9% of the 144-component cells have every one of their 64 directions inside U U U at r = 0.1 r = 0.1 r = 0.1 , and the mean fraction inside is 0.998 (the panel’s 128-direction grid gives 88.5% of pairs fully inside, Table 3 ). Pairing each component’s geometry with the population-median zero-ablation trajectory instead of its own gives adjacent-change partials of 0.50 for A c A_c A c (own 0.51), 0.12 for F g g F_{gg} F g g (own 0.97), and 0.13 for q Q ( 10 6 ) q_Q(10^6) q Q ( 1 0 6 ) (own 0.40); with a randomly chosen other component’s trajectory, 0.26, 0.03, and 0.08; the own association exceeds the median of the cross pairings for 82%, 99%, and 83% of the components. The circuit/other contrast of q Q ( 10 6 ) q_Q(10^6) q Q ( 1 0 6 ) holds within heads (adjacent-change partial 0.10 against 0.48) and within MLPs (0.08 against 0.26). Table 8 lists the associations of every geometric quantity with every target, over all components and by group, computed as in Section 5.3 (partial rank correlations given clean loss, weight scale, and training step, with A c A_c A c added for quantities other than A c A_c A c ; adjacent-change partials given the same covariates; moving-block bootstrap intervals with blocks of 16 checkpoints).
The quartic correction at large temperature
Estimator. Write K c = Q c + V c K_c = Q_c + V_c K c = Q c + V c with V c V_c V c the part of the loss beyond its quadratic approximation, and π Q = N ( 0 , ( τ F c + I / r 2 ) − 1 ) \pi_Q = \mathcal{N}(0,(\tau F_c + I/r^2)^{-1}) π Q = N ( 0 , ( τ F c + I / r 2 ) − 1 ) for the posterior of the quadratic loss on the prior φ r \varphi_r φ r . Expanding the estimate (4) to first order in τ V c \tau V_c τ V c gives
q K − q Q = τ E π Q [ V c ] − τ 2 Cov π Q ( Q c , V c ) + O ( τ 3 V c 2 ) , (43) q_K - q_Q = \tau\mathbb{E}_{\pi_Q}[V_c] - \tau^2 \operatorname{Cov}_{\pi_Q}(Q_c,V_c) + O(\tau^3 V_c^2), \tag*{(43)} q K − q Q = τ E π Q [ V c ] − τ 2 Cov π Q ( Q c , V c ) + O ( τ 3 V c 2 ) , (43)
which reduces to τ r 4 B c \tau r^4 B_c τ r 4 B c when τ r 2 μ i ≪ 1 \tau r^2\mu_i \ll1 τ r 2 μ i ≪ 1 for every eigenvalue. The first term is the quartic part of the loss averaged over the directions the posterior still explores; the covariance term involves only the directions that the posterior has pinned, whose variance is 1 / ( τ μ i ) 1/(\tau\mu_i) 1/ ( τ μ i ) rather than r 2 r^2 r 2 , and is of order N s t i f f / τ N_{\mathrm{stiff}}/\tau N stiff / τ , where N s t i f f N_{\mathrm{stiff}} N stiff is the number of such directions. We draw from π Q \pi_Q π Q exactly: with the Jacobian of the scored log-probabilities and Σ = diag ( p ) − p p T \Sigma= \operatorname{diag}(p)-pp^\mathsf{T} Σ = diag ( p ) − p p T the categorical Fisher metric at each scored position, w = τ J T Σ 1 / 2 ξ 1 + ξ 2 / r w = \sqrt{\tau}\,J^\mathsf{T}\Sigma^{1/2}\xi_1 + \xi_2/r w = τ J T Σ 1/2 ξ 1 + ξ 2 / r with standard normal ξ 1 , ξ 2 \xi_1,\xi_2 ξ 1 , ξ 2 has covariance τ F c + I / r 2 \tau F_c + I/r^2 τ F c + I / r 2 , so that u = ( τ F c + I / r 2 ) − 1 w u = (\tau F_c + I/r^2)^{-1}w u = ( τ F c + I / r 2 ) − 1 w , obtained by conjugate gradients on exact Hessian–vector products (relative residual 10 − 4 10^{-4} 1 0 − 4 ), is a draw from π Q \pi_Q π Q . Each draw costs one backward pass, the conjugate-
caption: tbl:8 : Associations across training from the spectrum measurement. Median over components of the partial rank correlation for levels and for adjacent-checkpoint changes; “A c A_c A c given F g g F_{gg} F g g ” adds the gain Fisher to the covariates. Over the 144 components the change association of A c A_c A c with the zero-ablation KL is positive for 97% and its block-bootstrap interval lies above zero for 88%; that of F g g F_{gg} F g g for 99% and 98%; that of q Q q_Q q Q at τ r 2 = 10 6 \tau r^2 = 10^6 τ r 2 = 1 0 6 for 95% and 67%.
:::
gradient solve, one further Hessian–vector product for Q c ( u ) = 1 2 u T F c u Q_c(u) = \frac{1}{2}u^\mathsf{T}F_c u Q c ( u ) = 2 1 u T F c u , and two forward passes for K c ( u ) K_c(u) K c ( u ) and K c ( − u ) K_c(-u) K c ( − u ) , whose average removes every odd-order term. On a two-layer transformer with a computable spectrum the covariance of J T Σ ξ 1 / 2 J^\mathsf{T}\Sigma_\xi^{1/2} J T Σ ξ 1/2 reproduces F c F_c F c , the solve agrees with a direct solve to 10 − 5 10^{-5} 1 0 − 5 , τ E [ Q c ] \tau\,\mathbb{E}[Q_c] τ E [ Q c ] agrees with Equation (5) within 2%, and K c ( u ) − Q c ( u ) K_c(u) - Q_c(u) K c ( u ) − Q c ( u ) agrees draw by draw with the quartic term isolated by the radius contrast of Appendix § 11 to 10 − 4 10^{-4} 1 0 − 4 ; on Pythia-1B, solving to a residual of 10 − 7 10^{-7} 1 0 − 7 instead changes Q c Q_c Q c and V c V_c V c by less than 0.5%.
Campaign. For the 33-component panel at the six calibration checkpoints and every sixth confirmation checkpoint (28 checkpoints), six draws at each of six settings, r = 0.025 r = 0.025 r = 0.025 with τ r 2 ∈ { 10 , 10 2 , 10 3 } \tau r^2 \in\{10, 10^2, 10^3\} τ r 2 ∈ { 10 , 1 0 2 , 1 0 3 } and r = 0.0125 r = 0.0125 r = 0.0125 with τ r 2 ∈ { 10 2 , 10 3 , 10 4 } \tau r^2 \in\{10^2, 10^3, 10^4\} τ r 2 ∈ { 1 0 2 , 1 0 3 , 1 0 4 } , in the fixed final-checkpoint coordinates and on the developmental batch; every draw is checked against the region U U U .
Results. Table 9 summarizes the 924 complete cells. At every setting the Monte Carlo quadratic part reproduces the quadratic q Q q_Q q Q at the same τ r 2 \tau r^2 τ r 2 (median ratio 1.001–1.022; within 10% for 99% of the cells at τ r 2 = 10 4 \tau r^2 = 10^4 τ r 2 = 1 0 4 , where the posterior is narrow, and within 20% for 81% at τ r 2 = 10 \tau r^2 = 10 τ r 2 = 10 , where six draws of a prior-dominated quadratic form are noisy), which cross-validates the Lanczos spectra with an independent estimator. The posterior-weighted quartic coefficient E [ V c ] / r 4 \mathbb{E}[V_c]/r^4 E [ V c ] / r 4 equals B c B_c B c at every setting (medians 0.91–1.05 of B c B_c B c ), so the quartic part of the loss lies in the directions the posterior still explores, the correction is tr r 4 B c \operatorname{tr} r^4 B_c tr r 4 B c throughout, and it grows linearly in τ \tau τ while q Q q_Q q Q grows sublinearly; its share of the estimate is 0.041% at τ r 2 = 10 \tau r^2 = 10 τ r 2 = 10 and 0.032% at 10 4 10^4 1 0 4 (medians; 90th percentiles below 0.33%). The expansion is first order in τ V c \tau V_c τ V c : max τ ∣ V c ∣ < 0.3 \max\tau|V_c| < 0.3 max τ ∣ V c ∣ < 0.3 holds for every cell at τ r 2 ≤ 10 2 \tau r^2 \le10^2 τ r 2 ≤ 1 0 2 , for 97% and 80% at 10 3 10^3 1 0 3 (r = 0.0125 r = 0.0125 r = 0.0125 and 0.025), and for 70% at 10 4 10^4 1 0 4 , the exceptions being MPLs, whose B c B_c B c is largest; for those cells the first-order correction is 0.12% of the estimate in median and 2.39% at most, and the sample covariance term stays below 0.42 of δ q \delta q δ q in median, so the large-temperature estimate remains the soft rank to within a percent. Every draw lies inside U U U . Projected with the measured spectra, the leading term τ r 4 B c \tau r^4 B_c τ r 4 B c would be 0.19% of q Q q_Q q Q at τ r 2 = 10 6 \tau r^2 = 10^6 τ r 2 = 1 0 6 for r = 0.0125 r = 0.0125 r = 0.0125 (median over cells).
Table 9. The quartic correction at large temperature. Medians over the panel cells of the share of the correction in the corrected estimate q ^ = τ E [ Q c ] + δ q \hat{q}=\tau\mathbb{E}[Q_c]+\delta q q ^ = τ E [ Q c ] + δ q , the Monte Carlo quadratic part relative to the quadrature q Q q_Q q Q at the same τ r 2 \tau r^2 τ r 2 , the posterior-weighted quartic coefficient relative to B c B_c B c , and the first-order validity max τ ∣ V c ∣ \max\tau|V_c| max τ ∣ V c ∣ .
Where the change of the Fisher trace comes from
Hybrid checkpoints. A c = 1 2 tr ( J T Σ c J ) A_c = \frac{1}{2}\operatorname{tr}(J^\mathsf{T}\Sigma_cJ) A c = 2 1 tr ( J T Σ c J ) depends on the component’s own weights (through its output map J = ∂ o c / ∂ u J = \partial o_c/\partial u J = ∂ o c / ∂ u ), the upstream weights that set its input, and on the downstream weights that set the metric Σ c \Sigma_c Σ c . For two adjacent checkpoints t < t ′ t<t' t < t ′ of the 28 checkpoints of Appendix § 13 we partition the parameters into three groups, the component’s own tensors (J), the embedding and the blocks before its layer (I), and everything else, the blocks after its layer, the final normalization, the unembedding, and the remaining tensors of its own block (M), and evaluate A c A_c A c on the eight models that take each group from t t t or from t ′ t' t ′ (the loader folds layer norms and centers writing weights within a block, so splicing whole blocks is exact). The reference distribution is each hybrid’s own predictions, the 64 directions are those of the spectrum campaign, and the evaluation of the all-t t t model replays the spectrum profile bit for bit, as the all-t ′ t' t ′ hybrid replays the next checkpoint. The change Δ A c = A c ( t ′ ) − A c ( t ) \Delta A_c=A_c(t')-A_c(t) Δ A c = A c ( t ′ ) − A c ( t ) is attributed exactly to the three groups by Shapley values over the eight evaluations, which sum to Δ A c \Delta A_c Δ A c cell by cell; the single-hybrid parts alone do not add up (their residual is 38% of the attributed magnitude), because a hybrid of checkpoints 6,000 steps apart is a mismatched network whose clean loss moves by up to 0.2 nats. All 891 cells (33 components, 27 pairs) are complete.
Results. Over the 615 cells in which ∣ Δ A c ∣ |\Delta A_c| ∣Δ A c ∣ exceeds 5% of A c A_c A c , the metric group carries 0.44 of the attributed change, the input group 0.35, and the component’s own tensors 0.21; the metric and input attributions have the sign of Δ A c \Delta A_c Δ A c in 68% and 67% of the cells (median signed shares 0.31 and 0.44), the own-tensor attribution in 49% (median share 0.00). Heads and MLPs differ: for heads the metric group carries a median share of 0.50 of a change and the input group 0.24; for MLPs the input group carries 0.92 and the metric group 0.06. The late rise of A c A_c A c (pairs from step 8,000 on, summed per component) is carried by the input group (median 1.08 of the rise, interquartile range 0.33–1.99), with the metric group at 0.05 (−0.86–0.47) and the component’s own tensors at −0.23 (−0.49–0.00): on their own, the component’s parameters would have lowered its Fisher trace. In adjacent-change associations with the component’s zero-ablation KL over these 28 checkpoints (partials given clean loss, weight scale, and, for the parts, A c A_c A c ), the full A c A_c A c series gives 0.44 (0.39 against the population-median trajectory), the cumulative metric series 0.17 (0.12), the own-tensor series 0.12 (−0.06; 0.35 in levels), and the input series 0.02, so no single group carries the co-movement.
Direct posterior sampling across temperatures at six checkpoints
The importance-weighted LLC estimates of Appendix D pass their diagnostics only where the posterior weights remain close to the prior. To cover larger inverse temperatures, where the posterior concentrates on the low-loss valley of a component, we sample the localized posteriors directly for seven components (L0H0, L0H2, L0.mlp, L2.mlp, L7H4, L11.mlp, L14H1) at six checkpoints (0, 512, 2,000, 8,000, 32,000, 143,000), three radii (0.05, 0.15, 0.4 in current-checkpoint tensor-RMS coordinates), and five inverse temperatures (10 2 10^2 1 0 2 , 10 3 10^3 1 0 3 , 10 4 10^4 1 0 4 , 10 5 10^5 1 0 5 , 10 6 10^6 1 0 6 ). Both the actual reference-KL target and the exact Fisher-quadratic target are sampled with the same prior: 1,260 targets, four chains each, 1,024 adaptive warmup iterations followed by 1,024 fixed-kernel draws per chain, with a Metropolis-adjusted prior-preserving Langevin proposal whose gradients are the exact full-batch gradients of the target (the final-checkpoint panel of Appendix § 20 uses preconditioned Crank–Nicolson (pCN) proposals; this sweep uses gradient-informed proposals). Model arithmetic is float32 with float64 loss reductions. A target passes the energy diagnostic when the four chains have R ^ < 1.01 \hat{R}<1.01 R ^ < 1.01 and effective sample sizes above 400 for the energy observable; 405 of the 630 matched actual/quadratic pairs pass for both targets, and 329 pass for every recorded observable. The targets that do not pass concentrate at τ ≥ 10 5 \tau\geq10^5 τ ≥ 1 0 5 , where the stiffest directions force a small step and the remaining directions decorrelate slowly, and at r = 0.4 r=0.4 r = 0.4 for the MLPs, where the sampled states leave the region U U U ; every target is reported (Figure 13 ).
Figure 13. Direct posterior sampling across temperatures. A: for L2.mlp at step 2,000, the LLC estimate q K q_K q K and its Fisher counterpart q Q q_Q q Q as functions of the inverse temperature at three radii. Both grow almost linearly in τ \tau τ over the sampled range, with a mild sublinearity toward a soft count of constrained directions; the actual response exceeds the Fisher response most at moderate τ \tau τ and large radius, where the posterior explores the scale at which quartic terms matter. B: the beyond-Fisher fraction ( q K − q Q ) / q Q (q_K-q_Q)/q_Q ( q K − q Q ) / q Q at r = 0.4 r=0.4 r = 0.4 and τ = 10 3 \tau=10^3 τ = 1 0 3 across the six checkpoints for all seven components.
The two regimes. At the smallest radius the two estimates agree to within 1.2% at every temperature and checkpoint: the geometry is quadratic at that scale. At r = 0.15 r=0.15 r = 0.15 the excess is 1–10% and at r = 0.4 r=0.4 r = 0.4 it reaches 30–60% for the MLPs, but at r = 0.4 r=0.4 r = 0.4 the MLP chains leave the region U U U (maximum closer KL up to 0.05 nats and ∣ Δ C E ∣ |\Delta\mathrm{CE}| ∣Δ CE ∣ up to 0.38), so only the heads pass there. Where the chains pass their diagnostics (τ ≤ 10 4 \tau\leq10^4 τ ≤ 1 0 4 at r ≤ 0.15 r\leq0.15 r ≤ 0.15 ), the response is linear in τ \tau τ and the relative excess is independent of τ \tau τ and proportional to r 2 r^2 r 2 : the posterior is still the prior, and the sweep measures the same prior-averaged quartic term τ r 4 B c \tau r^4B_c τ r 4 B c as the developmental grid, with the precision of 4,096 draws. At τ ≥ 10 5 \tau\geq10^5 τ ≥ 1 0 5 the sampled excess decreases toward zero, as expected when the posterior concentrates on states with K ≲ 1 / τ K\lesssim1/\tau K ≲ 1/ τ , but there the chains fail the R ^ \hat{R} R ^ and effective-sample-size screen, so the contraction regime is indicated rather than resolved. The information beyond curvature that these samplers can certify therefore resides in the finite-radius regime that the developmental grid measures at every checkpoint.
Developmental signal. The relative excess ( q K − q Q ) / q Q (q_K-q_Q)/q_Q ( q K − q Q ) / q Q is component-specific and changes across training. For L0.mlp at r = 0.4 r=0.4 r = 0.4 and τ = 10 3 \tau=10^3 τ = 1 0 3 it is − 13 % -13\% − 13% at initialization (the loss saturates below its quadratic approximation), rises to + 18 % +18\% + 18% at step 512 and + 34 % +34\% + 34% at step 2,000 during circuit formation, and decays to + 3 % +3\% + 3% at the final checkpoint. L2.mlp and L11.mlp rise to + 44 % +44\% + 44% and + 26 % +26\% + 26% at step 2,000 and remain elevated; the attention heads L14H1, L0H0, L0H2, and L7H4 stay between + 2 % +2\% + 2% and + 11 % +11\% + 11% . The same ordering holds at τ = 10 2 \tau=10^2 τ = 1 0 2 and at r = 0.15 r=0.15 r = 0.15 . Table 10 lists the complete values at both radii. The quartic coefficient of § 5.2 resolves the same quantity at every checkpoint with a direction-level Monte Carlo error rather than a chain diagnostic.
Table 10. Beyond-Fisher fraction from direct posterior sampling. Entries are 100 ( q K − q Q ) / q Q 100(q_K-q_Q)/q_Q 100 ( q K − q Q ) / q Q at inverse temperature 10 3 10^3 1 0 3 for two radii and six checkpoints (columns are training steps). † marks pairs in which at least one target fails the four-chain energy diagnostic. At r = 0.05 r=0.05 r = 0.05 every entry is within 1.2% of zero.
Experimental protocols
SGLD sweep of all components
A complementary sweep applies the SGLD estimator of prior work [24 ] to every component at every checkpoint. Its chains move the weights well beyond U U U at this scale (Appendix § 18 ), so we treat its outputs as finite perturbation responses at a larger scale than the local measurements of Appendix § 11 and compare the two in § 5.3 . The sweep contains 2 × 154 × ( 144 + 16 + 1 ) = 49,588 2 \times154 \times(144+16+1)=49{,}588 2 × 154 × ( 144 + 16 + 1 ) = 49 , 588 finite SGLD responses, covering both task variants, all single components, complete layers, and the full model. Each restriction uses two fp32 SGLD chains, comprising 100–150 initial updates and 60 subsequent draws. The response R C R_C R C averages all updates. The objective is full-vocabulary cross-entropy at closer targets, with zero contribution at other positions and normalization over all next-token positions. Sampling settings are given below; Appendix § 17 analyzes the averaging window, and Appendix § 18 measures the resulting parameter displacements and compares them with localized posterior sampling.
Bracket tasks and intervention scores
Synthetic examples use space-prefixed bracket tokens after a Brackets: prefix. The bounded Dyck generator uses ( k , m ) = ( 2 , 10 ) (k,m)=(2,10) ( k , m ) = ( 2 , 10 ) or ( 4 , 16 ) (4,16) ( 4 , 16 ) and opening probability 0.45 whenever either action is legal; the boundaries force the appropriate action. At the final checkpoint, the top prediction at the 5,475 scored closer positions of the k = 4 k=4 k = 4 developmental batch is the correct closer at 59.5% of them, another closer at 6.0%, an opening bracket at 31.0%, and another token at 3.5%, so 77% of the errors are openers; restricted to the four closers, the correct one ranks first at 86.0%. An opening bracket is a legal continuation at 89% of the scored positions; at the others the string’s end or the depth bound forces a closer. A scored closer position t t t predicts x t + 1 x_{t+1} x t + 1 . The pairwise contrast is the correct next-token logit minus one seeded incorrect-closer logit, averaged over scored positions. Full-vocabulary accuracy separately measures the next-token argmax.
For natural-context MI on the natural bracket contexts, we recognize plain and space-prefixed brackets, reconstruct the stack, and score stack-consistent closers against same-form distractors. The developmental SGLD objective masks recognized closer token IDs. The reference-KL posterior and intervention comparisons use identical inputs and masks.
Zero-ablation removes a head’s pre-output-projection aggregation or an MLP output. Position-conditioned mean replacement averages outputs across examples at each position. The longitudinal bank experiment additionally records a global mean over examples and positions; Figure 26 uses its zero-ablation condition. A positive contrast drop denotes a harmful intervention.
Figure 26. Late perturbation-response change accompanies changing causal organization. The first-layer bank’s finite SGLD response trajectory is compared with joint and marginal zero-ablation under a fixed synthetic protocol at six checkpoints. Signed and absolute marginal sums are both small near the response trough and approach the joint effect late in training.
Developmental SGLD sweep
We use the unpreconditioned SGLD update of Welling & Teh [43 ] with a component mask and localization term. The synthetic grid uses 128 strings, length 128, data seed 0, batch size 8, two fp32 chains, and the settings in Table 11 . Each chain saves 60 draws after its nominal burn-in. The reported statistic averages all updates: 160 for heads and MLPs, 180 for layers, and 210 for the full model. The loss is
Table 11. Developmental SGLD settings, shared by both Dyck variants. Larger restrictions use manually scaled settings following a head-level calibration at checkpoint 2,000.
ℓ t ( θ ; x ) = − 1 { x t + 1 ∈ V c l o s e } log p θ ( x t + 1 ∣ x 1 : t ) , L B = 1 B ( T − 1 ) ∑ b , t ℓ t ( θ ; x b ) . (44) \ell_t(\theta;x)=-\mathbf{1}\{x_{t+1}\in V_{\mathrm{close}}\}\log p_\theta(x_{t+1}\mid x_{1:t}),\qquad L_B=\frac{1}{B(T-1)}\sum_{b,t}\ell_t(\theta;x_b). \tag*{(44)} ℓ t ( θ ; x ) = − 1 { x t + 1 ∈ V close } log p θ ( x t + 1 ∣ x 1 : t ) , L B = B ( T − 1 ) 1 b , t ∑ ℓ t ( θ ; x b ) . (44)
The all-position denominator makes closer frequency part of the objective’s scale. The grid contains 49,588 finite estimates.
For displacement u u u from the checkpoint, the unpreconditioned update is
u s + 1 = u s − η 2 [ n β ∇ L B s ( u s ) + γ u s ] + η ξ s , ξ s ∼ N ( 0 , I ) . (45) u_{s+1} = u_s - \frac{\eta}{2}\left[n\beta\nabla L_{B_s}(u_s) + \gamma u_s\right] + \sqrt{\eta}\,\xi_s,\qquad\xi_s \sim\mathcal{N}(0,I). \tag*{(45)} u s + 1 = u s − 2 η [ n β ∇ L B s ( u s ) + γ u s ] + η ξ s , ξ s ∼ N ( 0 , I ) . (45)
Parameter masks restrict both drift and noise. A head includes its slices of W Q W_Q W Q , W K W_K W K , W V W_V W V , W O W_O W O and b Q b_Q b Q , b K b_K b K , b V b_V b V ; the shared attention output bias remains fixed. MLP restrictions include input/output weights and biases. The recorded configurations use initial seed 100, matched minibatch order across chains, and cyclic replay of a shuffled first epoch. Appendix § 17 compares averaging conventions and characterizes the finite-horizon sampling response.
A natural-context SGLD sweep covers 12 checkpoints and all 144 components; its cross-distribution associations are reported separately from the reference-KL comparison.
Gradient activity and temporal comparisons
The developmental gradient baseline accumulates gradients of closer-normalized cross-entropy over chunks and then takes the squared block norm:
G c = ∥ ∑ b ∇ θ c L ‾ b ∥ 2 . (46) G_c = \left\lVert\sum_b \nabla_{\theta_c}\overline{L}_b \right\rVert^2. \tag*{(46)} G c = b ∑ ∇ θ c L b 2 . (46)
Its magnitude reflects chunk normalization and cancellation between gradients. This quantity is labeled gradient activity throughout.
Temporal Spearman correlations pair each component’s k = 4 k = 4 k = 4 series with the k = 4 k = 4 k = 4 distance-probe trajectory. Rank-time adjustment regresses each ranked series on an intercept and checkpoint rank, then correlates the residuals. First differences measure increments between adjacent checkpoints on the nonuniform training grid.
The early-layer intervention analysis refits distance probes after zero-ablation, measuring recoverable decodability at 27 components and 12 checkpoints with position-level evaluation splits. The frozen-decoder experiment in Appendix § 24 .§ 5 instead uses disjoint strings and a common decoder across conditions.
Subspace and group interventions
Subspace replacement uses k = 4 k = 4 k = 4 , depth bound 16, maximum bracket length 48 and fp32 at checkpoint 143,000. Basis reconstruction uses 256 seed-42 fitting strings; evaluations use 128 strings per seed. The named TOKEN, STACK, POSITION, and ANSWER spaces are defined in Appendix § 24 . Ten MLPs and fourteen heads receive the designated replacements, with a token lookup at L0.mlp and position-conditioned means for the eight first-layer heads. Remaining components execute normally. Bases fitted on seed 42 are evaluated without refitting on seeds 43–45. Recovery divides the intervened contrast by the clean contrast on the same batch. The PCA comparator retains 128 directions per MLP and 64 per head. Named-space augmentation adds 64 orthogonal principal directions per component.
The cross-distribution bank experiment uses 128 examples, length 256, seed 42, and fp32, yielding 11,882 synthetic and 187 natural-context scored positions. Its final joint zero-ablation drop is 1.78.
The longitudinal experiment uses 256 synthetic examples, length 128, seed 0, and fp32 at the six checkpoints in Table 21 ; its final joint drop is 1.56.
Table 21. First-layer bank effects under the fixed longitudinal protocol. From step 8,000 on, the joint drop exceeds even the absolute marginal sum, establishing nonadditivity beyond signed cancellation.
Averaging sensitivity in the developmental sweep
We compare estimates averaged over all SGLD updates with estimates using only the final 60 draws. Available traces cover 23,155 unique k = 2 k = 2 k = 2 and 23,102 unique k = 4 k = 4 k = 4 conditions out of 24,794 per variant, including layer and full-model restrictions. Each trace is matched to a measurement by its initial loss, ordered chain coefficients, and restriction configuration.
Effect of the averaging window. The k = 4 k = 4 k = 4 atomic subset contains 20,654 paired cells: 142–146 checkpoints per component, with all 144 components represented at 140 checkpoints. On those complete checkpoints, the median correlation between the two component rankings is 0.994. Retained-only estimates are larger by a median 0.539; among all-update values above 0.1, the median ratio is 1.89. Thus averaging changes the absolute scale substantially while preserving the component ordering.
The developmental association persists when averaging only the final 60 draws (Table 12 ). At ten checkpoints with complete shared coverage, applying this window to the synthetic measurements changes the median cross-distribution correlation from 0.897 to 0.892; natural-context measurements use all-update averages. The paired comparison covers the checkpoints with stored traces, which excludes the final k = 4 k = 4 k = 4 checkpoint.
Table 12. Median componentwise association with the k = 4 k = 4 k = 4 layer-2 distance-probe trajectory on paired observations. First differences use adjacent released checkpoints for which both traces are available. Trace coverage determines the subset of the full trajectory analyzed here.
Finite-horizon behavior. Within the 60 retained draws, mean loss rises from the first to the last half in 90.5% of the 20,654 atomic cells. The median increase is 0.128 in all-position-normalized masked NLL. We therefore interpret the developmental sweep as a finite-horizon sampling response. The matched reference-KL experiment uses a separate posterior target and multichain diagnostic protocol.
Parameter displacement of SGLD chains
Finite diffusion and parameter displacement
The SGLD settings in Table 11 produce substantial diffusion within the recorded trajectories. With the loss-gradient term removed the localized update is u s + 1 = ( 1 − η γ / 2 ) u s + η ξ s u_{s+1} = (1 - \eta\gamma/2)u_s + \sqrt{\eta}\xi_s u s + 1 = ( 1 − η γ /2 ) u s + η ξ s with ξ s ∼ N ( 0 , I ) \xi_s \sim\mathcal{N}(0,I) ξ s ∼ N ( 0 , I ) , whose per-coordinate variance after s s s updates is η ∑ j < s ( 1 − η γ / 2 ) 2 \eta\sum_{j<s}(1-\eta\gamma/2)^2 η ∑ j < s ( 1 − η γ /2 ) 2 ; for η γ ≪ 1 \eta\gamma\ll1 η γ ≪ 1 : for a head this gives an endpoint standard deviation close to 160 ( 2.5 × 10 − 5 ) = 0.0632 \sqrt{160(2.5 \times10^{-5})} = 0.0632 160 ( 2.5 × 1 0 − 5 ) = 0.0632 per coordinate. The accumulated localization factor is η γ S / 2 = 2.4 × 10 − 5 \eta\gamma S/2 = 2.4 \times10^{-5} η γ S /2 = 2.4 × 1 0 − 5 . Across all 144 components and 154 checkpoints, the median ratio of the predicted endpoint standard deviation to the RMS of an individual weight matrix is 2.59. Bias tensors are excluded from this relative-scale comparison.
We directly measure the dynamics at steps 0, 2,000, 32,000, and 143,000 for L0H0, L0H2, L7H4, L11, m1p1, L1H4H1, and the full model. Each of these 24 conditions uses two chains under three protocols: the documented SGLD settings; a control with the loss-gradient term removed; and strongly localized SGLD with η = 10 − 7 \eta= 10^{-7} η = 1 0 − 7 and γ = 10 6 \gamma= 10^{6} γ = 1 0 6 . The first two protocols share every Gaussian increment. We execute the optimizer implementation of the reference library ([30 ] ) and record the applied loss-gradient, localization, and noise terms separately.
The median endpoint displacement relative to weight-matrix RMS is 4.76 under the documented settings and 0.078 under strong localization. Holding a chain at a displacement of 0.05 RMS requires γ ≈ 7 × 10 5 \gamma\approx7 \times10^{5} γ ≈ 7 × 1 0 5 (the stationary per-coordinate standard deviation is γ − 1 / 2 \gamma^{-1/2} γ − 1/2 ), seven orders of magnitude above the sweep’s value, at which the loss-gradient term is negligible and the chain samples the localizing prior. The median loss-gradient/noise norm ratio per update is 9.8 × 10 − 6 9.8 \times10^{-6} 9.8 × 1 0 − 6 ; the corresponding localization/noise ratio is 1.3 × 10 − 6 1.3 \times10^{-6} 1.3 × 1 0 − 6 . Nevertheless, the loss-gradient term measurably changes the predictive response. At the final full-model checkpoint, the clean closer loss is 1.379 nats, the mean sampled loss is 10.562 under SGLD, and it is 11.594 with that term removed. Strong localization gives 1.408. These finite-path controls establish the perturbation scale; equilibrium posterior measurements use the separate protocol below (Figure 14 ).
Figure 14. Parameter diffusion and its predictive effect. Full-model perturbations at checkpoint 143,000 under the documented SGLD settings, a matched control with the loss-gradient term removed, and strongly localized SGLD. Lines average two chains; shading spans their values. Gaussian increments are identical between the first two protocols.
Standard settings
The same 24 conditions were rerun with the settings of the reference implementation (Snell et al. [30 ] ): localization γ = 100 \gamma= 100 γ = 100 centered at the checkpoint, inverse temperature n β = n / log n = 636 n\beta= n/\log n = 636 n β = n / log n = 636 with n = 5,475 n = 5{,}475 n = 5 , 475 the scored closer positions of the batch (and n β = 1,700 n\beta= 1{,}700 n β = 1 , 700 ), step sizes η ∈ { 10 − 5 , 10 − 4 } \eta\in\{10^{-5},10^{-4}\} η ∈ { 1 0 − 5 , 1 0 − 4 } , 160 updates, minibatches of eight strings, two chains sharing their Gaussian increments across settings. Table 13 summarizes the endpoints. No chain stays inside U U U : the displacement is set by the noise term, η s \sqrt{\eta}s η s per coordinate, and is the same at both inverse temperatures, while the loss-gradient drift, one hundred times stronger than under the documented settings but still small relative to the noise, fine-tunes the model on the batch, so that the mean loss over the chain falls below the clean loss and the estimator’s own statistic n β ( L ‾ − L c l e a n ) n\beta\left(\overline{L}-L_{\mathrm{clean}}\right) n β ( L − L clean ) is negative for 21–24 of the 24 conditions (− 172 -172 − 172 and − 544 -544 − 544 in median at the two inverse temperatures for η = 10 − 5 \eta= 10^{-5} η = 1 0 − 5 ); at η = 10 − 4 \eta= 10^{-4} η = 1 0 − 4 the full model and the MLPs diverge. Localization strong enough to hold a chain inside U U U (γ ≈ 7 × 10 5 \gamma\approx7 \times10^{5} γ ≈ 7 × 1 0 5 , as above) leaves only the prior, so the region has to be imposed on the draws rather than reached by tuning the chain.
Table 13. SGLD endpoints under standard, documented, and strongly localized settings. Medians over the 24 conditions and two chains: endpoint displacement relative to weight-matrix RMS, chains whose endpoint lies inside U U U , closer KL from the clean checkpoint, per-update norm ratios of the loss-gradient and localization terms to the noise, and the closer cross-entropy of the full-model chain at the final checkpoint (clean 1.379).
Clean-loss and parameter-scale controls
With the covariates of § 5.3 (clean loss, weight scale, and training step), the finite SGLD response’s partial association with the opener-distance probe over the 148 confirmation checkpoints is 0.46 in levels (0.47 with A c A_c A c added) and 0.02 in adjacent changes (medians over the 144 components), the values quoted in § 5.3 .
The stored initial loss agrees with independently evaluated closer cross-entropy, after applying the all-position normalization, to within 7.8 × 10 − 7 7.8 \times10^{-7} 7.8 × 1 0 − 7 . The scored fractions are 0.342581 for k = 2 k = 2 k = 2 and 0.336799 for k = 4 k = 4 k = 4 . We reconstruct the mean perturbed loss L ‾ = L c l e a n + R / ( n β ) \overline{L}=L_{\mathrm{clean}}+R/(n\beta) L = L clean + R / ( n β ) and compare the full-model response with n β ( f log V − L c l e a n ) n\beta(f\log V-L_{\mathrm{clean}}) n β ( f log V − L clean ) , where V = 50,304 V=50{,}304 V = 50 , 304 and f f f is the closer fraction.
Across all 154 checkpoints, Pearson correlations between the response and negative clean loss are 0.989 and 0.991 for k = 2 k = 2 k = 2 and k = 4 k = 4 k = 4 . The unfitted uniform-output predictions have R 2 = − 0.110 R^2 = -0.110 R 2 = − 0.110 and 0.602. Fitting a constant mean perturbed loss gives descriptive R 2 = 0.855 R^2 = 0.855 R 2 = 0.855 and 0.918, with implied closer losses of 9.04 and 9.81 nats. The response is thus closely associated with clean loss without being identical to the uniform-output prediction. Restricting to checkpoints at or after step 2,000 reduces the constant-loss fit to R 2 = 0.182 R^2 = 0.182 R 2 = 0.182 and 0.486 (Figure 15 ).
Figure 15. Full-model response and clean-loss controls. Left: observed finite SGLD response, the unfitted uniform-output prediction, and a descriptive fit with constant mean perturbed loss. Right: clean loss and mean perturbed loss reconstructed from the stored response. Every panel contains all 154 checkpoints.
At component resolution, the negative-clean-loss association with the primary k = 4 k = 4 k = 4 distance target is −0.111 after rank-time adjustment. Controlling both checkpoint rank and clean-loss rank leaves a median SGLD association of 0.605. Inverse weight-matrix RMS has an adjusted association of 0.509; adding each component’s weight RMS to the controls reduces the SGLD association to 0.456 (0.237 for k = 2 k = 2 k = 2 ). These results separate the full-model clean-loss effect from the component response and its dependence on perturbation scale.
Calibration of local geometry against matched causal effects
A matched comparison of circuit geometry and causal effects
At the final checkpoint, geometric and causal measurements use the same full-vocabulary reference KL at identical closer positions. Each frozen batch contains 128 sequences: 5,493 synthetic k = 4 k = 4 k = 4 and 177 natural-context closer targets. We measure
q C = τ E π C [ K C ( u ) ] , π C ( u ) ∝ e − τ K C ( u ) N ( u ; 0 , r 2 I ) , (47) q_C = \tau\mathbb{E}_{\pi_C}[K_C(u)], \qquad\pi_C(u) \propto e^{-\tau K_C(u)}\mathcal{N}(u;0,r^2 I), \tag*{(47)} q C = τ E π C [ K C ( u )] , π C ( u ) ∝ e − τ K C ( u ) N ( u ; 0 , r 2 I ) , (47)
where K C K_C K C is (1) for component C C C in current-checkpoint RMS coordinates. We use r = 0.1 r = 0.1 r = 0.1 , τ = 1,000 \tau= 1{,}000 τ = 1 , 000 , and prior-preserving Metropolis sampling of the complete fixed sample batch ([9 ] ). All 288 target responses pass the four-chain diagnostic screen ([35 ] ). Zero or mean output replacement gives the matched MI scores. The estimate rank-correlates with the model Fisher trace in the same coordinates (a Hutchinson estimate with 32 probes) at 0.997 on the task inputs and 0.999 on natural contexts, as Equation (5) predicts at this temperature.
On synthetic bracket completion, the LLC estimate tracks component effects: its Spearman association with zero-ablation KL is 0.64 over all components, 0.51 within the 128 heads, and 0.80 within the 16 MLPs (Figure 16 ). Mean replacement gives an overall association of 0.64. The correspondence therefore extends within both component classes. Natural-context associations are weaker: 0.16 for zero replacement and 0.14 for mean replacement, with head-only values of −0.18 and −0.19.
Figure 16. Geometric sensitivity and causal circuit effects measured on the same behavior. Each point is one component: attention heads are blue and MLPs orange. The horizontal axis is the LLC estimate under component-restricted parameter perturbations; the vertical axis is reference-model KL under zero replacement of that component’s output. Both measurements use identical inputs and closer positions. Insets report Spearman correlations over all components and within each component class.
The matched panel also separates dependence on the input distribution from dependence on the measurement. The cross-distribution rank correlation is 0.61 for the LLC estimate, 0.67 for zero-ablation KL, and 0.59 for mean-ablation KL. Appendix § 21 gives the complete comparisons for conditional KL, target cross-entropy, pairwise closer contrast, and accuracy.
Across intervention strengths ϵ = 0.01 \epsilon= 0.01 ϵ = 0.01 to 1, the synthetic association remains approximately 0.64, while the natural-context association changes from 0.19 to 0.16. KL scales quadratically at the smallest strengths. The disagreement thus persists locally, distinguishing directional sensitivity from the LLC estimate’s aggregation over parameter directions (Appendix § 21 .§ 3 ).
A coordinate control. Repeating all 288 posteriors with an isotropic prior in the fixed converted TransformerLens tensors gives zero-ablation associations of 0.861 on synthetic inputs and 0.561 on natural contexts. This prior has absolute standard deviation 0.001, while the primary prior uses relative RMS radius 0.1, changing both parameter weighting and perturbation scale. Their causal alignment therefore depends on the specified parameter metric and localization (Appendix § 21 .§ 1 ).
Matched joint geometry. Across checkpoints 8,000, 32,000, and 143,000, the bank’s matched LLC estimate increases 1.07 → 1.80 → 3.90 on synthetic inputs and 0.20 → 0.35 → 0.91 on natural-context inputs. At the final checkpoint, the joint synthetic response is 5.8% below the sum of the eight head responses (3.90 versus 4.14). The gap exceeds a conservative Monte Carlo standard error (MCSE) bound accounting for shared seeds by a factor of 5.7. This joint deficit matches the coupling prediction of Equation (17) ; a separable local loss gives exact additivity. Natural-context joint and summed responses differ by less than 0.7% across all three checkpoints (Appendix § 23 .§ 2 ).
Across the fixed 18-component panel, factor-two changes in radius or inverse temperature preserve ranks (ρ = 0.975 \rho= 0.975 ρ = 0.975 –1.000 1.000 1.000 ); median responses follow τ r 2 \tau r^2 τ r 2 scaling within 3.6%. Changing the proposal coefficient reproduces all 36 component responses within three conservative MCSE bounds (Appendix § 23 ).
Matched localized posterior measurements
The matched experiment evaluates the parameter geometry and causal responses of the same model on the same frozen inputs and output readout. At checkpoint 143,000, we measure every attention head and MLP on synthetic and natural-language-context bracket data. This experiment uses a reference-model KL objective and a precisely specified localized posterior. We denote its response by q c q_c q c , and the developmental sweep’s finite SGLD response by R c R_c R c .
Reference loss and parameter chart
We use TransformerLens 3.1.0 ([27 ] ) with its standard LayerNorm folding, value-bias folding, residual-write centering, and unembedding centering. The parameter vector θ \theta θ denotes these converted tensors. Raw coordinates apply equal per-coordinate weighting to this fixed parameterization; RMS coordinates rescale its tensors as below. Value-bias folding places the pretrained constant head writes in the shared attention output bias, which remains fixed in both the head interventions and the parameter restrictions. All matched comparisons use this same component decomposition. Let θ ⋆ \theta_\star θ ⋆ denote the checkpoint parameters and let c c c select a component with d c d_c d c parameters. We write
θ ( u ) = θ ⋆ + D c u , u ∈ R d c , (48) \theta(u) = \theta_\star+ D_c u,\qquad u \in\mathbb{R}^{d_c}, \tag*{(48)} θ ( u ) = θ ⋆ + D c u , u ∈ R d c , (48)
where D c D_c D c embeds the component coordinates into the full parameter space and rescales each selected tensor by its pretrained root-mean-square value. For a selected tensor j j j , its scale is
s j = max { 1 d j ∥ θ ⋆ , j ∥ 2 2 , 10 − 3 median k ∈ W c 1 d k ∥ θ ⋆ , k ∥ 2 2 } . (49) s_j = \max\left\{\sqrt{\frac{1}{d_j}\lVert\theta_{\star,j}\rVert_2^2},\ 10^{-3}\operatorname{median}_{k\in\mathcal{W}_c}\sqrt{\frac{1}{d_k}\lVert\theta_{\star,k}\rVert_2^2}\right\}. \tag*{(49)} s j = max { d j 1 ∥ θ ⋆ , j ∥ 2 2 , 1 0 − 3 median k ∈ W c d k 1 ∥ θ ⋆ , k ∥ 2 2 } . (49)
with W c \mathcal{W}_c W c the selected weight matrices. The floor gives zero or very small tensors a positive perturbation scale. All scales are fixed before sampling and depend exclusively on the checkpoint parameters. A head restriction selects its slices of W Q W_Q W Q , W K W_K W K , W V W_V W V , W O W_O W O , b Q b_Q b Q , b K b_K b K , b V b_V b V ; the shared attention output bias remains fixed. An MLP restriction selects W i n W_{\mathrm{in}} W in , W o u t W_{\mathrm{out}} W out , b i n b_{\mathrm{in}} b in , b o u t b_{\mathrm{out}} b out .
Let A D \mathcal{A}_D A D be the frozen set of scored closer positions in distribution D D D , and let x a x_a x a denote the token prefix at position a a a . The primary loss is
K c , D ( u ) = 1 ∣ A D ∣ ∑ a ∈ A D KL ( p θ ( ⋅ ∣ x a ) ∥ p θ ( u ) ( ⋅ ∣ x a ) ) , (50) K_{c,D}(u) = \frac{1}{\lvert\mathcal{A}_D\rvert}\sum_{a\in\mathcal{A}_D}\operatorname{KL}\left(p_\theta(\cdot\mid x_a)\mathbin\Vert p_{\theta(u)}(\cdot\mid x_a)\right), \tag*{(50)} K c , D ( u ) = ∣ A D ∣ 1 a ∈ A D ∑ KL ( p θ ( ⋅ ∣ x a ) ∥ p θ ( u ) ( ⋅ ∣ x a ) ) , (50)
where both distributions range over the full vocabulary. Thus K c , D ( 0 ) = 0 K_{c,D}(0)=0 K c , D ( 0 ) = 0 , and the checkpoint is a minimum of the reference loss. Full-vocabulary KL registers changes both in the total probability assigned to brackets and in the relative probabilities of bracket types.
For coordinate scale r > 0 r>0 r > 0 and inverse temperature τ > 0 \tau>0 τ > 0 , define
π c , D ( d u ) = 1 Z c , D ( τ , r ) e − τ K c , D ( u ) N ( d u ; 0 , r 2 I ) , q c , D ( τ , r ) = τ E π c , D K c , D ( u ) . (51) \pi_{c,D}(du)=\frac{1}{Z_{c,D}(\tau,r)}e^{-\tau K_{c,D}(u)}\mathcal{N}(du;0,r^2I),\qquad q_{c,D}(\tau,r)=\tau\mathbb{E}_{\pi_{c,D}}K_{c,D}(u). \tag*{(51)} π c , D ( d u ) = Z c , D ( τ , r ) 1 e − τ K c , D ( u ) N ( d u ; 0 , r 2 I ) , q c , D ( τ , r ) = τ E π c , D K c , D ( u ) . (51)
The scalar r r r is the prior standard deviation of each coordinate: E ∥ u ∥ 2 / d c = r 2 \mathbb{E}\lVert u\rVert^2/d_c=r^2 E ∥ u ∥ 2 / d c = r 2 . The quantity q c , D q_{c,D} q c , D is the LLC estimate of Equation (4) . Equivalently, q c , D = − τ ∂ τ log Z c , D q_{c,D}=-\tau\partial_\tau\log Z_{c,D} q c , D = − τ ∂ τ log Z c , D at fixed r r r , connecting the measured loss response to the local partition function.
Connection to circuit-sensitive directions
The reference KL has zero gradient at u = 0 u=0 u = 0 . Its quadratic expansion is
K c , D ( u ) = 1 2 u ⊤ H c , D u + o ( ∥ u ∥ 2 ) , H c , D = D c ⊤ F D D c ⪰ 0 , (52) K_{c,D}(u)=\frac{1}{2}u^\top H_{c,D}u+o(\lVert u\rVert^2),\qquad H_{c,D}=D_c^\top F_DD_c\succeq0, \tag*{(52)} K c , D ( u ) = 2 1 u ⊤ H c , D u + o (∥ u ∥ 2 ) , H c , D = D c ⊤ F D D c ⪰ 0 , (52)
where F D F_D F D is the categorical Fisher metric pulled back through the model outputs on A D \mathcal{A}_D A D . Consequently, H c , D H_{c,D} H c , D measures the component’s task-sensitive directions in the fixed coordinates of Equation (48) .
For the quadratic model K c , D ( u ) = u ⊤ H c , D u / 2 K_{c,D}(u)=u^\top H_{c,D}u/2 K c , D ( u ) = u ⊤ H c , D u /2 , the posterior is Gaussian with covariance
Σ c , D = ( τ H c , D + r − 2 I ) − 1 . (53) \Sigma_{c,D}=(\tau H_{c,D}+r^{-2}I)^{-1}. \tag*{(53)} Σ c , D = ( τ H c , D + r − 2 I ) − 1 . (53)
Taking the expectation of the quadratic form gives
q c , D ( τ , r ) = 1 2 tr [ τ H c , D ( τ H c , D + r − 2 I ) − 1 ] = 1 2 ∑ i = 1 d c τ r 2 μ i 1 + τ r 2 μ i , (54) \begin{aligned}
q_{c,D}(\tau,r)=\frac{1}{2}\operatorname{tr}\left[\tau H_{c,D}(\tau H_{c,D}+r^{-2}I)^{-1}\right] \\
&=\frac{1}{2}\sum_{i=1}^{d_c}\frac{\tau r^2\mu_i}{1+\tau r^2\mu_i}, \tag*{(54)}
\end{aligned} q c , D ( τ , r ) = 2 1 tr [ τ H c , D ( τ H c , D + r − 2 I ) − 1 ] = 2 1 i = 1 ∑ d c 1 + τ r 2 μ i τ r 2 μ i , (54)
where μ i \mu_i μ i are the eigenvalues of H c , D H_{c,D} H c , D . Each positive-curvature direction contributes between zero and one half, and null directions contribute zero. In this quadratic setting, increasing τ r 2 \tau r^2 τ r 2 makes the response approach rank ( H c , D ) / 2 \operatorname{rank}(H_{c,D})/2 rank ( H c , D ) /2 . At finite localization, the response weights the same task-sensitive directions by their curvature. The scale and temperature therefore specify the geometric measurement, while the proposal parameter below determines how efficiently it is sampled.
Prior-preserving Metropolis sampling
We sample Equation (51) using the preconditioned Crank–Nicolson (pCN) proposal [9 ] . Given the current state u u u , propose
u ′ = 1 − a 2 u + a r ξ , ξ ∼ N ( 0 , I ) , 0 < a ≤ 1 , (55) u'=\sqrt{1-a^2}\,u+ar\xi,\qquad\xi\sim\mathcal{N}(0,I),\qquad0<a\leq1, \tag*{(55)} u ′ = 1 − a 2 u + a r ξ , ξ ∼ N ( 0 , I ) , 0 < a ≤ 1 , (55)
and accept with probability
A ( u , u ′ ) = min { 1 , exp [ − τ ( K c , D ( u ′ ) − K c , D ( u ) ) ] } . (56) A(u,u')=\min\left\{1,\exp\left[-\tau\left(K_{c,D}(u')-K_{c,D}(u)\right)\right]\right\}. \tag*{(56)} A ( u , u ′ ) = min { 1 , exp [ − τ ( K c , D ( u ′ ) − K c , D ( u ) ) ] } . (56)
This transition preserves the specified posterior. To see this, set b = 1 − a 2 b=\sqrt{1-a^2} b = 1 − a 2 . Under the Gaussian prior, the pair ( u , u ′ ) (u,u') ( u , u ′ ) has covariance
r 2 ( I b I b I I ) , (57) r^2\begin{pmatrix} I & bI \\ bI & I \end{pmatrix}, \tag*{(57)} r 2 ( I b I b I I ) , (57)
which is unchanged by exchanging the two states. The proposal is therefore reversible with respect to the prior, and its Gaussian density terms cancel in the Metropolis ratio. Multiplying by the likelihood factor e − τ K c , D e^{-\tau K_{c,D}} e − τ K c , D yields (56) and detailed balance for π c , D \pi_{c,D} π c , D .
Each proposal evaluates the complete frozen empirical loss. Each chain begins with an independent draw from N ( 0 , r 2 I ) \mathcal{N}(0,r^2I) N ( 0 , r 2 I ) ; rejected proposals retain the preceding state and its metrics. We use r = 0.1 r = 0.1 r = 0.1 , τ = 1000 \tau= 1000 τ = 1000 , and a = 0.9 a = 0.9 a = 0.9 , with four chains, 256 burn-in transitions, and 512 retained transitions per chain. Chain c ∈ { 0 , 1 , 2 , 3 } c \in\{0,1,2,3\} c ∈ { 0 , 1 , 2 , 3 } uses seed 20260918 + 100003c c c . The settings were chosen from acceptance and loss-trace diagnostics on 32-sequence pilots of L0H0 and L11.mlp on both distributions, before the component-association analysis.
Matched data, observables, and interventions
The final datasets each contain 128 sequences of length 128. The synthetic and natural-context datasets contain 5,493 and 177 scored closer positions, respectively. Tokens, closer masks, correct and distractor tokens, and same-surface-form closer options are frozen in the dataset archive. Both sampling and MI interventions use these exact arrays and normalize every metric by the number of scored positions.
In addition to full-vocabulary KL, every sampling transition records closer-conditional KL, full-vocabulary cross-entropy, and closer-conditional cross-entropy. The conditional distribution normalizes over the four closing-bracket tokens with the observed target’s surface form. These are observables under the primary full-KL posterior. For any recorded loss f f f , the corresponding scaled response is
R f = τ ( E π c , D f ( θ ( u ) ) − f ( θ ⋆ ) ) . (58) R_f = \tau\left(\mathbb{E}_{\pi_{c,D}} f(\theta(u)) - f(\theta_\star)\right). \tag*{(58)} R f = τ ( E π c , D f ( θ ( u )) − f ( θ ⋆ ) ) . (58)
A separately designated closer-conditional-KL run substitutes that objective into the posterior itself. This distinguishes changes in the measured observable from changes in the distribution over parameter perturbations.
The matched MI panel measures zero and position-mean replacement for each of the 144 components, together with joint first-layer attention and named-subspace interventions. Position means average the clean activations across all input sequences at each position, including padded rows. A head intervention replaces its pre-output-projection activation; linearity of the fixed W O W_O W O makes this equivalent to replacing its projected residual-stream write. The panel records the same four losses together with pairwise closer contrast and full-vocabulary accuracy.
Multichain diagnostics and numerical verification
For each recorded observable, we calculate the maximum of rank-normalized split and folded-split R ^ \hat{R} R ^ , together with mean, bulk, and tail effective sample sizes using ArviZ [23 ] [35 ] . The prespecified target screen requires four chains, R ^ < 1.01 \hat{R} < 1.01 R ^ < 1.01 , and all three effective sample sizes at least 400. Auxiliary observables and the mean squared coordinate have separately reported diagnostics. Mean Monte Carlo standard errors use the effective sample size of the raw observable; the standard error of a scaled response is multiplied by τ \tau τ . These errors quantify simulation uncertainty conditional on the checkpoint and frozen evaluation inputs.
When a target fails this screen, its diagnostic extension uses 1,024 retained transitions with unchanged seeds, burn-in, target, and sampler settings. Replaying the same random-number sequence extends the initial trajectory, and both outputs are retained. Extension decisions use sampling diagnostics. The saved trace places the initial prior draw at index zero; the primary retained window occupies indices 257 through 768, inclusive.
Model computation uses float32 with TF32 disabled, while log probabilities and KL calculations use float64. We compare the selected-position readout with complete model outputs. Computation before the restricted layer is cached, and complete scored log probabilities from cached and full forward passes are checked at the clean and first perturbed states with tolerance 10 − 5 10^{-5} 1 0 − 5 . Every cell records its parameter coordinates, seeds, proposal metrics, retained and rejected states, and diagnostics (Table 15 ).
Table 15. Matched measurements of the circuit. All final-checkpoint comparisons use the same inputs, scored positions, and component definitions. Derivative baselines are measured in both raw parameter coordinates and the pretrained-RMS coordinates of the posterior experiment.
Complete matched geometric and intervention comparisons
The primary comparison uses full-vocabulary reference KL for both the localized posterior and the intervention. We additionally evaluate five output metrics on the same closer positions: KL conditional on the four closer types sharing the target’s tokenization, full-vocabulary target cross-entropy, conditional target cross-entropy, the seeded pairwise closer contrast, and full-vocabulary closer accuracy. Higher importance denotes larger KL, increased cross-entropy, or decreased contrast or accuracy. The posterior response q C q_C q C remains the response under the full-KL target throughout these comparisons.
Figure 17 reports all six metrics under both replacement conventions and both distributions. Each comparison includes the complete component class indicated on its row. These stratified comparisons distinguish within-class correspondence from a difference in scale between attention heads and MLPs. Figure 18 compares the ranking of the same components across distributions, with full-KL interventions as the matched causal reference.
Figure 17. Complete matched measurement comparison. Cells show the Spearman correlation between the full-KL LLC estimate and each intervention score. Positive importance uses increased KL or cross-entropy and decreased contrast or accuracy. All 144 components, 128 heads, and 16 MLPs are reported separately. Zero and position-conditioned mean replacements act on identical components and evaluation positions.
Figure 18. Ranking stability across distributions with matched objectives. The synthetic and natural-context ranks of each component are compared for both raw- and RMS-coordinate LLC estimates and full-KL intervention effects. Blue points are heads and orange points are MLPs. All panels use the same 144 components; ties receive average ranks.
For every condition we record the mean response, Monte Carlo standard error, chain means, convergence diagnostics, and whether the estimate uses 512 draws or a diagnostic extension. Extensions preserve the target, prior, parameter coordinates, chain seeds, and initial transition sequence and are selected by sampling diagnostics alone. Component correlations describe the fixed model and evaluation batches; Monte Carlo errors quantify posterior sampling uncertainty.
Raw-coordinate posterior control
We evaluate all 144 components on both distributions with an isotropic Gaussian prior of absolute standard deviation σ = 0.001 \sigma= 0.001 σ = 0.001 in the converted TransformerLens tensors. Reference KL, inputs, output masks, inverse temperature τ = 1,000 \tau= 1{,}000 τ = 1 , 000 , proposal coefficient, and the four-chain protocol match the RMS experiment. All 288 conditions satisfy the diagnostic criteria: 287 use 512 retained transitions per chain, and L 1 H 0 L1H0 L 1 H 0 on natural contexts uses a 1,024-transition extension that preserves the initial trajectory.
Figure 19 reports the full comparison and both component classes. The raw posterior closely follows the independently measured raw-coordinate model Fisher: Spearman correlations are 0.9992 on synthetic inputs and 0.9986 on natural contexts. The componentwise median ratio to the linear-trace prediction τ σ 2 tr ( F C ) / 2 \tau\sigma^2 \operatorname{tr}(F_C)/2 τ σ 2 tr ( F C ) /2 is 1.004 and 1.007, respectively. Across components, the corresponding fifth-to-ninety-fifth percentile ranges are [0.957, 1.042] and [0.983, 1.069]. These measurements connect the raw-coordinate response to the small-curvature regime of Equation (26) .
Figure 19. Raw-coordinate geometry and causal effects. The isotropic raw prior uses absolute standard deviation 0.001 in the fixed converted parameterization. Each point is one component; blue denotes attention heads and orange MLPs. Insets give Spearman correlations over all components and within each component class. The causal scores and evaluation inputs are identical to Figure 16 .
Raw and RMS posterior responses have component-rank correlations of 0.722 on synthetic inputs and 0.556 on natural contexts. The priors differ in directional weighting and absolute scale while sharing the circuit, behavioral discrepancy, and inputs.
Named circuit variables on the posterior evaluation inputs
We evaluate STACK and ANSWER interventions on the posterior-evaluation datasets (Table 16 ). We remove the rank-24 STACK span in layers 1–5 or the rank-3 ANSWER span from the answer-writing components at every sequence position. Both interventions reduce closer accuracy on synthetic and natural-context inputs. Each batch contains 128 strings of length 128; the circuit-identification and held-out subspace experiments use length-48 strings. These measurements assess the same circuit variables across sequence lengths and input contexts.
Table 16. Full-vocabulary closer accuracy under STACK and ANSWER removal on the posterior-evaluation batches. Subspace bases are fitted on synthetic strings and held fixed across evaluation conditions.
Intervention strength and direction
For every component, we replace its output h C ( x ) h_C(x) h C ( x ) by
h C ( ϵ ) ( x ) = ( 1 − ϵ ) h C ( x ) + ϵ r C ( x ) , ϵ ∈ { 0.01 , 0.03 , 0.1 , 0.3 , 1 } , (59) h_C^{(\epsilon)}(x) = (1-\epsilon)h_C(x) + \epsilon r_C(x), \qquad\epsilon\in\{0.01, 0.03, 0.1, 0.3, 1\}, \tag*{(59)} h C ( ϵ ) ( x ) = ( 1 − ϵ ) h C ( x ) + ϵ r C ( x ) , ϵ ∈ { 0.01 , 0.03 , 0.1 , 0.3 , 1 } , (59)
where r C r_C r C is zero or the position-conditioned mean. Inputs, masks, reference predictions, and replacement means match the full-replacement comparison. We evaluate 2,304 partial-replacement conditions alongside the 576 full-replacement endpoints, recording all six output metrics and per-string numerators.
In the local regime, writing v C = r C − h C v_C = r_C - h_C v C = r C − h C for the stacked activation perturbation and H C a H_C^a H C a for the downstream pullback of the categorical Fisher metric gives
lim ε → 0 K ε ε 2 = 1 2 E D [ v C ( x ) T H C a ( x ) v C ( x ) ] . (60) \lim_{\varepsilon\to0} \frac{K_\varepsilon}{\varepsilon^2} = \frac{1}{2}\mathbb{E}_{D}\left[v_C(x)^{\mathsf{T}} H_C^a(x)v_C(x)\right]. \tag*{(60)} ε → 0 lim ε 2 K ε = 2 1 E D [ v C ( x ) T H C a ( x ) v C ( x ) ] . (60)
This is a directional quadratic form; Equation (26) instead aggregates curvature over parameter directions. The two measurements are distinct contractions of a shared behavioral metric.
The measured scaling is quadratic: between 0.01 and 0.03, the median fitted power is 2.000–2.001 across the four distribution/replacement combinations, and median changes in K ϵ / ϵ 2 K_{\epsilon}/\epsilon^2 K ϵ / ϵ 2 are 0.09–0.21%. Figure 20 shows that synthetic rank correspondence persists across strengths, while natural-context head associations remain negative. The latter difference is already present in the local directional response.
Figure 20. Local and full-strength interventions probe different directions from the posterior average. Each curve compares the same fixed LLC estimates with reference-KL effects along zero- or mean-replacement directions. All components and both component classes are included at every strength. Dividing KL by the fixed ϵ 2 \epsilon^2 ϵ 2 leaves the displayed rank correlation unchanged.
Posterior perturbations in semantic circuit subspaces
We measure the contribution of circuit subspaces to parameter-induced output changes using a fixed subset of RMS-posterior states. For L11.mlp and L14H1 on each distribution, we select retained indices 128 and 384 from each of four chains, giving eight states per condition. Inputs are identical to those used for posterior sampling. Each state is reconstructed from its random-number stream and acceptance decisions; the maximum difference between replayed and stored full KL is 5.32 × 10 − 9 5.32 \times10^{-9} 5.32 × 1 0 − 9 .
Let o C ( x ; u ) o_C(x;u) o C ( x ; u ) be the output of component C C C under a sampled parameter state u u u , and define
Δ o C ( x ; u ) = o C ( x ; u ) − o C ( x ; 0 ) . (61) \Delta o_C(x;u) = o_C(x;u) - o_C(x;0). \tag*{(61)} Δ o C ( x ; u ) = o C ( x ; u ) − o C ( x ; 0 ) . (61)
After restoring the checkpoint parameters, we patch the clean output with Δ o C \Delta o_C Δ o C , its projection P C Δ o C P_C\Delta o_C P C Δ o C onto the exact named span, its complement ( I − P C ) Δ o C (I-P_C)\Delta o_C ( I − P C ) Δ o C , its ANSWER projection, or projections onto two fixed random spans matching the named rank. Interventions act at all positions in each evaluated sequence. The named spans have rank 166 for L11.mlp and 94 for L14H1; ANSWER has rank 3. The random seeds are 8201 and 8202.
The full activation patch verifies the connection between parameter and activation interventions. It reproduces every scored log probability exactly for the MLP, and within 2.14 × 10 − 5 2.14 \times10^{-5} 2.14 × 1 0 − 5 for the head. The latter difference comes from the equivalent per-head and summed-output floating-point computations (Figure 21 ).
Figure 21. Projecting posterior-induced changes onto circuit variables. Bars pool KL over eight fixed states per cell. Whiskers show the range of per-state ratios; random bars average the two fixed subspaces, with whiskers spanning both. Projection and complement losses have signed cross effects, so their ratios are nonadditive and may exceed one. These states sample one fixed evaluation batch per distribution.
For each projection, Table 17 reports the sum of patch KL across the eight states divided by the sum of full parameter-perturbation KL. The strongest concentration occurs in the synthetic L14H1 readout: the three-dimensional ANSWER space alone yields ratio 0.668, compared with 0.032–0.037 for random 94-dimensional spans. The broad named span also exceeds both random controls for the MLP and on natural-context inputs. These comparisons link the localized response to specific represented variables, while retaining the different degrees of concentration across components and distributions.
Table 17. Pooled reference-KL ratios over eight fixed posterior states per cell. Named spans have rank 166 for L11.mlp and 94 for L14H1; each random control matches that named rank. ANSWER has rank 3. The numerator sums projected-patch KL across states and the denominator sums full parameter-perturbation KL.
We retain the signed interaction
K ( Δ o C ) − K ( P C Δ o C ) − K ( ( I − P C ) Δ o C ) (62) K(\Delta o_C) - K(P_C\Delta o_C) - K((I-P_C)\Delta o_C) \tag*{(62)} K ( Δ o C ) − K ( P C Δ o C ) − K (( I − P C ) Δ o C ) (62)
for every state. It captures cross terms and higher-order effects of the downstream computation.
Sampler sensitivity and joint circuit geometry
A fixed sensitivity panel
The sensitivity analysis includes 18 components: all eight first-layer heads, L0.mlp, L11.mlp, L14H1, L7H4, L9H2, L15.mlp, and four randomly selected comparators (L11H0, L5H7, L2H4, and L15H1). The components and comparator seed are fixed before measurement. The 288 conditions comprise checkpoints 8,000 and 32,000, four factor-two radius or inverse-temperature changes, a proposal-coefficient control, and a closer-conditional-KL target, each evaluated on both distributions.
Radius and inverse temperature define the posterior; the proposal coefficient controls the transition kernel. The closer-conditional target normalizes over the four closer tokens with the observed surface form. All conditions use the four-chain diagnostic criteria of the posterior experiment. Runs that fail at 512 draws are extended with the same seeds, preserving the initial acceptance sequence and metric trace. Of the 288 conditions, 246 meet the criteria at 512 retained draws, 37 at 1,024, and three at 2,048. The two wide-radius L15.mlp conditions use a proposal coefficient selected by the minimum effective samples per transition across distributions and meet the criteria with 4,096 retained draws.
Radius and inverse-temperature controls preserve component ranks (ρ = 0.975 \rho= 0.975 ρ = 0.975 –1.000 1.000 1.000 ); their median response ratios differ from the weak-localization prediction q ∝ τ r 2 q \propto\tau r^2 q ∝ τ r 2 by at most 3.6%. Changing proposal coefficient from 0.9 to 0.5 yields rank correlations 0.998 and 0.996 on synthetic and natural-context inputs, respectively. All 36 paired differences lie within three conservative MCSE bounds, computed by summing the two marginal MCSEs to allow shared-seed correlation. The closer-conditional target yields rank correlations 0.926 and 0.810, with median response ratios 0.683 and 5.91 × 10 − 4 5.91 \times10^{-4} 5.91 × 1 0 − 4 . Restricting the prediction to relative bracket identity therefore changes the measured geometry substantially on natural-context inputs.
Joint geometry and single components
The bank restriction varies all eight first-layer heads together using the exact product of their atomic RMS priors. Both its parameter values and each member’s scales are matched to the atomic runs. Write u = ( u 1 , … , u 8 ) u = (u_1,\ldots,u_8) u = ( u 1 , … , u 8 ) , K h ( u h ) = K ( 0 , … , u h , … , 0 ) K_h(u_h) = K(0,\ldots,u_h,\ldots,0) K h ( u h ) = K ( 0 , … , u h , … , 0 ) , and
Z b a n k ( τ ) = ∫ e − τ K ( u ) ∏ h φ h ( u h ) d u , q b a n k = − τ ∂ τ log Z b a n k . (63) Z_{\mathrm{bank}}(\tau) = \int e^{-\tau K(u)} \prod_h \varphi_h(u_h)\,\mathrm{d}u,\qquad q_{\mathrm{bank}} = -\tau\partial_\tau\log Z_{\mathrm{bank}}. \tag*{(63)} Z bank ( τ ) = ∫ e − τ K ( u ) h ∏ φ h ( u h ) d u , q bank = − τ ∂ τ log Z bank . (63)
If K ( u ) = ∑ h K h ( u h ) K(u) = \sum_h K_h(u_h) K ( u ) = ∑ h K h ( u h ) , the partition function factorizes, yielding
q b a n k = ∑ h q h . (64) q_{\mathrm{bank}} = \sum_h q_h. \tag*{(64)} q bank = h ∑ q h . (64)
A joint-versus-atomic difference consequently measures a departure from this separable local-loss model.
For the quadratic reference loss, Equation (17) strengthens this comparison to q b a n k ≤ ∑ h q h q_{\mathrm{bank}} \le\sum_h q_h q bank ≤ ∑ h q h , with strict inequality for cross-head curvature. This supplies a directional prediction for coupled task-sensitive parameter directions under the matched product prior (Figure 22 ) (Figure 23 ).
Figure 22. Joint and summed single-head LLC estimates on the same frozen inputs and product prior. Error bars show individual bank MCSEs and conservative summed-head MCSE bounds. The matched reference-KL response increases across the three checkpoints on both distributions.
Figure 23. Sensitivity-panel rank correlations with the primary LLC estimate. The first four controls alter the posterior; the proposal control preserves it. The closer-conditional objective changes the readout. Every rank comparison includes the complete fixed 18-component panel.
Monte Carlo uncertainty with shared seeds
The head runs and bank run reuse the same chain-seed convention. We therefore propagate uncertainty conservatively: for S = ∑ h q h S = \sum_h q_h S = ∑ h q h , we report s S = ∑ h s h s_S = \sum_h s_h s S = ∑ h s h , and for Δ = q b a n k − S \Delta= q_{\mathrm{bank}} - S Δ = q bank − S we use s Δ = s b a n k + s S s_\Delta= s_{\mathrm{bank}} + s_S s Δ = s bank + s S . These upper bounds on the propagated standard error allow arbitrary cross-run correlation. The ratio uses the first-order bound s q / S = s b a n k / S + q b a n k s S / S 2 s_{q/S} = s_{\mathrm{bank}}/S + q_{\mathrm{bank}}s_S/S^2 s q / S = s bank / S + q bank s S / S 2 . Ratios and signed differences are both retained.
Detailed developmental and group comparisons
The developmental sweep evaluates 49,588 component-, layer-, and model-restricted SGLD responses across both synthetic task variants and all 154 checkpoints. These measurements perturb the weights at the larger scale characterized in Appendix K. The statistic averages the full finite sampling trajectory; Appendix I gives the protocol, and Appendix J compares averaging windows. Across the 12 checkpoints measured on both input distributions, finite SGLD response component rankings have median Spearman correlation 0.89 between synthetic and natural-context data, compared with 0.35 for signed zero-ablation effects and 0.37 for their magnitudes. The difference is strongest among attention heads: the corresponding finite SGLD response and signed-ablation correlations are 0.88 and 0.27, compared with 0.87 and 0.81 among MLPs. These paired observations
caption: Matched joint-bank and atomic-head measurements. The head-sum and difference uncertainties use sums of marginal MCSEs, allowing arbitrary correlation from shared seeds. The ratio is joint response divided by the atomic sum.
:::
Table 19. Qualified sensitivity comparisons on all 18 prespecified components. q 0 q_0 q 0 is the primary r = 0.1 r = 0.1 r = 0.1 , τ = 1000 \tau= 1000 τ = 1000 full-KL response at checkpoint 143,000. Radius and inverse temperature alter the posterior, proposal coefficient preserves it, and closer KL changes the readout.
establish substantial stability of the perturbation-response ranking across the two bracket distributions.
We also compare the k = 4 k = 4 k = 4 layer-2 prefix-state distance trajectory with zero-ablation effects and gradient activity (Table 14 ): the finite SGLD response’s association (0.75, and 0.60 after removing the linear checkpoint-rank trend) persists where the ablation and gradient baselines’ associations (0.15 and 0.27) fall to zero after the same adjustment.
Table 14. Tracking feature development across all 154 checkpoints. Median within-component association with the fixed k = 4 k = 4 k = 4 layer-2 prefix-state distance probe. Rank-time adjustment removes the linear checkpoint-rank trend. Adjacent changes use log Fisher and gradient values. Component measurements cover the same 144 components; clean loss is one shared task-level series. The finite SGLD response uses the nonlocal perturbation regime in Appendix § 18 .
This correspondence is stable under changes to the averaging window. On matched observations, all-step and retained-only component rankings have median correlation 0.994; their rank-time-adjusted feature associations are 0.589 and 0.615. Excluding the initial sampling updates therefore preserves the developmental association (Appendix § 17 ).
A componentwise intervention comparison provides a further connection to the represented variable. We zero-ablate each of 27 early components and refit the distance probe to measure the remaining recoverable information. Across 12 checkpoints, median componentwise correlations with the resulting loss of probe R 2 R^2 R 2 are 0.25 for finite SGLD response and 0.28 for contrast-based ablation; at the final checkpoint they are 0.59 and 0.45. Both measurements therefore carry information about the components supporting this representation, with particularly clear correspondence at the trained checkpoint.
Probe trajectories. On the k = 2 k = 2 k = 2 strings the held-out R 2 R^2 R 2 of the layer-2 prefix-depth probe is 0.864 at initialization, whereas that of the layer-2 open-distance probe rises from 0.034 at initialization to 0.558 at step 2,000 and 0.684 at the final checkpoint (on k = 4 k = 4 k = 4 strings, 0.226, 0.637, and 0.589); Figure 25 A shows the full trajectories.
Figure 25. Feature formation and changing component roles across all 154 checkpoints. A: held-out layer-2 prefix-state probes. B: full-model finite SGLD response. C–D: the same three layer-7 heads on k = 2 k = 2 k = 2 , measured by finite SGLD and signed zero-ablation. The horizontal axis is logarithmic above step 128; shading marks steps 256–2,000. The finite SGLD response uses the perturbation regime characterized in Appendix K.
A distributed attention bank relates parameter responses to group effects
All eight first-layer heads rank among the top 23 of 144 components by synthetic finite SGLD response at the final checkpoint. Joint interventions establish the causal contribution of this group. In the cross-distribution intervention batch, zeroing the bank lowers closer contrast by 1.78 on synthetic data and 2.28 on natural-context data. Position-conditioned mean replacement gives drops of 0.14 and 1.69, respectively. Relative to clean contrasts of 3.56 and 13.16, the mean-replacement drops are 3.9% and 12.8%. This comparison uses 11,882 scored synthetic positions and 187 natural-context positions.
The contrast between replacement rules reveals how the bank contributes to the task. A position-conditioned mean retains the average write at each position; zero replacement removes the entire write. Their different effects show that much of the bank’s contribution on synthetic inputs survives position-wise averaging, whereas natural-context completion depends more strongly on its input-specific output. The perturbation-response ranking identifies the bank as a candidate group, and the intervention battery resolves how that group supports the computation.
Late perturbation-response changes accompany computational reorganization
In the first-layer bank, the late change of the nonlocal SGLD response accompanies a marked shift in the relationship between joint and marginal causal effects. At step 32,000, joint zero-ablation lowers contrast by 2.80, while the signed sum of the eight individual drops is 0.078. Their absolute sum is also small, 0.462, establishing substantial nonadditivity beyond cancellation of positive and negative effects. At the final checkpoint, the joint and marginal effects are much closer: 1.56 and 1.33, respectively.
The 128-string evaluation across all checkpoints shows the same change in joint and marginal contributions (Figure 24 ). Clean contrast is 3.55 at step 32,000 and 3.54 at step 143,000. Full-vocabulary accuracy fluctuates across the intervening checkpoints. The joint interventions distinguish changes in the allocation of causal contributions despite the similar endpoint contrasts (Figure 27 ).
Figure 24. Circuit development and collective organization. A: layer-2 state-feature probes. B: causal effects of three layer-7 heads on k = 2 k = 2 k = 2 . C: local RMS-posterior responses of the first-layer bank and the sum of its heads at three checkpoints; whiskers show Monte Carlo standard errors, with conservative bounds for the sum. D: joint and marginal bank ablations on k = 4 k = 4 k = 4 . A and B include all 154 checkpoints; D shows training from step 8,000 onward. A, B, and D use the developmental batches; C uses the fixed posterior calibration batch.
Figure 27. Finite perturbation responses and late circuit development. A: the eight first-layer finite SGLD trajectories on k = 4 k = 4 k = 4 . B: joint bank ablation and signed and absolute sums of individual effects. C: clean closer contrast. All checkpoints from step 8,000 onward are included; the SGLD regime is characterized in Appendix § 18 .
Algorithmic structure and causal verification
A distributed realization of bracket matching
Let σ t \sigma_t σ t denote the stack before token x t x_t x t is consumed. The reference computation pushes the type of an opener, pops the last unmatched type on a closer, and predicts close(top(σ t + 1 \sigma_{t+1} σ t + 1 )) whenever the next token is a closer. This formulation separates the represented prefix state from the output decision. In Pythia, the prefix state is distributed across attention and MLP outputs, and several later components write the corresponding closer evidence into the residual stream.
Writing h ℓ , t h_{\ell,t} h ℓ , t for the residual stream entering layer ℓ \ell ℓ , a parallel Pythia block has the form
h ℓ + 1 , t = h ℓ , t + ∑ n a ℓ n , t ( h ℓ , ≤ t ) + m ℓ , t ( h ℓ , t ) . (65) h_{\ell+1,t} = h_{\ell,t} + \sum_{n} a_{\ell n,t}(h_{\ell,\leq t}) + m_{\ell,t}(h_{\ell,t}). \tag*{(65)} h ℓ + 1 , t = h ℓ , t + n ∑ a ℓ n , t ( h ℓ , ≤ t ) + m ℓ , t ( h ℓ , t ) . (65)
At layer zero, the MLP input depends only on the token embedding, so m 0 , t = v x t m_{0,t} = v_{x_t} m 0 , t = v x t is exactly a token lookup. First-layer attention acts in parallel, supplying prefix information. The early stack-writing group consists of L1-L5.mlp, L1H1, L2H1, and L2H5. The selected attention readouts are
H r e a d = { L 7 H 0 , L 7 H 2 , L 7 H 4 , L 7 H 6 , L 8 H 1 , L 8 H 4 , L 9 H 0 , L 9 H 4 , L 9 H 5 , L 13 H 0 , L 14 H 1 } . \mathcal{H}_{\mathrm{read}} = \{\mathrm{L7H0}, \mathrm{L7H2}, \mathrm{L7H4}, \mathrm{L7H6}, \mathrm{L8H1}, \mathrm{L8H4}, \mathrm{L9H0}, \mathrm{L9H4}, \mathrm{L9H5}, \mathrm{L13H0}, \mathrm{L14H1}\}. H read = { L7H0 , L7H2 , L7H4 , L7H6 , L8H1 , L8H4 , L9H0 , L9H4 , L9H5 , L13H0 , L14H1 } .
Together with L11.mlp, these heads provide positive closer evidence at several depths. L13-L15.mlp contribute negative answer-direction effects on average. Their signed contributions explain why removing a component can improve the task score even when that component participates in the same computation.
Subspace interventions
For an orthonormal basis U C U_C U C , projection replaces a component output o C o_C o C by U C U C T o C U_C U_C^{\mathsf T} o_C U C U C T o C ; removal replaces it by ( I − U C U C T ) o C (I - U_C U_C^{\mathsf T})o_C ( I − U C U C T ) o C . The broad named space spans TOKEN, STACK, POSITION, and ANSWER directions. TOKEN is obtained from first-layer token-conditioned MLP outputs; STACK from additive slot/type fits; POSITION from position-conditioned residual means; and ANSWER from closer unembedding directions. The broad STACK span combines the fitted spaces at several neighboring layers, allowing the representation to rotate as computation progresses.
All named spans are constructed by rank-revealing SVD with relative tolerance 10 − 6 10^{-6} 1 0 − 6 . The six-slot, four-type additive stack code has rank 24; the centered closer-answer space has rank 3. TOKEN has rank 11 and POSITION retains 32 principal directions. Their narrow union has rank 38, while the broad union spans 70–190 directions depending on the number of subsequent layers included. Augmentation adds 64 principal directions of each component’s residual output covariance, fitted on the same 256 seed-42 strings.
The cumulative replacement acts on 10 MLPs and 14 attention heads, with the eight first-layer heads additionally replaced by position-conditioned means. All other components execute normally. The held-out recovery in Table 23 measures the behavior retained by this representational bottleneck in the complete network. The named variables retain 76.3% of the closer contrast; adding 64 orthogonal principal directions increases this to 91.9% (Figure 29 ).
Figure 29. Effects of stack- and answer-subspace removal. Interventions are evaluated on 384 held-out strings. Left: retained pairwise closer contrast. Right: loss of full-vocabulary closer accuracy. Error bars for the identified subspaces are 95% paired string-bootstrap intervals; random-control bars summarize five fixed rank-matched bases and their range.
Table 23. Clean pairwise contrast retained under cumulative replacement. Values are means and sample standard deviations over three held-out batches of 128 strings. Bases are fitted on 256 seed-42 strings; the seed-42 column evaluates 128 strings. The PCA control uses 128 directions per MLP and 64 per head.
Held-out causal validation. Rank-revealing SVD constructs the 24-dimensional STACK and three-dimensional ANSWER spaces from the frozen seed-42 fits, using a relative singular-value tolerance of 10 − 6 10^{-6} 1 0 − 6 . We evaluate all interventions on the same 384 strings (128 each from seeds 43–45). The model, strings, distractors, and scored positions are identical across conditions. The early group removes STACK from five MLPs and three heads; the answer-writing group removes
ANSWER from 11 heads and four MLPs. For each joint intervention, five independently sampled random subspaces match the saved basis rank at every intervened layer. All measurements use fp32 at step 143,000.
The matched controls establish that the named directions carry task-specific information (Table 22 ). Joint STACK removal reduces closer accuracy from 61.4% to 14.8%, whereas random removal retains 58.4–62.6%. ANSWER removal reduces accuracy to 39.3%, compared with 60.4–61.8% for its random controls. The agreement between contrast and full-vocabulary accuracy connects the recovered stack representation to the model’s output decisions.
Table 22. Held-out removal of algorithmic subspaces. Recovery is the intervened pairwise contrast divided by the clean contrast. Confidence intervals use 2,000 paired string bootstraps stratified by evaluation seed. Accuracy changes are for full-vocabulary argmax at closer targets, in percentage points; negative values indicate a decrease. Random-control rows give the range over five frozen random bases, separately from sampling uncertainty.
Geometry of the represented stack variables
The stack directions expose structure below the component level. For each layer, an additive fit represents the stack-dependent part of the residual as
h ^ ℓ , t = b ℓ + ∑ i = 1 d t ( o ℓ i + e ℓ i , σ t [ i ] ) , ∑ a = 1 4 e ℓ i , a = 0. (66) \hat{h}_{\ell,t} = b_{\ell} + \sum_{i=1}^{d_t} \left(o_{\ell i} + e_{\ell i,\sigma_t[i]}\right), \qquad\sum_{a=1}^{4} e_{\ell i,a} = 0. \tag*{(66)} h ^ ℓ , t = b ℓ + i = 1 ∑ d t ( o ℓ i + e ℓ i , σ t [ i ] ) , a = 1 ∑ 4 e ℓ i , a = 0. (66)
Here o ℓ i o_{\ell i} o ℓ i denotes slot occupancy and e ℓ i , a e_{\ell i,a} e ℓ i , a the type offset at that slot. This separates how much of the stack is occupied from which bracket type occupies it.
A separate fully nested dataset covers depths 1–15. At layer 10, the top four singular directions capture 90.0% of the squared norm of the fitted occupancy coefficients. The four type offsets at each slot form a centered tetrahedral configuration. After orthogonal Procrustes alignment, every slot has similarity at least 0.948 to the first slot; the type-offset Frobenius norm at slot 15 is 6.15 times that at slot 1. Thus the fitted type configurations retain a similar shape across slots while their scale changes substantially. The combined occupancy/type coefficient matrix requires 10 singular directions to capture 95% of its squared norm (Figure 30 ).
Figure 30. Structure within the fitted stack code. Left: the norm of centered type coefficients by slot in a separate fully nested dataset. Right: cumulative squared singular values of the layer-10 type and occupancy coefficient matrices. These spectra describe the stack-dependent additive fit; the fits account for 8.1%, 8.8%, and 13.1% of total residual variance at layers 5, 10, and 15, respectively. Slots are counted from the bottom.
The geometry identifies reusable variables for interventions. The held-out projection and removal experiments then establish how those variables support behavior: compact stack-dependent directions can matter substantially for bracket prediction even when most residual variance carries other information.
Coverage of the mechanistic reference
Adding position and multilayer stack directions increases held-out recovery from 0.449 to 0.763; the removal experiments above test the causal contribution of these variables.
A matched held-out stack-decoding control
We additionally evaluate a frozen linear decoder of stack contents at layer 5, using independently generated strings for probe fitting and assessment. We fit four logistic-regression probes on 128 strings from seed 42 and evaluate each on three disjoint batches of 128 strings (seeds 43–45). Rank-revealing SVD constructs the rank-24 stack spaces at layers 1–5 from the seed-42 coefficient fits. Every clean and perturbed score uses exactly the same held-out strings and positions. The probes and subspaces remain fixed throughout the intervention battery.
The model is Pythia-1B at checkpoint 143,000, evaluated in fp32 on Dyck-(4, 16) with opening probability 0.45 and maximum bracket length 48. For slot i i i , the target is the bracket type at the i i i th stack position counting from the bottom, immediately before the current token is consumed; we score all real positions where that slot exists. The current token remains visible to the model. These scores quantify stack-state decodability, separately from the next-closer task metric. We compare the clean residual, removal of each layer’s stack subspace from the outputs of L1–L5.mlp or L1–L3.mlp, and removal of an independently drawn random orthonormal subspace of the same rank at each of L1–L5.mlp (one fixed random seed). All four probes converge before the 1,000-iteration cap (Table 24).
Removing the selected stack directions at L1–L5 lowers held-out decoding accuracy by 19.8–34.1 percentage points, whereas the rank-matched random control lowers it by 0.1–2.1 percentage points. This supports the information-bearing role of the selected directions under the stated intervention and decoder. The behavioral contribution of these directions is tested separately through the held-out output interventions in Appendix Q. The three evaluation batches quantify variation across input strings with one fixed basis fit and one random-control draw.
Table 24. Matched held-out stack-decoding accuracy. Entries are means ± sample standard deviations across three independently generated batches. The pooled position counts for slots 1–4 are 7,605, 4,769, 2,869, and 1,690. Every entry in a row uses the same examples.
Matching-opener retrieval: content and position
The late readout head L14H1 combines content and positional information. Its 256-dimensional query/key space contains 64 rotary coordinates and 192 unrotated coordinates. Recomputing attention from the two score contributions separately shows that the unrotated coordinates retain most of the observed matching-opener localization. Averaged equally over matching-distance bins 2–24, the attention argmax coincides with the matching opener in 17.57% of clean cases, 15.27% with unrotated coordinates alone, and 0.041% with rotated coordinates alone. This diagnostic uses 256 k = 4 k = 4 k = 4 strings generated with seed 30, maximum bracket length 64, and token windows of length 80.
A complementary key intervention holds the positional phase fixed. At the matching-opener position, we substitute a key from another sequence with a different opener type at that same position. Across 6,212 query–donor pairs, mean attention at the matching opener falls from 0.1833 to 0.0632, a 65.5% reduction. The matched position fixes the rotary phase while the donor changes key content. Together, the score decomposition and key intervention identify content-dependent retrieval as an important part of the head’s computation. The head’s answer-direction intervention separately establishes how its output contributes to bracket prediction.
Redistribution of a computational role during training
The mechanistic trace resolves changes in which components support a behavior. Table 25 follows three layer-7 heads on the k = 2 k = 2 k = 2 task. L7H4 contributes early, L7H5 is prominent at step 3,000, and L7H2 retains a positive contribution at the final checkpoint. By step 32,000, removing L7H4 or L7H5 improves the contrast. These signed trajectories distinguish a component’s represented information from the way its output is used by the surrounding network.
Table 25. A changing allocation of the bracket computation. Entries are the drop in pairwise closer contrast under individual zero-ablation in the saved k = 2 k = 2 k = 2 trajectory. Positive values indicate a contribution to the score, and negative values indicate an opposing effect.
This developmental account supplies a circuit-level reference for the geometric trajectories: both the represented variables and the components carrying their causal effects evolve during pretraining.
Consolidation in the greedy circuit trace. The k = 2 k = 2 k = 2 greedy search first reaches 80% clean-contrast recovery with 24, 22, 16, 17, and 13 components at steps 1,000, 4,000, 16,000, 32,000, and 143,000, respectively. The threshold size varies across training and is smaller at convergence. L14H1 moves from eighteenth in the addition order at step 1,000 to second at the three intermediate checkpoints and fifth at convergence.