WorksComputer Science
Same Function, Different Mechanism: Implementation-Local Geometry Beyond Accuracy
Abstract
Two models can make exactly the same predictions yet respond differently when their implementations are perturbed. A lookup table can change one case independently; a shared rule couples changes across cases. We compare these mechanisms through an implementation chart: a specified local family of logit perturbations around a fixed predictive distribution. Our main result counts the independent prediction changes allowed by exact finite-group sharing: the active dimension is the number of input-output orbits minus the number of input orbits. For softened modular addition, lookup and shared cyclic charts therefore have ranks and despite identical predictions. The classical Fisher pullback connects these dimensions to local KL curvature; mechanism-preserving reparameterizations leave the rank unchanged. We also bound the spectral tail under approximate sharing and separate quadratic rank from higher-order learning coefficients. Trained modular-addition models verify that chart rank is measurable, while controlled compiled transformers distinguish compiler-supported intermediate charts from matched alternatives. Negative controls show where this discrimination fails. The framework evaluates specified mechanism hypotheses; it does not infer a unique mechanism from predictions or predict training dynamics.
Introduction
Suppose two systems answer every modular-addition question correctly, with exactly the same confidence. One stores a separate answer for each input pair. The other reuses one cyclic rule. Their predictions cannot distinguish them, but an intervention can: changing one stored answer need not affect another, whereas changing the shared rule affects many inputs together. This paper asks how to measure that difference while holding the predictive distribution fixed.
The key quantity is the number of independent local changes to predictions that a specified mechanism permits. We call this its active dimension. To define it, we need more than the model’s outputs: we must say which parts of the implementation can change and which constraints remain in place. An implementation chart records that choice. For example, perturbing a convolutional kernel while preserving weight tying defines a different chart from independently perturbing its copies at every position. Both charts can pass through the same predictive distribution.
A running example. Let . Think of the output logits as a cyclic array indexed by . A shared implementation reads one kernel at the shifted position ; an unrestricted lookup implementation stores a separate array for each . Adding the same constant to all logits does not change softmax probabilities, so each array has meaningful directions. Lookup permits independent changes; sharing permits only . Sections 2–6 formalize this example and explain why the gap survives coordinate changes and redundant parameters.
Preprint.
Main contributions.
An orbit-count formula for shared mechanisms. Theorem 5.2 computes active dimension from group orbits, without a coordinate-level Hessian calculation. Modular addition gives an unbounded factor- gap at the same predictive distribution.
A precise comparison of local mechanism hypotheses. Charts with the same tangent image represent the same first-order perturbation family. Reparameterization, dead coordinates, and duplication preserve its rank (Proposition 4.2); changing the allowed family changes the question. The Fisher pullback used to measure rank is classical background (Lemma 3.1).
An operational test and its boundaries. We compute chart ranks on trained networks and compare intervention-defined charts in compiled transformers (Section 7). A spectral-tail bound handles approximate sharing, and explicit higher-order examples show why rank alone is not a general learning-coefficient estimator.
Why local geometry? Singular learning theory (SLT) associates Bayesian complexity with a local statistical model, not just its predictions [Watanabe, 2009, 2013, 2018]. Work on neural local learning coefficients (LLCs) and program singularities motivates comparing implementations at a common predictive distribution [13] (Wei et al. 2023; Lau et al. 2025; Wang et al. 2024; Murfet and Troiani 2025). We isolate the part visible to second-order geometry and state where higher-order terms matter. Neither an unrestricted Hessian nor the results below determine which algorithm a network learned, or predict grokking and other training transitions [Power et al., 2022, Nanda et al., 2023]. Chart specification requires architectural, compiler, or intervention evidence.
Finite-output mechanisms and logit geometry
We first fix the predictions, then specify their allowed perturbations.
Let be a finite input domain with and let be a finite label set. Let be a full-support target conditional distribution for each . Logits are identifiable only up to additive constants, so define the quotient logit space
The quotient removes one unobservable offset per input. In the running example, it leaves logit contrasts for each of the input pairs.
The softmax map is well-defined on . Write and . For , define the population loss
Then and represents the predictive distribution, not a particular neural parameterization.
Definition 2.1 (Implementation chart). An implementation chart is a map with and . The index denotes the intended mechanism: for example, independent lookup entries, a shared rule, a low-rank factorization, a circuit subspace, or a compiler-specified program. The resulting local loss (its behavior in a neighborhood of , also called a germ) is
Two charts are extensionally equivalent at the base point if they have the same ; they may still have different local geometry because their perturbation families differ.
For Hessian-rank statements, the assumption is sufficient. Whenever we discuss an actual real log canonical threshold or local learning coefficient, we additionally assume the relevant chart germ is analytic and the prior density is positive and smooth in a neighborhood of the minimizing set.
For the cyclic example, the lookup coordinates change the input-specific arrays independently, while shared coordinates change the common kernel. At both give the same .
This definition intentionally separates the predictive distribution from the local family of alternatives. A neural network restricted to a finite evaluation domain induces such a chart by composing parameter perturbations with the logits on that domain. A hand-written algorithmic or architectural mechanism induces a chart directly by specifying which perturbations are allowed.
Measuring active dimension: the Fisher pullback
For each , define the Fisher covariance on logits
It annihilates the gauge direction and is positive definite on . Let be the block-diagonal positive-definite bilinear form on given by
where representatives in may be chosen arbitrarily because .
Lemma 3.1 (Classical logit-Fisher pullback). Let be an implementation chart and let . Then
Consequently,
On the active quotient , the quadratic germ is regular and has quadratic active contribution
If is analytic and its tangent-kernel directions split off as genuinely dead local coordinates, this quadratic contribution is the local learning coefficient under a positive smooth prior. Restriction to an active transverse slice also gives , but describes that restricted chart rather than the full germ. Otherwise (8) is only the second-order active contribution; higher-order protected directions in may change the actual learning coefficient.
How to read the lemma. The Jacobian converts parameter changes into logit changes. Multiplication by weights those changes by predictive uncertainty and input probability. Its rank counts independent local prediction changes; its squared singular values give the nonzero eigenvalues of the pullback. In the lookup example, different inputs can move independently. In the shared example, all inputs reuse the same directions. This is the standard Fisher chain rule applied to a specified mechanism chart, not a new identity.
Proof sketch. For one input , the gradient of with respect to logits is , which vanishes at . The Hessian at is . Summing over inputs gives the ambient Hessian . The chain rule gives because the first derivative of vanishes at . Since is positive definite on the quotient logit space, the rank of the pullback is exactly . A rank- positive quadratic form has regular real log canonical threshold on its active coordinates. Appendix A gives the quotient-space details. □
Corollary 3.2 (No function-only active dimension). Fix any full-support finite conditional distribution . For any linear subspace , the affine chart with implements at and has active rank . Thus the local active dimension is not determined by the conditional distribution alone.
Corollary 3.2 is the formal version of the slogan “same function, different mechanism.” It also explains why a chart condition is unavoidable: Bayesian evidence and SLT attach complexity to a local statistical model, not only to a point distribution.
What chart dependence is allowed?
The numerical curvature of a loss depends on coordinates. Its rank at a stationary point survives invertible smooth coordinate changes. Here we ask a broader question: which changes preserve the same family of first-order prediction changes?
Definition 4.1 (First-order mechanism equivalence). Two implementation charts and at the same are first-order mechanism equivalent if their tangent images in agree:
They represent the same local perturbation mechanism to first order, although they may use different coordinates or redundant parameters.
Proposition 4.2 (Invariance under reparameterization, dead code, and duplication). Let be , , and suppose is surjective. Then the composed chart has the same active rank as :
This includes local diffeomorphisms, adding dead coordinates , and duplicated coordinates such as . If the charts and are analytic, the prior has positive smooth product density, and the local fibers have finite volume, then the pullback germ has the same learning coefficient as the original germ. In the dead and duplicated examples the extra coordinates split off as inert fibers after a local change of variables.
Proof sketch. The tangent image of is by surjectivity. The rank statement follows from Lemma 3.1. Under the analytic and prior assumptions, the learning-coefficient statement follows from the constant-rank theorem: locally, the redundant coordinates split into active coordinates and inert fibers. See Appendix B.
Remark 4.3 (Changing the perturbation class changes the question). Let be two mechanism subspaces and consider the charts and . They implement the same distribution at the base point but have active ranks and . In particular, a shared implementation embedded as a point in a larger lookup model has lookup-rank local geometry if the prior permits arbitrary lookup perturbations, and shared-rank local geometry if the prior is concentrated on the shared submodel.
Remark 4.3 identifies a modeling distinction rather than a coordinate artifact. The posterior preference is over a local model class or prior support; the same parameter value can be evaluated under a broad ambient model or under a mechanism-preserving submodel.
Shared mechanisms from finite-group structure
Many mechanisms reuse computation because a symmetry identifies perturbations across examples. An orbit is a set of entries related by that symmetry. A shared logit perturbation assigns one value to each input-output orbit. Some of these values merely shift every output at an input together and are unobservable. Subtracting those input-orbit offsets gives the main result.
For the cyclic example, translating by and by preserves the residue . Every input pair can be reached from every other, so there is one input orbit. Triples with the same residue form one orbit, giving input-output orbits. The count is therefore .
Let a finite group act on and on compatibly with the projection to . Equivalently, sends an input to and relabels outputs over that input. Assume is -equivariant: . The group acts linearly on raw logits and on the gauge subspace , hence on .
Definition 5.1 (Lookup and -shared charts). The lookup chart is the full quotient logit chart . The -shared chart is the invariant affine subspace , where denotes the -fixed subspace.
Theorem 5.2 (Orbit-count active dimension for exact sharing). For full-support -equivariant , the lookup chart has active rank
The -shared chart has active rank
where is the number of -orbits in . Their quadratic active contributions are and , and these equal the LLCs of these affine charts by Lemma 3.1.
Proof sketch. The lookup rank is . For the shared chart, Lemma 3.1 reduces the problem to computing . Since is finite and the field has characteristic zero, taking invariants is exact, so . The dimension of is the number of orbits on input-output pairs, and the dimension of is the number of orbits on inputs. Appendix C gives the full argument.
Theorem 5.2 connects local Bayesian complexity to weight sharing and equivariance [7, 16, 4]. It is not a claim that all useful mechanisms are symmetries; rather, exact group sharing is a large class where the mechanism-local rank is computable without inspecting coordinates one by one.
Modular addition: lookup versus a shared cyclic rule
Let , , and define the softened modular-addition target
The lookup implementation assigns an independent quotient-logit vector to every pair . The shared cyclic implementation uses logits that depend only on the residue .
This is the group-sharing theorem with acting by
There is one orbit on inputs and orbits on input-output triples, classified by . Hence:
Corollary 6.1 (Same distribution, different local geometry). For the softened modular-addition distribution in Eq. (13), after quotienting softmax gauges,
The quadratic active contributions (and the LLCs of these affine charts) are
Both charts implement exactly the same conditional distribution at the base point.
The factor- gap is not a parameter-count artifact: by Proposition 4.2, adding dead parameters or duplicating the shared coordinates does not change the active rank. It is also not an invariant of the point distribution: by Remark 4.3, allowing arbitrary lookup perturbations around the shared point changes the local model and recovers the lookup rank.
Section 7 tests these ranks on trained checkpoints. Appendix F gives separate deterministic algebra checks and distinguishes Fisher rank from hard-label score-Gram rank.
Evaluating charts on neural networks
A chart is a hypothesis about permitted interventions, not something predictions uniquely identify. Natural choices include perturbing tied convolutional kernels, varying an equivariant layer within its constrained parameter space, changing a compiled RASP program, or intervening on specified heads, MLP blocks, Fourier directions, or sparse-autoencoder features. These choices need external support and need not be unique.
Evaluation procedure. First specify the candidate and its intervention coordinates. Next measure its quotient-logit Jacobian over the evaluation domain and form . Count its independent directions, or inspect its spectrum when sharing is approximate. Finally ask whether those directions explain an independently specified intervention response: fit its tangent coefficients on one set of inputs and measure the Fisher-weighted residual on held-out inputs. A low residual supports the candidate relative to that response and intervention family. Rank alone is not a selection score: projecting onto a small subspace already limits rank by construction.
Trained modular-addition checkpoints
We train a two-hidden-layer tanh MLP of width 64 and a residue-shared network on the complete softened modular-addition domain (). We measure the unrestricted chart, a prespecified shared projection of the same MLP Jacobian, and the shared architecture’s own chart. All runs reach 100% accuracy. Table 1 recovers the predicted ranks across three seeds for , 7 and two for , at relative singular-value thresholds , , . Appendix I gives the protocol and numerical fit errors.
| Seeds | Unrestricted MLP | Shared projection | Shared architecture | |
| 5 | 3 | 100 | 4 | 4 |
| 7 | 3 | 294 | 6 | 6 |
| 11 | 2 | 1210 | 10 | 10 |
| Theory |
Table 1. Active ranks from trained checkpoints. Each entry is unchanged across the tested seeds and thresholds. The projected chart is imposed on the same unrestricted MLP; it is not discovered by training.
The interpretation is limited but useful: chart rank can be computed from a trained network, and the same checkpoint admits different chart ranks. Random-initialization controls at already have the same ranks (Appendix I); the table therefore validates measurement, not emergence of modular structure. Fitting the function does not turn an unrestricted parameter chart into a shared chart.
Controlled transformer candidate comparison
To test discrimination rather than rank calculation, we compile direct and decomposed RASP programs for the adjacent-token rule “previous token is a and current token is b” using Tracr [Weiss et al., 2021, Lindner et al., 2023]. They agree on all 98 output positions in the complete domain of 30 binary sequences of lengths one through four. At four compiler-known intermediate computations, each candidate independently changes the gains of two orthonormal residual directions. We compare the compiler pair with every incorrect named pair and 500 random pairs of the same dimension.
The target is the logit response to ablating the compiler pair. Candidate coefficients are fit on three sequence folds and evaluated on the fourth. The mean relative Fisher-weighted held-out residual is below for the compiler chart, at least 0.67 for every named alternative, and 0.009–0.058 for the best random pair in each task (Table 2). The defining ablations change 17.3% of output positions. These results test the ability to explain a known causal response; the target itself uses compiler information.
| Intermediate | Compiler | Named best | Random best | Random median |
| Direct: previous token | 0 | 0.691 | 0.057 | 0.601 |
| Decomposed: previous token | 0 | 0.691 | 0.030 | 0.606 |
| Decomposed: current-is-b | 0.674 | 0.058 | 0.674 | |
| Decomposed: previous-is-a | 0.691 | 0.009 | 0.586 |
Table 2. Independent-gain chart comparison. Residuals are means over four held-out sequence folds; lower is better. Random columns summarize the fixed pool of 500 candidates per task, not repeated training runs.
Negative controls define the scope. At both final-output computations, all 500 random pairs tie the compiler chart within . Allowing arbitrary mixing within each candidate subspace, rather than independent gains, also removes discrimination in three of four intermediate tasks. The positive result thus depends on the specified intervention family. It establishes neither prediction-only identification nor automatic circuit discovery in learned transformers.
Approximate sharing: inspect the spectral tail
Exact rank can jump when a small perturbation breaks symmetry. A more stable question is how much spectrum lies outside the shared directions. Fix parameter coordinates, let be the tangent map, and let be the -orthogonal projection onto the shared subspace of dimension . With and , Appendix G proves
where eigenvalues are ordered decreasingly. Small symmetry-breaking sensitivity therefore implies a small tail, even if exact rank increases. The converse is not guaranteed: a small tail alone need not identify the correct shared subspace. Numerical almost-shared models confirm the bounds; they are spectral diagnostics in fixed coordinates, not coordinate-invariant estimates of the full LLC.
Bayesian model selection and relation to SLT
For a local statistical model with empirical loss and prior density positive near a minimizing set, SLT gives the free-energy expansion
where is the learning coefficient and is its multiplicity [Watanabe, 2009, 2013, 2018]. In an analytic regular rank- active chart with positive smooth prior, . Therefore, when two specified model-prior pairs fit the same distribution equally well and differ only in regular active rank, the lower-rank mechanism receives the smaller asymptotic complexity penalty. This is a local asymptotic comparison: prior mass, constants, and finite-sample effects can dominate at finite .
What rank misses. For an analytic nonnegative loss germ with Hessian rank , analytic splitting gives local coordinates in which , with having no quadratic part. Under a positive smooth local prior,
where if is identically zero. Appendix H gives the argument. Thus rank measures the quadratic contribution, while higher-order directions can add complexity.
Example 8.1 (Equal rank, different learning coefficients). The losses and both have Hessian rank one. Nevertheless, their LLCs are and (Proposition D.1). These behaviors also occur in finite-output charts: near a full-support distribution, logits varying as or have KL losses comparable to these forms. First-order mechanism equivalence does not imply equality of higher-order singularities.
The present results should be read as a local, finite-output complement to Programs as Singularities [Murfet and Troiani, 2025]. That work constructs smooth relaxations of Turing-machine codes and relates the Taylor expansion of the loss to error syndromes internal to a program. Here the “code perturbations” are mechanism charts in quotient logit space. The shared message is that Bayesian/SLT complexity can depend on implementation-local geometry even when predictive behavior agrees. The difference is that our main theorems are finite-dimensional linear and group-theoretic at second order; Appendix D records a simple higher-order normal form that begins to recover the singular phenomena emphasized by SLT.
The theory also clarifies the relationship to flatness-based generalization arguments. Hessian spectra and flatness are not invariant under arbitrary neural reparameterizations [8]. Our active-rank statements deliberately avoid arbitrary parameter-space flatness: they are invariant only under mechanism-preserving changes that leave the tangent image in logit space unchanged. This is a weaker but better-posed claim.
Related work
Classical Fisher geometry. Lemma 3.1 is closely related to standard Fisher-information geometry for parameterized statistical manifolds and tangent spaces [2, 15, 1]. An implementation chart is mathematically a local parameterization of a restricted statistical model; it does not replace that established concept. Our focus is the comparison of mechanism-supported restricted models through the same point distribution, after removing the per-input softmax gauge. Equality of tangent images separates a coordinate change from a change of perturbation family. The orbit-count formula then computes that dimension for exactly shared mechanisms. This comparison and computation, rather than a new notion of statistical manifold, are the contribution.
Singular learning theory and local complexity. SLT replaces regular dimension penalties by birational invariants of the KL germ [Watanabe, 2009, 2013, 2018]; related singular-model and algebraic-statistics perspectives include [10], [9]. Earlier information-geometric and Bayesian complexity perspectives include [3], Schwarz [1978], MacKay [1992]. The LLC has recently been proposed as a practical singularity-aware complexity measure for deep networks [Lau et al., 2025], with follow-up work connecting degeneracy to stagewise development and component specialization [13] (Wang et al. 2024). Our theorems do not estimate LLCs in large networks; they identify a tractable class of implementation-local KL germs where the quadratic active contribution can be computed exactly.
Programs, algorithms, and structural priors. Algorithmic information theory and MDL give function- or code-length-based accounts of simplicity [14] (Solomonoff 1964; Rissanen 1978; Vereshchagin and Vitanyi 2004). Program-synthesis geometry and Programs as Singularities instead study smooth neighborhoods of program codes [6] (Murfet and Troiani 2025). Our contribution is closer to the latter: the relevant object is not only which function is computed, but which perturbations of the implementation are locally available.
Mechanistic interpretability and compiled transformers. Mechanistic interpretability uses interventions, causal abstractions, and circuit-level hypotheses to describe internal algorithms [11, 12, 5]. RASP and Tracr provide programmatic laboratories where transformer mechanisms are known by construction [Weiss et al., 2021, Lindner et al., 2023]. Benchmarks such as MIB emphasize that localization methods must be compared against strong patching and attribution baselines [Mueller et al., 2025]. Our paper supplies a theoretical target for one kind of diagnostic—mechanism-local dimension—rather than a new benchmark-winning localization algorithm.
Equivariance and sharing. Weight sharing and equivariance are central tools for building models whose local perturbations respect a symmetry [7, 16, 4]. Our finite-group theorem can be read as the local Fisher-rank consequence of exact sharing in the finite-output setting: invariant logit functions have one degree of freedom per input-output orbit, minus the input-gauge orbits.
Limitations
The results require a finite evaluation domain and an externally specified chart. The trained-model experiment checks measurement; it does not show that sharing was learned. The Tracr experiment uses compiler-defined ablations and a restricted intervention family, with explicit non-identifiability controls. Neither experiment establishes chart discovery or performance on generic learned circuits. Evaluating complete Jacobians also becomes expensive as the domain and parameter count grow.
The orbit formula concerns exact sharing and full-support predictions. Approximate sharing admits the spectral bounds in Appendix § 18, but rank remains discontinuous and probabilities near zero make numerical rank ill-conditioned. The bounds depend on fixed intervention coordinates. Higher-order terms can change LLCs at identical active rank, and hard-label score–Gram ranks are not calibrated LLC estimates. Local asymptotic Bayesian complexity does not by itself predict finite-sample preference or training dynamics.
Conclusion
Equal predictions leave the allowed local changes undetermined. Specifying those changes makes a mechanism hypothesis measurable: finite-group sharing has an exact orbit-count dimension, and controlled experiments show how chart spectra and intervention residuals can be evaluated. The useful next step is evidence-based candidate generation in learned networks, with the same attention to negative controls and intervention choice.
Proof of the logit-Fisher pullback lemma
Choose any representative of a quotient logit vector. The function
is invariant under . Its gradient in is , and its Hessian is
At this Hessian is . Since has full support, for every nonconstant representative ,
so is positive definite on . Summing with positive weights gives the positive-definite quotient metric in Eq. (5).
Let . Since , the second derivative chain rule has no term involving , and therefore
The nullspace is , because is positive definite. Hence the Hessian rank is . On the quotient , this is a nondegenerate quadratic minimum. Under the analytic and prior assumptions stated in Section 2, the zeta integral of a nondegenerate rank- quadratic form has leading pole at , giving quadratic active contribution . If tangent-kernel coordinates are not removed or genuinely dead, this calculation describes only the active quadratic block.
Proof of invariance and the perturbation-class observation
For Proposition 4.2, set . The composed tangent map is . If is surjective, then , so the active rank is unchanged by Lemma 3.1. If is a local diffeomorphism, this is ordinary coordinate invariance. If , the extra coordinates do not enter the germ. If , the linear change of variables separates active and redundant coordinates.
For the learning-coefficient statement, assume the analytic and prior conditions in Proposition 4.2. The analytic constant-rank theorem gives local coordinates near and near in which on the image coordinates. In these coordinates, the local zeta integral of the pullback germ is
over sufficiently small neighborhoods. The inner integral is a positive smooth factor with finite nonzero value near the origin, so it does not change the leading pole of the zeta function. Hence the RLCT, and therefore the learning coefficient, is unchanged. This is the formal version of saying that dead or duplicated coordinates contribute only finite fiber volume.
For Remark 4.3, the charts and have derivative images and , respectively. Lemma 3.1 gives ranks and . This proves that changing the local perturbation family changes the statistical model being compared.
Proof of the group-sharing theorem
Let be raw logits and let be the gauge subspace of functions constant over for each fixed . Then . Compatibility of the action with the projection implies is -stable.
Because is finite and we work over , the Reynolds operator projects every representation onto its invariant subspace. Hence taking invariants is an exact functor, and the short exact sequence yields
Thus .
A function on a finite -set is invariant exactly when it is constant on orbits. Therefore . Similarly, a gauge function is specified by one scalar per input, and -invariance means this scalar is constant on input orbits; hence . This proves Eq. (12). Since the target is full support, Lemma 3.1 converts this tangent dimension into active Hessian rank.
Higher-order protected directions
The pullback rank is a second-order invariant. SLT is sensitive to higher-order terms when some first-order perturbations are protected. The following normal form records the basic calculation used throughout singular learning theory.
Proposition D.1 (Product normal form). Suppose an analytic nonnegative germ is locally comparable to
where are dead coordinates. That is, near the origin for some . Then has the same local learning coefficient as ,
Proof. For a nonnegative analytic germ with positive smooth local density, the learning coefficient is the critical integrability exponent: is finite for and divergent for . The inequalities give the same threshold for and .
To compute it for the displayed sum, substitute and in its sublevel set . The rescaled active set has finite nonzero volume independent of . Its Jacobian contributes ; dead coordinates supply a finite factor. A positive smooth prior is bounded above and below locally and leaves this exponent unchanged. This sublevel exponent equals the critical integrability threshold, proving the formula [Watanabe, 2009, 2018].
This proposition explains the difference between dead code, duplicated code, and error-protected code. Dead coordinates do not affect and contribute nothing. Duplicated coordinates such as become one quadratic coordinate plus one dead coordinate. A protected coordinate that first changes logits at order contributes rather than . Programs as Singularities develops a much richer version of this idea for noisy Turing-machine codes and error syndromes [Murfet and Troiani, 2025].
Modular-addition derivations
Let in residue coordinates, with
The per-input Fisher matrix is . It has one gauge eigenvector with eigenvalue zero. Among incorrect residues, the independent contrasts have eigenvalue . The remaining centered direction contrasts the correct residue with the average incorrect residue and has eigenvalue . Thus rank on the quotient.
In the lookup chart, the expected KL Hessian is block diagonal with copies of , one for each input pair. Hence the rank is . In the shared cyclic chart, every input uses the same residue-logit perturbation, relabeled by the cyclic output action, so input averaging gives one copy of and rank .
For the hard-label influence diagnostic, the score vector at the correct label is . Lookup scores occupy disjoint blocks, so the input-level Gram matrix is . Shared cyclic scores all equal , so the Gram matrix is .
Deterministic numerical checks and score conventions
The accompanying code supplement contains finite-orbit enumeration, explicit small-matrix Hessian checks, per-input Fisher spectra, quadratic-posterior calculations, and protected/dead/duplicated normal-form checks. These verify algebra, separately from the trained and compiled-network experiments. Running instructions, environment versions, and saved results are included in the supplement. The original anonymous artifact remains at https://anonymous.4open.science/r/review-artifact-2026-16-FD3C; it predates the rebuttal experiments and is not their reproducibility package.
| influence ranks | |||||
| 3 | 18 | 2 | 9 | 1 | 9/1 |
| 5 | 100 | 4 | 50 | 2 | 25/1 |
| 7 | 294 | 6 | 147 | 3 | 49/1 |
| 11 | 1210 | 10 | 605 | 5 | 121/1 |
Table 3. Deterministic modular-addition ranks for the lookup and shared cyclic charts. The values are independent of the finite temperature ; the supplement recomputes them by explicit small-matrix checks for these values.
These are deterministic algebraic sanity checks, not empirical validation. Their purpose is to make the size of the rank gap visible and to catch implementation mistakes in the finite calculations.
Influence-rank diagnostic. If per-input scores are taken from the hard-label cross-entropy at the matched soft distribution, the same contrast appears in an observable Gram matrix. Let the label be and let in residue coordinates. Lookup scores for different inputs lie in disjoint parameter blocks, so the input-level Gram matrix is and has rank . Shared cyclic scores are identical across inputs, so the Gram matrix is and has rank 1. This diagnostic convention is separate from the Fisher rank: the population sampled-label Fisher score covariance recovers the Hessian rank, while finite empirical score matrices require enough sampled labels to span the quotient directions and soft-target KL gradients vanish at the optimum.
Approximate sharing and conditioning
Proposition G.1 (Spectral tail outside a candidate shared subspace). Fix a full-support conditional distribution, positive input masses, and Euclidean coordinates on the chart parameters. Let denote the positive-definite quotient-logit metric, a tangent map, and a subspace of dimension . Let be the -orthogonal projection onto and set , , and . With singular values and eigenvalues ordered decreasingly (and padded by zeros),
The statement applies in particular to with from Theorem 5.2.
Proof. Since , has rank at most . The variational characterization of singular values bounds the st singular value of by its operator-norm distance to any rank-at-most- matrix, hence by . For the Frobenius bound, take a singular value decomposition of . Truncating it after terms minimizes squared Frobenius approximation error over rank-at-most- matrices; the minimum is and cannot exceed . Finally, , so its eigenvalues are the squared singular values of .
The proof requires a candidate subspace, not exact equivariance of the base distribution. When is not equivariant, an affine chart through its base logits with tangent represents shared perturbations, not an exactly equivariant predictive family. The -orthogonal projection remains well-defined, but need not equal unweighted group averaging. At a checkpoint that does not exactly fit the chosen target, is a target-weighted tangent diagnostic; it need not equal the exact loss Hessian there, whose chain rule also includes a nonzero-gradient term.
Full-support conditioning. For one input, choose the representative satisfying . If for every output, then
This follows directly from the diagonal weights after the centering term vanishes. Thus Fisher weighting is nondegenerate away from the simplex boundary. Numerical rank still requires a singular-value gap relative to the threshold; full support alone does not supply that gap. Small probabilities can make nonzero directions difficult to resolve even though the exact orbit-count rank is unchanged.
Numerical checks. The supplement tests at . Synthetic shared charts receive symmetry-breaking corrections with weighted operator norms , , . Almost-shared networks add a -scaled dense correction to a residue-shared network, with and seed zero. Every recorded operator and Frobenius bound holds. For , , the weighted residual norm is approximately , the sixth eigenvalue is , and the seventh is , approximately the squared bound. These are controlled numerical checks of the inequality, not a test of spontaneous learned equivariance. The parameter coordinates and their scaling are fixed throughout.
Quadratic and higher-order contributions to the LLC
Let be a nonnegative analytic germ with and Hessian rank . Split its initial coordinates into in the positive Hessian block and in its kernel. The analytic implicit-function theorem solves as near zero. Set . The analytic Morse lemma with parameters then gives a local analytic coordinate change for which
Nonnegativity implies ; the kernel choice gives . A positive smooth local prior and the coordinate-change Jacobian are bounded above and below by positive constants on a sufficiently small neighborhood, so they do not affect the leading integrability exponent.
For the displayed separated form, the local Laplace integral with constant density factors into a Gaussian integral in and an integral of in . The Gaussian factor has leading order . The analytic nonnegative germ contributes up to logarithmic factors by the usual local SLT asymptotics [Watanabe, 2009, 2018]. Comparing with the original positive density yields
If , its integral has constant positive volume and we set . This establishes the lower bound and its limitation: a tangent-kernel direction can be inactive quadratically without being dead at higher orders. The statement is local to the chosen germ and positive prior, not a formula obtained by averaging ranks over training checkpoints.
Trained-checkpoint protocol and initialization controls
Data and models. For each , we enumerate all pairs with uniform input mass and target from Eq. (13), using . The unrestricted network takes concatenated one-hot encodings of (dimension ), has two tanh hidden layers of width 64, and outputs logits. The shared network maps a one-hot encoding of residue through a width-16 tanh hidden layer and one scalar output; the same map is reused for every . Both include biases. Thus sharing is architectural in the second model and imposed as a candidate intervention in the projected first model.
Training. We minimize full-domain soft-target cross-entropy in float64 with PyTorch L-BFGS, learning rate 1, at most 500 iterations, strong-Wolfe line search, gradient tolerance , and objective-change tolerance . Seeds are 0, 1, 2 for and 0, 1 for . There is no train/test split in this fitting experiment: it is a complete-domain geometry check, not a generalization benchmark. Widths and optimizer settings are fixed across seeds; no best-seed selection is used.
Coordinates and projection. The quotient coordinates are for . Their metric uses the upper-left block of , scaled by . We differentiate these coordinates with respect to all parameters to form . For sharing, let contain the quotient-coordinate effects of the residue-kernel contrasts. If is an orthonormal basis of , the projected spectrum is computed from . This is the nonzero spectrum of the Fisher-orthogonal projected tangent, expressed without an explicit large projector. The original rebuttal script used a Euclidean quotient-coordinate projection; the present implementation uses Fisher-orthogonal projection consistently. Independent reruns reproduce every reported rank with this convention.
Numerical precision and variability. A singular value counts as nonzero when it exceeds times the largest, for . All seeds and thresholds give the entries of Table 1; the observed rank range for each entry is a singleton. Every trained model has 100% full-domain classification accuracy. In the camera-ready rerun, mean KL is below for and for . The archived rebuttal run had a smaller maximum (); numerical optimizer stopping changes this fit error without changing any rank. These finite-precision fits approximate, rather than exactly satisfy, the theorem’s matching-distribution hypothesis.
Random-initialization control. With training disabled and the same three seeds for , the unrestricted, projected, and shared ranks are respectively 100/4/4 and 294/6/6 at all three thresholds. At initialization the target-weighted pullback is not generally the loss Hessian, since the target has not been fitted. This control shows that the rank gap can already follow from the architecture and candidate choice. It rules out interpreting the trained rank table as evidence that modular structure emerged during optimization.
Compiled-transformer protocol and full controls
The direct RASP program detects an adjacent ab pair through a joint rule. The decomposed program combines separate previous-is-a and current-is-b predicates. Tracr compiles both to transformer weights. We enumerate all 30 sequences over of lengths one through four, prepend the compiler’s beginning-of-sequence token, and score the 98 non-BOS positions. The two programs have zero mismatched positions. The intervention-capable forward pass is checked against the compiled model before candidate scoring.
For a component output row and orthonormal residual directions , a candidate’s independent-gain chart is
The compiler supplies a pair of categorical features at each selected component. Setting both gains to ablates that pair and defines the target logit-contrast response . Every incorrect pair of named residual directions and 500 random orthonormal pairs is evaluated, using seed 20260726 and the same two-coordinate family. Candidate tangents are derivatives at zero intervention, computed by automatic differentiation through the downstream model. The finite ablation response is not assumed to be linear a priori; the residual measures how well a tangent predicts it.
We use the softened binary reference distribution with . Its binary quotient-logit Fisher weight is the same scalar at every position, with uniform position mass . Therefore Fisher weighting cancels from the relative residual in this particular benchmark; discrimination comes from the response directions, not heterogeneous confidence weights.
Sequences are enumerated deterministically and assigned to four folds by sequence index modulo four. For each held-out fold , least squares on the other folds gives
Here in binary contrast coordinates, and the pseudoinverse cutoff is . Rows from one sequence stay together. We report the arithmetic mean of the four valid fold residuals, not a test-set discovery accuracy. The candidate pool is fixed before residual scoring; comparisons use a practical tie tolerance of .
All four compiler charts have rank two and are uniquely best among the named pairs; none of the 500 random candidates per task ties within tolerance. The defining ablation changes 17 of 98 output positions in each task. These descriptive comparisons do not establish a distribution-free significance guarantee for circuit recovery. The intervention target is compiler-supported ground truth, and the sequence domain is deliberately small.
Output and broader-intervention controls. For both final-output components, all 500 random independent-gain pairs tie the compiler chart. The broader family permits four coordinates (all entries of a matrix ), with the same family used for every candidate in that comparison. In the two previous-token tasks and the previous-is-a task, 99.8–100% of random subspaces tie. Current-is-b remains separated (best random residual about 0.006). Independent gains distinguish individual directions; arbitrary mixing depends only on their span. Broadening the family thus changes the mechanism hypothesis and can erase identifiability.
Reproduction and compute. The supplement includes the exact Tracr source snapshot used for compilation, its Apache-2.0 license, checksums, package versions, benchmark script, per-candidate scores, and summary tables. The camera-ready verification uses Python 3.11, NumPy 1.26.4, JAX/JAXlib 0.4.28, Haiku 0.0.12, Chex 0.1.86, and PyTorch 2.9.0 for the separate modular experiments. All reported checks run on an AMD Ryzen 9 7900X3D CPU, with one BLAS/OpenMP thread; no external dataset is needed. Verification takes approximately 5 seconds for the six fits, 16 seconds for the two fits, 2 seconds for initialization controls, 11 seconds for Tracr, and 3 seconds for approximate-sharing checks. Peak resident memory is below 1 GiB per process. Exact measurements and commands are recorded with the supplement. These measurements cover verification of the reported experiments, not the broader exploratory research preceding this paper.
References
- [1]Shun-ichi Amari. Information Geometry and Its Applications. Springer, 2016.
- [2]Shun-ichi Amari and Hiroshi Nagaoka. Methods of Information Geometry. American Mathematical Society and Oxford University Press, 2000.
- [3]Vijay Balasubramanian. Statistical inference, Occam’s razor, and statistical mechanics on the space of probability distributions. Neural Computation, 9(2):349–368, 1997.arxiv.org/abs/cond-mat/9601030
- [4]Michael M. Bronstein, Joan Bruna, Taco Cohen, and Petar Veličković. Geometric deep learning: Grids, groups, graphs, geodesics, and gauges. arXiv preprint arXiv:2104.13478, 2021.arxiv.org/abs/2104.13478
- [5]Lawrence Chan, Adrià Garriga-Alonso, Nicholas Goldowsky-Dill, Ryan Greenblatt, Olga Nitishinskaya, Ansh Radhakrishnan, Buck Shlegeris, and Nate Thomas. Causal scrubbing: A method for rigorously testing interpretability hypotheses. Alignment Forum, 2022. URL https://www.alignmentforum.org/posts/JvZhzhzCu2Yd57RN/causal-scrubbing-a-method-for-rigorously-testing.
- [6]James Clift, Daniel Murfet, and James Wallbridge. Geometry of program synthesis. arXiv preprint arXiv:2103.16080, 2021.arxiv.org/abs/2103.16080
- [7]Taco Cohen and Max Welling. Group equivariant convolutional networks. In Proceedings of the 33rd International Conference on Machine Learning, volume 48 of Proceedings of Machine Learning Research, pages 2990–2999. PMLR, 2016.arxiv.org/abs/1602.07576
- [8]Laurent Dinh, Razvan Pascanu, Samy Bengio, and Yoshua Bengio. Sharp minima can generalize for deep nets. In Proceedings of the 34th International Conference on Machine Learning, volume 70 of Proceedings of Machine Learning Research, pages 1019–1028. PMLR, 2017.arxiv.org/abs/1703.04933
- [9]Mathias Drton and Martyn Plummer. A Bayesian information criterion for singular models. Journal of the Royal Statistical Society: Series B, 79(2):323–380, 2017.arxiv.org/abs/1309.0911
- [10]Mathias Drton, Bernd Sturmfels, and Seth Sullivant. Lectures on Algebraic Statistics. Birkhäuser, 2009.DOI
- [11]Nelson Elhage, Neel Nanda, Catherine Olsson, Tom Henighan, Nicholas Joseph, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, Nova DasSarma, Dawn Drain, Deep Ganguli, Zac Hatfield-Dodds, Danny Hernandez, Andy Jones, Jackson Kernion, Liane Lovitt, Kamal Ndousse, Dario Amodei, Tom Brown, Jack Clark, Jared Kaplan, Sam McCandlish, and Christopher Olah. A mathematical framework for transformer circuits. Transformer Circuits Thread, 2021. URL https://transformer-circuits.pub/2021/framework/index.html.
- [12]Atticus Geiger, Duiligur Ibeling, Amir Zur, Maheep Chaudhary, Sonakshi Chauhan, Jing Huang, Aryaman Arora, Zhengxuan Wu, Noah Goodman, Christopher Potts, and Thomas Icard. Causal abstraction for mechanistic interpretability. arXiv preprint arXiv:2301.04709, 2023.arxiv.org/abs/2301.04709
- [13]Jesse Hoogland, George Wang, Matthew Farrugia-Roberts, Liam Carroll, Susan Wei, and Daniel Murfet. Loss landscape degeneracy drives stagewise development in transformers. arXiv preprint arXiv:2402.02364, 2024.arxiv.org/abs/2402.02364
- [14]Marcus Hutter. Universal Artificial Intelligence: Sequential Decisions Based on Algorithmic Probability. Springer, 2005.
- [15]Robert E. Kass and Paul W. Vos. Geometrical Foundations of Asymptotic Inference. Wiley, 1997.
- [16]Risi Kondor and Shubhendu Trivedi. On the generalization of equivariance and convolution in neural networks to the action of compact groups. In Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pages 2747–2755. PMLR, 2018.arxiv.org/abs/1802.03690