WorksComputer Science
AROMA: Adaptive Representation-Oriented Multimodal Actuation for Vision-Language Model Control
Abstract
Mechanistic localization does not by itself yield usable control: causal sensitivity may occupy little activation energy, an accurate local direction may fail at finite scale, and a reachable numeral target may be wrong. We introduce AROMA (Adaptive Representation-Oriented Multimodal Actuation), a frozen-VLM control framework separating where to intervene, which geometry and gain to use, how to reach a target, which target to pursue, and when to act. In Llama-3.2-11B-Vision-Instruct, four dimensions of a causally validated cross-attention head explain 89.82% of cardinality sensitivity but only 9.52% of median activation energy; magnitude-normalized causal subspace actuation converts this mismatch into finite control, improving exact-count accuracy on untouched procedural data by +8.00 pp. Iterative Minimal-Effort Actuation (MEA) reaches 81.25% of oracle targets versus 15.18% for a frozen direction under matched step budgets. With target and gate frozen before evaluation, targeted MEA improves the previous frozen policy from 34.444% to 38.730% on a new natural Stage-2 cohort (N=630; paired p=4.63e-7) and yields 65.741% versus 65.153% over all 7,992 examples while editing only 63 inputs. Prospective routing, replication across heads and models, and comparisons with attribution-steering baselines further characterize each stage, separating localization, actuation, reachability, target quality, and selective deployment as distinct control problems.
Introduction
A child who can count to ten has already mastered a skill that today’s vision–language models (VLMs) still find difficult: even frontier models behave as “4-knowers,” enumerating reliably only within the subitizing range [25], and real-world counting remains hard for modern multimodal systems [24, 17] (Rong et al. 2026). Counting requires object individuation, persistence of partial visual evidence, and exact aggregation, making it a controlled probe of whether a VLM’s answer is grounded in its internal visual computation.
Mechanistic work increasingly suggests that the relevant computation is often present before the answer is emitted. Counting can be traced to identifiable attention heads [4], and the correct count can remain decodable even when the verbalized answer is wrong (El-Shangti et al., 2026). The natural next question is therefore not only where a behavior lives, but how to convert that localization into reliable control.
Existing causal localization methods identify components implicated in a behavior, while activation steering specifies interventions through a direction, magnitude, or conditional application [14, 27, 35, 12]. Neither step alone establishes that a localized component can be actuated at finite scale, that a chosen numeral target is correct, or that an edit should be deployed on a particular example. Our central distinction is therefore between discovering a causal mechanism and turning it into a reliable, target-aware deployment policy—with independently testable geometry, finite control, target selection, and gating.
AROMA addresses this mechanism-to-control gap by factorizing the control stack into five decisions (Figure 1). Where: a cross-attention head is admitted as an actuator only after bidirectional intervention shows that it moves the count. Which geometry: magnitude-normalized low-rank edits or whole-head scaling can be selected within the same actuator. How: a first-order response law, validated for small edits, is extended to explicit numeral targets by Minimal-Effort Actuation (MEA), which re-linearizes a target-competitor margin after every step. Which target: a ground-truth-free rule proposes the numeral to pursue. When: a learned gate and candidate router decide, per example, whether to intervene and with which geometry and gain. Counting supplies a discrete output space in which active edits can be audited as repairs, breaks, or unchanged predictions, with abstention tracked separately; prospective deployment claims are confirmed on untouched data after their methods, comparators, and endpoints were frozen.

Figure 1. AROMA at a glance. Mechanism (top): a frozen VLM exposes a causally validated cross-attention head; its Top-4 sensitivity subspace supports magnitude-normalized MN/SA edits, while WHOLE is an alternative full-head intervention geometry on the same actuator. Policy (bottom): a ground-truth-free gate decides when to act; on gated examples, either a target rule selects which target to pursue and MEA determines how to reach it, or a candidate router selects geometry and gain without an explicit target.
Our contributions are:
A factorized mechanism-to-control formulation that separates where, which geometry, how, which target, and when into distinct testable stages rather than one steering heuristic (§3).
Low-rank causal actuation. Inside a validated causal actuator, we identify a sensitivity–energy mismatch and introduce magnitude-normalized causal subspace actuation (MN-CSA), which yields a frozen +8.00 pp gain on untouched data; bidirectional actuator behavior replicates in 6/7 candidate heads, and the low-rank geometry ends_mid=1
is independently recovered in IDEFICS-9B-Instruct (§5.1, §5.3).
A validated local control law and iterative target controller. Projected first-order predictions are near-exact for small edits; MEA extends them to explicit targets and raises oracle reachability to 81.25%, decisively above frozen-direction and activation-aligned controls under matched budgets (§5.2).
Prospectively confirmed deployment with audited controls. Frozen direction/gain and geometry routers improve development-selected restricted comparators on untouched procedural and natural populations. On a second untouched natural cohort, runner-up targeting with MEA improves the frozen Stage-2 policy by +4.286 pp (29:2 paired discordance, ), and an identity-fallback reconstruction on all 7,992 examples gives 5,254 versus 5,207 correct (+0.588 pp). Under the same gate, it outperforms an attribution-steering comparator by +3.016 pp ( (§5.4–5.6, Appendix I).
Related Work
Counting in VLMs. Counting has long exposed the limits of generic attention in VQA [33, 26, 1], and large multimodal models remain unreliable counters beyond the subitizing range and in cluttered scenes [25, 24, 17, 5] (Rong et al. 2026). Mechanistically, Che et al. [4] trace counting to functionally distinct attention heads and regularize them during fine-tuning; El-Shangiti et al. [6] show that the correct count is often decodable but misaligned with the verbalized answer, and correct errors through probe-direction steering and re-prompting; attention-based interventions reweight count-relevant visual tokens at inference time [21] (Alzahroni et al. 2026). AROMA instead keeps the VLM frozen and acts on a causally validated actuator rather than on attention maps, probe directions, or a dedicated counter [18].
Mechanistic intervention and activation steering. Causal mediation analysis and causal tracing made intervention the standard of evidence for mechanistic claims [16, 28, 14], and recent work localizes multimodal components that causally influence grounding, hallucination, and task behavior [29, 34, 30] (Golanov et al. 2025; Budman et al. 2026; Rahmanzadehgerveri et al. 2025). Activation engineering steers hidden states at inference time [27, 35], with studies of intervention magnitude [23], conditional deployment [12], and VLM-specific steering [22, 32, 31]; subspace interpretations can nonetheless mislead without held-out intervention (Makelov et al., 2024). GrAnIS [36] constructs fixed steering directions from gradient attribution, token ablation, and representation differences; two explicitly task-adapted implementations serve as our external baselines (Appendix I). AROMA tests the boundaries between validated actuator, sensitivity geometry, finite target execution, target choice, and gating, rather than assuming that steering at one stage guarantees deployment utility (Appendix J.3).
AROMA
Problem Formulation
Control objective. For an image–question pair with integer answer , let denote correctness under action , with the identity. Realized repair utility is , so a controller must trade repairs against breaks rather than maximize intervention magnitude.
Actuators. Let be the concatenated cross-attention head state immediately before its output projection, and reserve baseline for the unmodified model. A localized head can be scaled as a whole, , where is identity, attenuates, and amplifies. We call this WHOLE or full-head actuation; it is one intervention geometry on the same actuator and serves throughout as a strong comparator to low-rank geometries. A candidate actuator is characterized by the normalized expected numeral over numeral support and must show bidirectional finite-step behavior rather than attention or gradient saliency alone.
Sensitivity Geometry and Magnitude-Normalized Actuation
For the selected head, let be the token-level head output and its pooled coordinate. Define local cardinality sensitivity and its second moment , with the top- eigenspace and its projector. only proposes candidate directions inside an actuator already validated by intervention; spectral concentration alone is not treated as causal proof. To quantify how much of the actuator state the subspace can act on, we measure the projected activation-energy share , where
A naive projected analogue of whole-head scaling is . If captures high sensitivity but little activation norm (Figure 2), this edit is much smaller than whole-head scaling. With the activation-aligned direction , AROMA therefore uses magnitude-normalized causal subspace actuation (MN-CSA),

Figure 2. Sensitivity is not activation energy. (a) The top four dimensions explain 89.82% of the uncentered cardinality-sensitivity second moment and 86.78% after centering; both are far above the rank-matched null distribution. (b) The same Top-4 subspace contains little of the pooled activation norm: median , corresponding to only 9.52% median projected activation energy. This mismatch predicts why same- low-rank scaling under-actuates and motivates magnitude normalization.
so , matching the pooled displacement magnitude of whole-head scaling. We apply the same to every token state, preserving token residual differences. A gain rescales the edit, : is the equal-norm test, and is explicitly a stronger low-rank intervention that is never compared as equal-strength efficiency.
Local Response Law and Minimal-Effort Actuation
Local response. For a small edit in the frozen Top-4 subspace, the response obeys the first-order law with . Among equal-norm directions in the subspace, the sensitivity-aligned direction is locally optimal for increasing , whereas is activation-aligned. Their cosine is the relative local efficiency of SA-MN-CSA; it diagnoses geometry and is not assumed to predict finite-gain correctness.
Target-specific margin control. For an explicit target numeral , let be the current highest-scoring non-target numeral, the logit margin, and its projected gradient. The minimum-norm first-order step that reaches the local decision boundary is . Large one-shot extrapolations are unreliable, so MEA applies this step inside a trust region,
with trust radius and per-example reference budget , where is the Stage-1 proposal of §3.4. The model is re-evaluated after every step; , the margin, and are recomputed for at most 20 iterations.
Oracle and deployable targets. Oracle MEA sets to the ground-truth numeral and is used only to measure reachable actuator capacity. Deployable MEA never observes ground truth: on gated examples (§3.4) it pursues the runner-up target
where is the frozen Stage-1 baseline prediction and ties choose the smaller numeral. The rule is intentionally simple: its role is to isolate which target from how to reach.
Deployment Policy: Gating and Routing
Deployment Policy: Gating and Routing
Stage 1: Proposal Controller. A Proposal Controller maps 39 ground-truth-free features (numeral probabilities, uncertainty statistics, and geometric summaries) to a coarse cardinality action . Proposals with are executed directly; upward proposals () are refined by Stage 2.
Stage 2: candidate-level utility routing. Stage 2 selects, per example, among abstention (IDENTITY) and candidate edits . The Candidate Utility Router operates within the low-rank family (15 decisions per example); the Unified Geometry Router (UGR) additionally routes the intervention family itself, (43 candidates per example, with WHOLE applied at the matched scale). Each router scores every candidate with a candidate-conditioned L2-regularized logistic model over ground-truth-free features and deploys , breaking ties toward IDENTITY and smaller gain. For natural images, UGR is factorized into a gate that decides whether to retain IDENTITY or APPLY an edit and a pairwise ranker that selects among non-identity candidates only when the gate fires. Ground truth is used only to fit and evaluate routers; gain grids are given in Appendix D.
Two deployment regimes. AROMA supports two post-gate control laws. Candidate routing constructs no explicit target: it decides when to intervene and selects a bounded geometry/gain action. Targeted MEA keeps the same frozen gate, uses Eq. (3) for which target, and uses iterative margin control for how to reach. The frozen hierarchical UGR policy, which we call old Unified, is therefore the natural reference for targeted MEA: both act on the identical APPLY set, and only the post-gate control law changes. Note that only triggers Stage 2 and sets a reference budget; it is not a constraint that the final prediction must increase.
Experimental Setup
Models and scoring. The primary pipeline uses frozen Llama-3.2-11B-Vision-Instruct [15] with a deterministic numeral proxy over 0–15. Cross-model experiments use frozen IDEFICS-9B-Instruct, a Flamingo-style model with gated cross-attention blocks (Alayrac et al. [2]; Hugging Face, 2023), scored by teacher-forced sequence log-likelihood over complete numeral strings. We compare directions and effects within each model, not raw scores across models.
Data. Procedural counting uses Proc-Count-Causal cohorts with zero seed and pixel-hash overlap: v1 () for mechanism discovery, v2 () for controller development, v3 () for frozen confirmation and CSA development, and v4 (), generated only after MN-CSA was frozen. Routers are developed and confirmed on separately generated cohorts. Natural evaluation uses TallyQA (Acharya et al. [1]; Goyal et al. [8]; Krishna et al. [11]): disjoint samples for zero-shot transfer and Stage-1 recalibration, an untouched 8,000-example cohort for natural UGR, and a second untouched, pixel-deduplicated cohort of , constructed only after runner-up targeting and MEA were fully specified and sharing no question, image ID, or pixels with earlier natural data. HoloCount () is an external benchmark and GQA (Hudson and Manning [9]) an off-target stress test.
Evaluation protocol and evidence classes. A result is called prospective only if its method, comparator, endpoint, and statistical test were frozen before outcome execution on the evaluation cohort; the local-response audit and oracle MEA are mechanism/controller validations rather than deployment confirmations. Paired correctness uses two-sided exact McNemar tests, with paired bootstrap intervals where they were part of the frozen analysis chain; directional steering uses exact binomial tests with Holm correction. Stage-2 primary endpoints are conditional by design: the compared policies act on the prespecified upward-proposal branch. Separately, we report an audited, identity-fallback reconstruction over the full natural cohort derived from the same frozen per-example predictions. Appendix A gives the full chronology and statistics, and Table 1 there lists every headline result with its evidence class.
| Experiment | Primary result | Interpretation | Evidence class | |
| Llama directional characterization | 300 | 91.33% down; 90.67% up; 88.67% strict | Primary bidirectional causal actuator | Held-out characterization |
| Llama Top-4 geometry | 300 / 653 | 89.82% sensitivity (); 9.52% median projected energy ( audit) | Sensitivity–energy mismatch | Mechanism validation |
| Llama untouched v4 | 2000 | 50.25% base; 58.25% MN-CSA; +8.00 | Same-policy low-rank utility | Prospective untouched |
| E1 continuous local response | 160 | at : Pearson , Spearman , sign 100% | First-order projected control is locally accurate | Mechanism validation |
| Oracle MEA reachability | 112 | 81.25% target success; frozen direction 15.18%; activation-aligned 0.89% | Re-linearization converts local geometry into global reachability | Controller validation |
| Utility Router untouched final | 294 | 60.54% global MN 66.33%; +5.782 | Low-rank candidate routing | Prospective untouched |
| Procedural UGR untouched final | 633 | 70.93% restricted 73.14%; +2.212 | Geometry routing | Prospective untouched |
| Natural hierarchical UGR final | 631 | 33.439% Whole 35.341% Unified; +1.902 | Natural geometry routing | Prospective untouched |
| Prospective runner-up + MEA v2 | 630 | 34.444% old Unified 38.730% (+4.286 ; 29:2; ) | GT-free target + frozen gate converts reachability into utility | Prospective untouched |
| Full-cohort identity fallback | 7992 | 5207/7992 identity 5254/7992 gated MEA (+0.588 ) | Audited whole-population selective utility, derived from frozen outputs | Post-confirmation reconstruction |
| External GrAInS-inspired comparison | 630 | Same-gate 225/630 vs MEA 244/630; global GrAInS-inspired 247/630 | Policy allocation and deployment scope are distinct | Post-confirmation supplemental |
| Llama multi-actuator robustness | 300 | 6/7 heads replicate bidirectionally; 5/6 non-primary | Within-model actuator robustness | Prospective robustness |
| IDEFICS actuator / geometry | 300 | 82.00% down; 82.67% up; calibrated rank-4 retains 81.09%/81.49% effect | Independent cross-model hierarchy | Held-out replication |
| GQA off-target stress test | 1000 | WHOLE 1.443% vs MN 1.186% break burden; CI crosses 0 | Low collateral burden; directional advantage inconclusive | Prospective stress test |
Table 1. Headline evidence across the AROMA control stack. The table separates mechanism/controller validation from untouched prospective confirmation. The final targeted-deployment row uses a new natural cohort evaluated only after target, gate, reference, and MEA execution semantics are frozen.
Results
Low-Rank Geometry and Finite Actuation
A bidirectional causal actuator. Causal screening over 256 multimodal cross-attention heads yields seven candidates, and held-out intervention evidence motivates L18H13 as the primary actuator. On , lowers on 91.33% of examples, raises it on 90.67%, and holds on 88.67%.
The sensitivity–energy mismatch. For , the learned subspace explains 89.82% of the uncentered sensitivity second moment (86.78% after centering), far above all 10,000 rank-matched isotropic null replicates. Yet across 653 frozen non-NoOp v3 examples, its median projected activation-energy share is only (Figure 2). This mismatch predicts the failure of same- projected scaling: on the development intervention subset, naive projection reaches only 33.23%, whereas MN-CSA reaches 52.53% (+19.30 pp, McNemar ) and slightly exceeds whole-head actuation (50.995%). Magnitude normalization alone does not explain this recovery: under identical rank and displacement, Bottom-4 reaches 26.80% and 20 pre-frozen random rank-4 subspaces average 27.18% (maximum 30.78%).
Untouched confirmation. On v4, generated only after MN-CSA was frozen, MN-CSA improves accuracy from 50.25% to 58.25% (+8.00 pp, 95% CI [+6.7, +9.3] pp, McNemar ) and exceeds same-policy whole-head actuation (57.25%) by +1.00 pp (95% CI [+0.3, +1.7] pp, ), with every visual condition improving over baseline (Table 7). A four-dimensional subspace of the 128-dimensional head matches the finite control of full-head actuation under the same policy once its displacement is magnitude-matched.
| Condition | Base | Whole | MN | MN–Base | MN–Whole | |
| Dense | 400 | 45.50 | 54.00 | 55.25 | +9.75 | +1.25 |
| Distractors | 400 | 60.00 | 65.75 | 65.25 | +5.25 | -0.50 |
| Grid | 400 | 48.25 | 55.75 | 59.75 | +11.50 | +4.00 |
| Random sparse | 400 | 55.50 | 64.75 | 63.25 | +7.75 | -1.50 |
| Row | 400 | 42.00 | 46.00 | 47.75 | +5.75 | +1.75 |
Table 7. Table 7. v4 result by visual condition.
Local Response and Target Reachability
Continuous response validates the local law. On the continuous-response cohort, the median cosine between the activation- and sensitivity-aligned directions is 0.7884, so MN-CSA retains substantial but incomplete local efficiency. At , observed response versus first-order prediction has Pearson , Spearman , and 100% sign agreement (Figure 3a); correlation remains high at () and 0.20 () while magnitude error increases. The projected gradient is thus a valid local control object: what degrades is finite-step calibration, not the derivative.

Figure 3. Local projected geometry is quantitatively valid at small scale, but global target control requires iterative re-linearization. (a) Pearson agreement between first-order predicted and realized numeral movement is near-perfect at and declines as intervention scale grows. (b) Oracle MEA accumulates reachability as it recomputes the current target–competitor margin and projected gradient. (c) Matched-budget controls under the same realized per-step norms: iterative MEA, a frozen initial target-gradient direction, and activation-aligned control.
Iterative re-linearization yields high oracle reachability. On baseline-wrong examples, oracle MEA reaches the ground-truth target on 91/112 = 81.25% within 20 trust-region steps, accumulating from 5.36% after one step to 55.36% after ten (Figure 3b). Given exactly the realized MEA step norms, a frozen initial target-gradient direction reaches only 15.18% (exact paired ), and an iterative activation-aligned control only 0.89% (; Figure 3c). Therefore, path length or intervention norm cannot explain the gain, as it depends on repeatedly updating a target-specific projected margin geometry, including the active competitor.
Replication Across Heads and Models
To test whether controllability is specific to the selected head, we freeze the complete seven-head candidate pool and apply no head-specific tuning: six heads, including five of six non-primary candidates, satisfy the bidirectional criterion on a fresh cohort after Holm correction across 14 directional tests (Appendix E). Repeating the search in IDEFICS-9B-Instruct rather than transplanting Llama coordinates identifies head B2H28, whose raw Top-4 subspace explains 91.36% of sensitivity trace while carrying only 4.14% mean activation energy; with a development-selected gain (, ), low-rank actuation recovers 81.09%/81.49% of same-run whole-head mean effect on an untouched cohort (Appendix F). Low-rank causal geometry thus replicates across architectures, while finite gain is model-specific.
Routing over Direction, Gain, and Geometry
Prospective routing on procedural data. The frozen Stage-1 Proposal Controller improves untouched v3 from 49.95% to 58.05% (+8.10 pp; 173 repairs against 11 breaks), but within Stage 2 no single edit dominates: MN, SA, and WHOLE reach nearly tied peak accuracies (61.4%, 61.0%, 61.6%) at different gains, so the edit must be chosen per example. On an untouched upward-proposal population (), the frozen Candidate Utility Router improves a frozen global MN comparator () from 60.54% to 66.33% (+5.782 pp, McNemar ), with 41% fewer breaks (16 vs. 27). UGR then routes the intervention family itself: on its untouched final Stage-2 population (), it reaches 73.14% versus 70.93% for the stronger single-geometry router selected in development (+2.212 pp, McNemar ; 27:13 paired discordance) while using every intervention family (Appendix § 13).
Transfer to natural images. The frozen Top-4 geometry transfers to TallyQA without refitting (+1.40 pp over baseline for MN-CSA), but the synthetic Proposal Controller does not (−2.975 pp): the dominant error type reverses from undercounts to overcounts. Recalibrating only the Stage-1 utility map on natural examples restores utility on a disjoint 4,000-example set (64.925% → 67.050%, +2.125 pp, ). Frozen and then evaluated on the prescribed Stage-2 population of a new 8,000-example cohort (), hierarchical UGR reaches 35.341% versus 33.439% for hierarchical Whole-only routing and 31.854% for identity (+1.902 pp, McNemar ; 14:2 paired discordance), intervening on only 52/631 examples with 26 repairs against 4 breaks (Appendix § 16).
Targeted Deployment on Natural Images
A strong controller can faithfully execute a bad target policy. The first deployable target rule, , restricts candidates to numerals larger than the frozen baseline prediction. Across 623 targetable examples, MEA reaches 417 targets (66.93%), yet forced deployment loses 18.54 percentage points, with 17 repairs against 134 breaks; under the frozen gate, breaks fall to four, but only one repair remains: among the 48 gated examples with such a target, only one target equals ground truth. High reachability therefore amplifies target-policy error rather than guaranteeing utility. Natural development data then motivate the runner-up rule of Eq. (3), which is frozen; the development record and a retrospective bridge evaluation are given in Appendix § 12.
Untouched prospective confirmation closes the deployment loop. With the target rule, gate, old-Unified reference actions, and MEA protocol all frozen, we evaluate the new cohort. Frozen Stage 1 emits upward proposals, and the frozen gate fires on 63 of them. On this Stage-2 population, identity is correct on 197/630=31.270%, old Unified on 217/630=34.444%, and runner-up + MEA on 244/630=38.730% (Figure 4a). The primary paired improvement over old Unified is +4.286 pp, with 29 MEA-only versus 2 Unified-only correct examples (exact two-sided McNemar ). Relative to identity, old Unified produces 23 repairs and 3 breaks, whereas targeted MEA produces 51 repairs and 4 breaks: the improvement comes almost entirely from additional repairs rather than relaxed break control.

Figure 4. Target quality converts controllability into prospectively confirmed deployment utility. (a) Accuracy on the prespecified Stage-2 population (+4.286 pp over old Unified; 29:2 paired discordance; exact McNemar ). (b) Repairs and breaks relative to identity. (c) Correct predictions on the 63 frozen APPLY cases under the same gate; the GrAInS minimal port, the counting-adapted GrAInS-inspired comparator, and the constant-zero rule are post-confirmation controls.
Once the target is correct, execution is effectively saturated. The frozen runner-up target equals ground truth on 51/63=80.95% of APPLY examples, and MEA reaches 61/63 = 96.83% of assigned targets, including all 51/51 correct ones, in a median of 9 iterations against the 20-step horizon. On the active branch, residual error is therefore dominated by which target is selected rather than by failure to reach it.
Full-Cohort Utility and External Controls
Audited full-cohort effect under identity fallback. The Stage-2 is exactly the upward-proposal subset of the new cohort: all 630 Stage-1 and Stage-2 identity predictions agree. Keeping identity predictions outside the 63 frozen APPLY cases yields 5,254/7,992 correct (65.741%) for runner-up + MEA, versus 5,207 (65.153%) at identity and 5,227 (65.403%) for old Unified: 47 net correct predictions over identity (+0.588 pp; exact paired ), with only 0.788% of inputs receiving MEA. This accounting is derived from the frozen outcomes and establishes that positive conditional utility survives dilution over the entire evaluated natural population.
External steering and matched-gate controls. On the same AROMA-selected APPLY cases (Figure 4c), runner-up + MEA is correct on 52/63 versus 33/63 for a counting-adapted GrAInS-inspired steering comparator and 6/63 for an image-preserving GrAInS minimal port; under the identical gate, this becomes 244/630 versus 225/630 for the adapted comparator (+3.016 pp; exact paired ). Steering all 630 examples, the adapted baseline reaches 247/630 (39.206%), statistically indistinguishable from MEA (), which edits only 63 inputs: where and when to apply a controller matter as much as the steering family. The gate concentrates on zero-count overcounting: 51 of the 63 APPLY examples have true count zero, and a post-hoc rule that emits zero on the same gated outputs is correct on 51/63, compared with 52/63 for MEA (exact paired ). MEA attains this accuracy without access to that prior, selecting its target per example, and remains correct on 5/12 nonzero cases where the constant rule is correct on none. The prospective improvement thus establishes the utility of the frozen selective gate–target–controller stack; the contribution of iterative control itself is isolated by the local-response, oracle-target, and matched-budget experiments (§ 5.2; Appendix § 18).
Discussion
Mechanism discovery is not control. The combined experiments support a stricter hierarchy than “find a causal component and steer it”: sensitivity actuation energy local directional efficiency finite-step calibration global reachability target quality deployment utility. Each inequality is witnessed empirically. Sensitivity concentrates in a subspace that carries little activation energy, which magnitude normalization repairs. Projected gradients predict infinitesimal movement but lose calibration as steps grow, which re-linearization repairs. High reachability can still be harmful under a bad target rule; a frozen runner-up rule and gate then convert the same actuator into positive deployment utility. The contribution is therefore a control stack, not a single steering vector, and its evidence is reported as separate endpoints—mechanistic reachability, policy-level deployment utility, and population-level gain.
Generalization is factorized rather than binary. Actuator behavior replicates across Llama heads and the low-rank hierarchy reappears in IDEFICS, yet finite gain is model-specific; Llama’s geometry transfers to natural images, yet routing requires recalibration; and on HoloCount the action family retains +12.258 oracle headroom while the transferred policy is neutral (Appendix § 16). Off-target preservation is a separate axis: across matched GQA conditions, WHOLE breaks 1.4429% of baseline-correct conditions and MN 1.1857%, a low collateral burden for both geometries (Appendix H). These boundaries, like the failed upward-only target rule, are part of the causal argument, and they are interpretable only because discovery, development, and confirmation used disjoint data (Appendix § 19).
Conclusion
We presented AROMA, a factorized mechanism-to-control framework for frozen VLMs: a validated actuator, its low-rank sensitivity geometry, a local response law, iterative re-linearization, routing, and a separate target policy. With the target, gate, comparator, and controller frozen before outcome execution, runner-up + MEA improves old Unified by +4.286 on the Stage-2 population of a new untouched natural cohort while reaching all 51/51 correctly assigned targets. Its identity-fallback extension to the complete cohort improves 65.153% to 65.741% with 63 active interventions; under the same gate it outperforms an attribution-steering comparator by +3.016 . More broadly, interpretability becomes operational only when causal relevance, intervention geometry, finite calibration, target specification, and deployment policy are validated as distinct stages of control.
Limitations
Scope of mechanisms and architectures. Cardinality prediction is the behavioral probe throughout, and the full local-law/MEA pipeline is executed only on Llama L8H13. Multi-actuator replication establishes bidirectional finite control for 6/7 historically screened Llama candidates, but we do not claim that all replicated heads share a common low-rank basis, gain, or target-control geometry. IDEFICS independently reproduces the actuator–low-rank-geometry hierarchy, not the exact Llama head/basis identity, and the reported recovery of whole-head mean effect uses a calibrated gain rather than equal norm. Architectures that mix image and text exclusively through shared self-attention remain untested.
Scope of MEA. Oracle MEA uses ground truth only to define the target and therefore measures controllability, not deployable accuracy. The 81.25% strict success rate is the primary reachability endpoint; near-boundary failures are retained as failures. The matched-budget frozen-direction control updates neither the projected target-gradient direction nor the active competitor, so it isolates state-dependent re-linearization as a package rather than attributing the gain to a single subcomponent. We do not claim that unlimited iterations would produce 100% reachability, nor that the controller has been replicated across models.
Scope of deployable MEA v2. The runner-up rule is ground-truth-free and now receives untouched prospective confirmation, but the confirmatory targeted-control experiment is still limited to one Llama actuator and one natural counting domain. Only 63/630 Stage-2 examples are active under the frozen gate, and 51/63 of these have ground-truth count zero, where a gate-conditional zero rule performs comparably to MEA; the paired effect therefore characterizes this frozen selective deployment regime rather than a universal intervention rate or target prior. The runner-up constructor is intentionally simple and can fail when the correct answer is not the second-highest baseline numeral; its 80.95% fresh target accuracy leaves clear headroom. MEA itself has not yet been prospectively replicated across architectures or non-counting target spaces.
Scope of routing and transfer. UGR uses finite pre-specified geometry/gain families, and its prospective Stage-2 tests condition on the original upward-proposal population. The procedural and natural routers are not parameter-identical because natural development motivated a hierarchical gate-plus-ranker. On HoloCount, large oracle headroom remains unrecovered by the transferred controller and tested linear readout. The GQA stress test covers one actuator and 1,000 baseline-correct examples and is not a general safety evaluation; its directional MN advantage is not statistically conclusive.
Comparison to external methods. The prospective comparisons isolate mechanism, geometry, finite-step calibration, and deployment policy using identity, whole-head, matched-budget direction controls, and frozen restricted routers. Post-confirmation supplemental GrAInS-inspired and minimal-port comparisons add external attribution-derived steering controls on the same natural images; these comparisons are task-adapted rather than compute-matched or official GrAInS reproductions, and our claim concerns improvement under the specified frozen gate (Appendix I).
Numerical implementation. Natural MEA execution uses BF16 model inference. Exact logit ties can leave deterministic argmax at the competitor, so the frozen controller includes a boundary handling rule that takes one legal trust-radius step in the already selected projected-gradient direction when the margin is exactly zero but the target is not the argmax. This amendment changes neither the target rule, trust budget, nor 20-step horizon, but future work should test numerical robustness across precisions and model families.
Ethical Considerations
Inference-time intervention can repair specific behaviors without updating model weights, but the same capability can also introduce regressions when deployed outside validated conditions. Our results therefore argue for reporting repairs and breaks, abstention behavior, intervention magnitude, off-target break burden, finite-step diagnostics, transfer failures, and confirmation after calibration rather than accuracy gains alone. The GQA stress test provides one prospective collateral-behavior check, not a general safety guarantee; AROMA remains primarily evaluated through counting-centered mechanism and control experiments.
Appendix Roadmap, Evidence Classes, and Chronology
This appendix gives the full record behind the main text; numbers are identical to those reported there. Appendix A states the evidence classes, the experimental chronology, and a claim–evidence map. Appendix B records mechanism discovery, the sensitivity geometry, specificity controls, and the untouched v4 confirmation in Llama. Appendix C covers the local response law and MEA, including the complete targeted-deployment chain. Appendix D covers routing over direction, gain, and geometry. Appendices E and F give the multi-actuator and IDEFICS replications. Appendix G covers natural-image transfer and the HoloCount boundary, and Appendix H the off-target stress test and finite-step diagnostics. Appendix I records the post-confirmation external steering comparisons, gate-conditional zero control, and full-population identity audit. Appendix J contains an extended discussion, the scope of claims, and positioning against related work; Appendix K the reproducibility and provenance record; and Appendix L a consolidated result ledger.
Experimental Chronology and Statistics
Data chronology. For Llama, Proc-Count-Causal v1 () supports mechanism discovery, v2 () original router development, v3 () frozen whole-head confirmation and later CSA development, and v4 () is generated only after MN-CSA is closed; v4 has zero seed/pixel-hash overlap with v1–v3. Natural evaluation uses two disjoint 4,000-example TallyQA samples for transfer and Stage-1 recalibration, a later untouched 8,000-example TallyQA cohort for Natural UGR v2, and HoloCount () as an external boundary. The 8,000-example Natural UGR cohort is balanced 4,000 simple/4,000 complex and has zero question, image-ID, and pixel overlap with the earlier natural data. IDEFICS uses mutually disjoint formal cohorts: held-out actuator confirmation (), geometry estimation (), equal-norm MN-CSA (), screen (), dev validation (), and final untouched confirmation (); a further reserve remains unused.
Unified-geometry routing chronology. UGR development uses a new 2,000-example procedural cohort generated under the same renderer/distribution rules but with zero overlap against the earlier procedural cohorts. The frozen Stage-1 Proposal Controller invokes Stage 2 on 654 examples. Grouped OOF development yields 68.81% for both restricted routers; the whole-head restricted router is selected prospectively because it has fewer breaks (34 versus 37). The unified router reaches 71.87% OOF. Only after the complete UGR protocol, feature map, candidate grid, router bundles, comparator, and statistics are frozen do we generate a separate untouched 2,000-example final cohort. Its realized Stage-2 population contains 633 examples (), and no router retraining or candidate-family modification is permitted after final outcomes are observed.
Natural hierarchical UGR chronology. Natural Stage-2 development uses the upward proposals produced by the frozen natural Proposal Controller from the earlier TallyQA development population. A flat unified candidate classifier underperforms Whole-only routing (35.959% versus 38.014%), and pairwise ranking alone remains slightly below Whole (36.301% versus 36.986%). These are development results. The selected hierarchical design separates IDENTITY-vs-intervene gating from non-identity pairwise ranking and reaches 35.616% versus 34.247% for the matched hierarchical Whole comparator (+1.370 pp), with 16 versus 12 repairs and zero breaks for both. The complete hierarchical policies, 47-D feature construction, 43-candidate action space, gate threshold, and model hyperparameters are then frozen before a new 8,000-example TallyQA final cohort is evaluated. Frozen Stage 1 produces 631 upward proposals; all 43 candidates are exhaustively executed for each, yielding 27,133 candidate outcomes. No final outcome is used to refit either hierarchical router.
Prospective targeted-MEA chronology. The runner-up target rule is proposed only after diagnosing the upward-only v1 failure and is first supported on natural development OOF. Its initial v2 comparison is retained as retrospective because that population had already been inspected. The rule is then frozen without modification before a new prospective natural cohort is processed. Candidate selection and pixel deduplication are outcome-independent and completed before model inference; the final cohort contains 7,992 examples (3,996 simple/3,996 complex), zero historical question/image-ID/pixel overlap, and zero within-cohort pixel duplicates. Frozen Stage 1 yields upward Stage-2 examples. The historical hierarchical gate is replayed exactly before use (maximum probability error ; zero APPLY disagreements), then produces 63 prospective APPLY decisions. The 630 gate decisions and runner-up targets are frozen at commit 3a52e8; old-Unified reference method/gain actions are frozen separately at commit 51afb84. Only after these plans are closed are the 63 reference actions and 63 MEA trajectories executed. No 43-candidate prospective sweep, router refit, target retuning, or development update is performed.
Statistics and evidence classes. We report exact McNemar tests for paired correctness, exact directional binomial tests for sign consistency, and paired bootstrap intervals for mean shifts, retention ratios, routing accuracy differences, and off-target break burden. In the multi-actuator study, attenuation and amplification are tested separately against 0.5 directional consistency and Holm correction is applied across the 14 head-by-direction tests; a head is counted as bidirectionally replicated only when both corrected directional tests pass. The low-rank Utility Router final primary test is two-sided exact McNemar on the frozen upward population, with a 20,000-repetition paired bootstrap interval (seed 20260920). The procedural UGR final primary test compares frozen unified versus prospectively selected whole-head restricted routing on the realized untouched Stage-2 population using two-sided exact McNemar and a 20,000-repetition paired bootstrap interval (seed 20260922). The Natural UGR primary test analogously compares the frozen hierarchical Unified and Whole policies on the prespecified untouched upward Stage-2 population using two-sided exact McNemar and a paired bootstrap interval. The prospective targeted-MEA primary test compares old Unified against runner-up + MEA on the identical frozen Stage-2 population using two-sided exact McNemar; the observed discordance is 29 MEA-only versus 2 Unified-only. The GQA stress test reports the paired mean difference in break burden across seven budgets with a paired bootstrap interval; post-freeze robustness includes seed-stability checks and a 1,000,000-draw paired sign-flip randomization test. Development, post hoc diagnosis, frozen external validation, held-out confirmation, untouched confirmation, prospective robustness, and external stress tests are labeled separately.
Claim–Evidence Map
| Claim | Primary evidence | Status |
| Llama L18H13 is a bidirectional cardinality actuator | v1 intervention directionality, dose response, identity check, held-out candidate-vs-random validation | Mechanistic development + held-out validation |
| Claim | Primary evidence | Status |
| Bidirectional actuator behavior is not unique to L18H13 | frozen seven-head historical pool; fresh common protocol; 6/7 bidirectional replications, including 5/6 non-primary | Prospective multi-actuator robustness confirmation |
| Llama cardinality sensitivity is strongly low-rank | rank-4 total/centered spectra; 10,000 norm-matched null replicates | Mechanistic observation; conservative promotion criterion retained as unsatisfied |
| Naive Llama low-rank actuation under-actuates | same- failure plus label-free strength audit | Developmental diagnosis |
| Learned Llama Top-4 directions are behaviorally privileged | MN-CSA recovery, Bottom-4 control, 20 pre-frozen random rank-4 controls | Developmental specificity evidence |
| Frozen Llama MN-CSA improves unseen procedural counting | newly generated v4, zero seed/pixel overlap, frozen primary endpoint | Confirmatory |
| Llama gain/direction candidates are not globally ordered | common gain-budget audit; SA/MN matched-gain crossover on fresh and prospective cohorts | Development + prospective diagnostic |
| MN/SA/WHOLE have distinct finite-step trajectories without a stable global winner | separate matched-budget peak audit plus full matched-gain trajectories on the upward development population | Developmental geometry–gain robustness diagnosis |
| Ground-truth-free routing uses complementary mechanistic/action context | grouped-OOF feature-family ablations and feature-group permutation on the Utility Router development population | Developmental representation diagnostic |
| Frozen Utility Router improves over global MN | grouped OOF frozen external newly generated untouched final | Development + external + confirmatory |
| Low-rank geometry is not intrinsically dominant under adaptive/matched routing | post-hoc adaptive whole-head challenge plus matched-budget MN/whole diagnostics | Post-hoc robustness diagnosis |
| Unified geometry routing improves the strongest restricted geometry policy | disjoint UGR development OOF frozen whole restricted comparator untouched final | Development + prospective untouched confirmation |
| A bidirectional cardinality actuator independently replicates in IDEFICS | independent head screening followed by fresh Q7 confirmation at B2H28 | Held-out cross-model mechanism replication |
| Low-dimensional sensitivity geometry independently replicates in IDEFICS | separate geometry , raw/centered spectra, random-subspace null, unit-gradient robustness | Mechanistic replication + robustness |
| IDEFICS low-rank geometry supports finite causal control | fresh equal-norm MN-CSA with significant downward/upward sign consistency | Held-out causal confirmation |
| Gain calibration recovers most IDEFICS whole-head effect | screen dev selects , untouched final | Development + untouched confirmation |
| A synthetic router does not zero-shot transfer to natural counting | frozen TallyQA-A zero-shot run | Confirmatory negative result |
| Natural utility can be re-learned with lightweight recalibration | OOF curve on TallyQA-A, then frozen controller on image-disjoint TallyQA-B | Development + confirmatory |
| Hierarchical Natural UGR improves Whole-only Stage-2 routing | natural Stage-2 development frozen hierarchical policies new 8,000-example TallyQA cohort with untouched upward | Development + prospective untouched confirmation |
| Actuator headroom can persist after policy/readout transfer fails | TallyQA oracle and HoloCount oracle vs transferred-policy/readout failures | Post-hoc diagnostic + independent external boundary |
| Matched-budget intervention largely preserves baseline-correct non-counting behavior | frozen GQA baseline-correct population; seven paired displacement budgets; WHOLE 1.443% vs MN 1.186% aggregate break burden | Prospective off-target stress test; directional trend not conclusive superiority |
| Claim | Primary evidence | Status |
| Finite movement creates a repair–break trade-off | SA-dev response curves plus natural tertiles | Post-hoc finite-step diagnostic |
| Natural repairs concentrate in target-aligned movement | TallyQA/HoloCount active baseline-error direction analysis | Post-hoc GT-dependent diagnostic |
| Selectivity can persist when utility transfer fails | active/inactive TallyQA and HoloCount decomposition | Descriptive transfer diagnosis |
| Correct-answer loss geometry drifts at large finite gain | L18H13 finite-step gradient-drift analysis | Auxiliary local-geometry diagnostic |
Table 2. Claim–evidence matrix for the cross-model scientific master. “Confirmatory” means the reported endpoint or fixed configuration was selected before evaluation on that dataset.
Mechanism Discovery and Low-Rank Geometry in Llama
Actuator Discovery and the Stage-1 Action Space
The final method focuses on L18H13, but the discovery process provides evidence for adaptive rather than fixed mechanistic control. Exhaustive screening across 256 cross-attention heads produced seven candidates: L33H1, L3H4, L18H13, L8H30, L3H11, L13H11, and L33H21. Held-out candidate-vs-random validation gave mean interaction −0.2859 with bootstrap 95% CI [−0.4533, −0.1089] under the experiment’s sign convention. A per-sample best-head oracle over 27 held-out pairs gave mean correct-answer log-probability improvement +0.0932 (95% CI [+0.0613, +0.1269]) on baseline-wrong examples versus +0.0126 on baseline-correct examples, for specificity +0.0806 (95% CI [+0.0457, +0.1163]). Every candidate head wins on at least one sample. A multi-head equalization attempt was not robust because nonlinear interactions could cancel, motivating one stable actuator plus per-example routing. In the 300-sample characterization of L18H13, Figure 5, attenuation lowers on 274/300 examples and amplification raises it on 272/300.

Figure 5. Bidirectional cardinality steering. (a) In the 300-sample characterization, shifts the normalized expected numeral downward on 91.33% of samples, and shifts it upward on 90.67%. Strict ordering holds on 88.67%; downward and upward response magnitudes are strongly coupled (Spearman , ). (b) On Proc-Count-Causal v2, gains produce monotonic mean expected-numeral shifts, from −0.151 at to +0.439 at ; exactly reproduces baseline. These results support a bidirectional, dose-responsive cardinality actuator.
| 0 | .25 | .5 | .75 | 1 | 1.25 | 1.5 | 2 | 3 | 4 | |
| Repairs | 20 | 16 | 13 | 8 | 0 | 18 | 35 | 60 | 80 | 89 |
Table 3. Full v2 gain sweep; repairs are among the 497 baseline-wrong examples.
The full-action oracle repairs 139/497 errors and reaches 64.2% (+13.9 ). The compressed set repairs 138/497 errors and reaches 64.1% (+13.8 ), retaining essentially all oracle headroom with a smaller deployable action space.
Sensitivity Spectrum and Null Statistics
The formal rank-4 sensitivity experiment used , head dimension 128, rank , 10,000 norm-matched isotropic random-direction null replicates, and seed 20260915. Both exceeded the empirical 99th percentile and all 10,000 null replicates.

Figure 6. Frozen procedural confirmation. The synthetic router is frozen before evaluation on the untouched 2,000-example Proc-Count-Causal v3 set. (a) Accuracy increases from 999/2000 = 49.95% to 1161/2000 = 58.05% (+8.10 pp; paired 95% CI [+6.85, +9.40] pp). There are 173 repairs, 11 breaks, and +162 net repairs, with interventions on 32.65% of examples. (b) All prespecified visual conditions improve: dense, +10.25 pp; distractors, +7.25 pp; grid, +12.00 pp; random_sparse, +7.00 pp; and row, +4.00 pp. These gains support generalization beyond v2 within the procedural benchmark family.
| 1 | 2 | 4 | 8 | 16 | 32 | 64 | 128 | |
| Total second moment | 0.762162 | 0.841718 | 0.898229 | 0.944624 | 0.977195 | 0.993469 | 0.999231 | 1.000000 |
| Centered residual | 0.695283 | 0.797791 | 0.867775 | 0.927978 | 0.970304 | 0.991503 | 0.999001 | 1.000000 |
Table 4. Explained cardinality-sensitivity energy by rank.
Figure 2 contrasts sensitivity concentration with projected activation norm. The mean-gradient direction accounts for 0.232831 of total trace. For , the total null median/q95/q99 are 0.211418/0.216653/0.219041, yielding 4.249× enrichment and empirical ; centered null median/q95/q99 are 0.211493/0.216737/0.219176, yielding 4.103× enrichment and the same empirical . The original additional 5× promotion heuristic fails in both cases and is retained as a NO-GO record rather than retroactively changed. Subsequent CSA experiments are therefore presented as development followed by an independent v4 confirmation, not as a retroactive Gate-A pass.
Actuation-Strength Audit and Specificity Controls
For the 653 frozen non-NoOp v3 examples, has quantiles 0/10/25/50/75/90/100% of 0.0543, 0.2326, 0.2805, 0.3086, 0.3393, 0.3790, and 0.5102. Mean is 0.3082 and median projected energy share is 0.0952. By selected action, mean is 0.339 for , 0.442 for , 0.357 for , and 0.293 for . This audit is label-free and diagnoses an intervention-strength mismatch rather than selecting examples by outcome.
| Method | Accuracy | Repairs | Breaks | Net |
| Baseline | 26.799% | – | – | – |
| Whole-head | 50.995% | 169 | 11 | 158 |
| Same- CSA | 33.231% | 49 | 7 | 42 |
| MN-CSA Top-4 | 52.527% | 185 | 17 | 168 |
Table 5. B2 magnitude-normalization recovery on the 653 non-NoOp v3 intervention examples.
| Comparison | Accuracy | Difference vs Top-4 | 95% CI |
| Top-4 MN-CSA | 52.527% | — | — |
| Bottom-4 MN-CSA | 26.799% | −25.727 | [−29.556, −22.052] |
| Random-4 mean (20) | 27.175% | −25.352 | [−28.936, −21.792] |
| Best random-4 | 30.781% | −21.746 | — |
Table 6. Rank-matched specificity controls on the same intervention subset.
MN-CSA minus whole-head is +1.531 with paired bootstrap 95% CI [−0.766, +3.828] and McNemar ; MN-CSA minus same- CSA is +19.296 with 95% CI [+16.08, +22.665] and McNemar .
No one of the 20 frozen random rank-4 controls reaches Top-4; the empirical rank probability is 1/21 = 0.0476, the minimum attainable with 20 controls.
Figure 7. Figure 7. From low-rank sensitivity to independently confirmed control. (a) Same- rank-4 CSA severely under-actuates; magnitude normalization restores and slightly exceeds whole-head accuracy on the v3 development intervention subset. (b) The learned Top-4 subspace exceeds every one of 20 frozen random rank-4 controls; random mean is 27.18% and random maximum is 30.78%, versus 52.53% for Top-4. (c) After the CSA method is closed, a newly generated v4 confirmation set yields 50.25% baseline, 57.25% whole-head, and 58.25% MN-CSA. (d) MN-CSA obtains 174 repairs and 14 breaks versus 150 repairs and 10 breaks for whole-head, producing 20 additional net repairs.
Untouched v4 Confirmation
| Count | Fraction | |
| 0 | 27 | 1.35% |
| 1 | 1396 | 69.80% |
| 1.5 | 32 | 1.60% |
| 2 | 47 | 2.35% |
| 4 | 498 | 24.90% |
Table 8. Table 8. v4 action distribution of the frozen synthetic router.
| Count | Base | Whole | MN | Whole–Base | MN–Base | MN–Whole |
| 1 | 1.000 | 1.000 | 1.000 | 0.0 | 0.0 | 0.0 |
| 2 | .995 | 1.000 | 1.000 | +0.5 | +0.5 | 0.0 |
| 3 | .880 | .960 | .940 | +8.0 | +6.0 | -2.0 |
| 4 | .750 | .860 | .850 | +11.0 | +10.0 | -1.0 |
| 5 | .515 | .690 | .730 | +17.5 | +21.5 | +4.0 |
| 6 | .425 | .545 | .565 | +12.0 | +14.0 | +2.0 |
| 7 | .135 | .300 | .360 | +16.5 | +22.5 | +6.0 |
| 8 | .145 | .155 | .155 | +1.0 | +1.0 | 0.0 |
| 9 | .125 | .120 | .135 | -0.5 | +1.0 | +1.5 |
| 10 | .055 | .095 | .090 | +4.0 | +3.5 | -0.5 |
Table 9. Table 9. v4 result by ground-truth count. Accuracies are fractions; differences are percentage points.
The primary frozen comparison is MN-CSA versus baseline: +8.00 , 95% CI [+6.7, +9.3] , exact McNemar . MN-CSA versus whole-head is a prespecified secondary comparison: +1.00 , 95% CI [+0.3, +1.7] , exact McNemar . The latter represents a modest, statistically supported gain.
Local Response Law and Minimal-Effort Actuation
This section records the principled extension that connects the original AROMA geometry/routing pipeline to explicit target-directed control. The experiments preserve the frozen Llama model revision, causal actuator L18H13, and Top-4 basis. Evidence classes are stated separately because the extension contains local mechanism validation, oracle control, a deliberately retained deployment failure, development target-policy validation, a retrospective bridge evaluation, and a final untouched prospective confirmation on a new natural cohort.
E1: Continuous Local-Response Validation
For each example, let , , and . The formal run evaluates continuous numeral movement under matched-norm activation- and gradient-aligned directions. The median is 0.788441, with 25th/75th percentiles 0.73356/0.84122; only 1/160 examples has non-positive cosine in the formal continuous archive. At , the regression of realized on first-order predicted movement has slope 1.0573, intercept , Pearson , Spearman , MAE .001313, RMSE .003178, and 100% sign agreement. At and .20, Pearson correlation remains .9769 and .9371, while magnitude error increases. A four-example finite-difference check using relative steps .01 and .02 yields 100% sign agreement and sub-percent median relative error, independently validating the implemented derivative.
| Slope | Pearson | Spearman | MAE | Sign agreement | |
| 0.05 | 1.0573 | 0.9936 | 0.9990 | 0.001313 | 100% |
| 0.10 | 1.0994 | 0.9769 | 0.9964 | 0.005227 | 100% |
| 0.20 | 1.1000 | 0.9371 | 0.9873 | 0.0194 | 100% |
Table 10. Continuous local-response validation. The local law remains directionally reliable as scale grows, but magnitude calibration degrades.
Oracle MEA Protocol and Horizon
Oracle MEA sets the target to ground truth only for the purpose of measuring controllability. At each step it identifies the current highest non-target numeral, computes the target–competitor logit margin, projects the margin gradient into the frozen Top-4 subspace, takes the minimum predicted crossing step clipped to , and re-linearizes. The primary 20-step run contains 112 baseline-wrong examples and no projected-gradient abstentions.
| Max steps | 1 | 2 | 3 | 5 | 10 | 15 | 20 | |
| Success (%) | 5.36 | 10.71 | 12.50 | 20.54 | 55.36 | 75.89 | 81.25 |
Table 11. Cumulative oracle MEA target reachability.
The strict primary success count is 91/112. Of the 21 unresolved examples, nine finish within margin of the target boundary, but these are not promoted to successes. Failures are associated descriptively with larger initial effort, sustained clipping, and more active-competitor changes; no causal claim is made from these associations. In BF16, an exact zero margin can still leave deterministic argmax at the competitor; in that boundary case MEA takes one legal trust-radius step in the same projected gradient direction rather than introducing a new clearance threshold.
Matched-Budget Controls
The frozen-direction control computes the initial target-specific projected margin direction once and then receives the exact realized MEA step norms. It reaches 17/112=15.18% versus 91/112=81.25% for MEA; the paired table is 17 both, 74 MEA-only, 0 frozen-only, and 21 neither (exact ). An iterative activation-aligned control using the same realized budget reaches 1/112=0.89%; the paired table is 1 both, 90 MEA-only, 0 activation-only, and 21 neither (exact ). These controls show that target-specific state-dependent re-linearization is necessary under the tested budget; they do not imply that activation geometry is useless for other steering objectives.
Deployable Target-Policy Chronology
The first deployable protocol used
with eight natural abstentions at the maximum numeral. On 623 targetable examples, MEA reaches 417 assigned targets (66.93%) with zero projected-gradient abstentions. Despite this, the archived forced-deployment replay loses 18.542 percentage points and yields 17 repairs against 134 breaks. Retaining the old frozen Unified gate reduces the intervention count to 52/631 and breaks to four, but the v1 controller produces only one repair. In the gated cohort, equals ground truth on only 1/48 targetable examples, and on none of the 24 targetable old-router repairs.
The runner-up rule in Eq. (3) was then evaluated on development OOF data before the v2 controller run. Under the unified hierarchical family, 30/292 development examples are APPLY decisions; the runner-up target matches ground truth on 24/30=80.0% and on 15/16=93.75% of old-router repairs. For the Whole-only hierarchical family, the corresponding rates are 17/21=80.95% and 10/12=83.33%. Moreover, 28/30 development APPLY errors require a downward correction even though Stage 2 was invoked by an upward proposal. This development result is the basis for freezing target policy v2 (commit 3c065cb).
Retrospective Final-Population MEA v2
The v2 runner leaves the Unified APPLY/NOOP gate frozen and executes MEA only on the 52 APPLY examples. The remaining 579 retain the identity prediction. Table 12 records the paired end-to-end result.
| Policy | Correct | Accuracy | vs ID | Repairs | Breaks |
| Identity | 201 | 31.854% | – | – | – |
| Old frozen Unified | 223 | 35.341% | +3.487 | 26 | 4 |
| Runner-up + MEA v2 | 234 | 37.084% | +5.230 | 38 | 5 |
Table 12. Retrospective final-population targeted-deployment comparison on the frozen natural Stage-2 cohort.
Versus identity, v2 has exact McNemar . Directly against old Unified, 15 examples are v2-only correct and four are old-only correct, giving +1.743 and exact paired . On the 52 APPLY examples, the deploy target equals ground truth on 37/52=71.15%, 50/52=96.15% assigned targets are reached, all 37/37 correct targets are reached, there are zero projected-gradient abstentions, median iterations are 7, and mean iterations are 7.365.
Untouched Prospective MEA-v2 Confirmation
After the runner-up rule is frozen, a separate natural cohort is finalized before outcome execution, with 3,996 simple and 3,996 complex questions, zero historical question/image-ID/pixel overlap, and zero within-cohort pixel duplicates. Frozen Stage 1 yields 630 upward proposals. The unchanged hierarchical gate fires on 63; both the runner-up target plan and the old-Unified reference method/gain plan are frozen before the 63 outcome pairs are executed.
| Policy | Correct | Accuracy | vs ID | Repairs | Breaks |
| Identity | 197 | 31.270% | – | – | – |
| Old frozen Unified | 217 | 34.444% | +3.175 | 23 | 3 |
| Runner-up + MEA v2 | 244 | 38.730% | +7.460 | 51 | 4 |
Table 13. Untouched prospective targeted-deployment confirmation on the frozen natural Stage-2 population.
Directly against old Unified, the prospective controller is uniquely correct on 29 examples and old Unified on two, yielding +4.286 and exact two-sided McNemar . The runner-up target equals ground truth on 51/63=80.95% of APPLY examples; 61/63=96.83% assigned targets are reached, including all 51/51 correct targets. Median/mean MEA iterations are 9/8.238, projected-gradient abstentions are zero, and the live baseline reproduces the archived baseline on all 63 APPLY examples. The prospective result therefore upgrades the target-directed branch from a retrospective diagnosis to an untouched deployment confirmation. Frozen Stage 1 attains 65.153% full-cohort baseline accuracy on this cohort, and the gate fires on 63/630=10.0% of the upward proposals. Relative to identity, old Unified nets +20 examples and targeted MEA nets +47. The prospective effect is numerically larger than the earlier retrospective +1.743 estimate; we do not treat the cross-cohort difference itself as an inferential comparison.
Estimands and comparator fairness. The final old-Unified-versus-MEA experiment estimates deployment utility under two frozen post-gate policies; it is not an equal-norm or equal-compute efficiency comparison. Both policies receive the same frozen gate and therefore act on the identical APPLY set, but old Unified executes its frozen ranked geometry/gain action whereas MEA executes a frozen explicit target with iterative re-linearization. Their realized intervention paths can consequently differ by design. Questions about whether re-linearization itself is useful under matched intervention magnitude are answered separately by the oracle-MEA controls in § 5.2, where frozen-direction and activation-aligned baselines receive the exact realized MEA step norms. We therefore interpret the prospective +4.286 result as a policy-level utility improvement, not as a claim of superior accuracy per unit intervention norm or per unit computation.
Importantly, denotes the historical Stage-1 intervention proposal that triggers Stage 2 and defines a reference actuation budget; it is not a constraint that the final numeral prediction must increase. Nonlinear class transitions under MN, SA, WHOLE, or MEA can move the predicted label in either direction. This distinction is essential on natural data, where most development APPLY errors require downward correction even though they arose from an upward Stage-1 proposal.
Routing over Direction, Gain, and Geometry
Direction and Gain Candidates
The historical v4 comparison uses the already-frozen proposal controller and the then-fixed intervention convention. To separate geometry from gain, we later run a development-only common gain-budget audit on the 629 v3 upward-action examples. At reference strength , whole-head accuracy is 52.78% and magnitude-normalized low-rank accuracy is 54.37%. Independent tuning changes the ordering: whole-head peaks at 64.07% for , while MN peaks at 62.80% for . The correct interpretation is therefore not that low-rank actuation globally dominates the full head, but that gain is a separate calibration variable.
A fresh SA-dev cohort () then compares activation-aligned MN and sensitivity-aligned SA inside the same frozen Top-4. The projected directions have median cosine 0.7898; 44/500 (8.8%) are non-positive. On the separately generated prospective cohort, matched-gain accuracy differences SA−MN are +1.8 at (9:0 paired discordance, exact ), +1.8 at (11:2, ), and −1.8 at (0:9, ). This replicated crossover is the main reason both directions are retained as candidates rather than selecting one globally.
A simpler tempered sensitivity-gain rule is also tested prospectively. Although sensitivity strongly predicts the first gain at which some errors become repairable on development data, the frozen rule reaches 59.6% on the prospective cohort, below both separately selected MN (61.0%) and SA (61.6%). This negative prospective result is retained because it motivates candidate utility prediction: response threshold is informative, but it is not equivalent to correctness utility.
A separate matched-budget audit compares the best observed MN, SA, and WHOLE finite-step accuracies under the same candidate budget. The three peak accuracies are nearly tied, but their optimal gains differ.
| Comparison | Peak accuracies | Difference | 95% CI |
| WHOLE − MN | 61.6% vs 61.4% | +0.2 | [−1.4, +1.8] |
| WHOLE − SA | 61.6% vs 61.0% | +0.6 | [−1.0, +2.4] |
| MN − SA | 61.4% vs 61.0% | +0.4 | [0, +1.0] |
Table 14. Matched-budget peak comparison on the separate geometry audit. Confidence intervals are paired peak-to-peak descriptive intervals; they do not define a winner.
The routing-relevant upward-proposal development subset provides the full common-grid geometry trajectory. Gains are 0.25, 0.5, 0.75, 1, 1.25, 1.5, 1.75, 2, 2.25, 2.5 in column order below.
| .25 | .50 | .75 | 1.00 | 1.25 | 1.50 | 1.75 | 2.00 | 2.25 | 2.50 | |
| MN | 36.88 | 40.00 | 45.00 | 54.38 | 56.88 | 59.38 | 61.25 | 63.75 | 63.75 | 60.63 |
| SA | 38.13 | 40.63 | 52.50 | 59.38 | 61.25 | 61.88 | 62.50 | 59.38 | 58.75 | 55.63 |
| WHOLE | 35.00 | 38.75 | 44.38 | 50.63 | 53.13 | 56.88 | 61.88 | 62.50 | 64.38 | 63.13 |
Table 15. Matched-gain exact accuracy on the upward-proposal development population. The crossover and non-monotonic tails motivate geometry/gain routing rather than a global geometry claim.
Candidate Utility Router
Stage 2: Candidate Utility Router. For an upward proposal from the frozen Proposal Controller, the Candidate Utility Router chooses among abstention and a bounded family of low-rank direction/gain interventions. The frozen candidate set is
where
The identity candidate applies no second-stage displacement, . Thus, each upward proposal has 15 possible second-stage decisions: identity plus 14 direction/gain interventions.
For each candidate , a frozen candidate-conditioned logistic model estimates candidate correctness,
where contains only information available without ground truth at inference time.
The sample representation contains the 39 frozen Proposal Controller features, the proposal action and proposal score, and six transformed mechanistic features: log projected-gradient norm, log pooled-head norm, log SA sensitivity, projected-gradient/projected-activation cosine, gradient capture, and activation capture. Eight candidate channels encode method, gain, gain squared, and method–gain combinations ( low-rank interventions plus identity); interactions with the 47 sample features produce the frozen 431-dimensional logistic design. Training is grouped by sample so candidate rows from one example never split across folds.
The deployed second-stage decision is
Deterministic ties are resolved in favor of identity, then smaller gain, then MN before SA.
Ground truth is used only to fit and evaluate the Candidate Utility Router; it is never used to construct inference-time features or to select a candidate at deployment. Because Stage 2 is invoked only when , all formal Candidate Utility Router results are conditional on the corresponding upward-proposal population. Accordingly, the final routing experiment evaluates second-stage refinement of frozen upward proposals rather than end-to-end routing over all examples and intervention directions.
Candidate Utility Router development and frozen external validation. The Candidate Utility Router is developed only on the 160 upward-proposal examples selected by the frozen Proposal Controller from the SA-dev cohort. Grouped five-fold out-of-fold evaluation keeps all candidate rows from the same example in the same fold.
The fold-selected global candidate reaches 62.50% accuracy, whereas the Candidate Utility Router reaches 67.50%, a +5.00 pp improvement with paired bootstrap 95% CI [+1.25, +9.375] pp and exact McNemar .
The Utility Router feature-family audit is performed only on grouped OOF development. It is interpretive rather than confirmatory: no ablation is used to refit or modify the frozen external/final policy.
| Feature configuration | OOF accuracy |
| FULL | 67.50% |
| NO_INTERNAL | 66.875% |
| NO_OUTPUT_GEOMETRY | 65.625% |
| NO_PROPOSAL | 65.00% |
| BASELINE+PROPOSAL | 65.00% |
| OUTPUT_GEOM+PROPOSAL | 65.00% |
| INTERNAL_MECH+PROPOSAL | 63.125% |
Table 16. Grouped-OOF Candidate Utility Router feature-family ablation on the development population.
A complementary feature-group permutation audit produces accuracy drops of 14.481 pp for baseline-state features, 4.525 pp for proposal context, 4.223 pp for internal-mechanistic features, and 3.948 pp for output-geometry features. The baseline state is the strongest individual information source, but every mechanistic/action-context group contributes measurable complementary information to the mature representation.
After model fitting and candidate design are frozen, the router is evaluated on a separately generated prospective cohort containing 156 upward-proposal examples. The frozen global MN–CSA comparator reaches 66.67%, while the Candidate Utility Router reaches 71.15%, a +4.487 pp difference with bootstrap 95% CI [+0.641, +8.974] pp. The corresponding exact McNemar test gives . Because this cohort had previously supported a different prospective sensitivity-only experiment, we treat this stage as frozen external validation, not pristine confirmation.
| Stage | Global MN | Router | Evidence class | ||
| SA-dev grouped OOF | 160 | 62.50% | 67.50% | +5.00 | Development / OOF |
| Prospective frozen external | 156 | 66.67% | 71.15% | +4.487 | Frozen external |
| Utility-final untouched | 294 | 60.54% | 66.33% | +5.782 | Confirmatory |
Table 17. Utility Router evidence chain. The final row is the only pristine confirmatory evaluation. Accuracy is measured on the prespecified upward-action population of each stage.
Untouched upward-proposal confirmation. Before the final cohort is generated, the Proposal Controller, candidate family, feature map, Candidate Utility Router, frozen global comparator, primary endpoint, and statistical test are all fixed. A new procedural cohort is then generated with zero seed and exact image-hash overlap with the earlier procedural, SA-dev, and prospective cohorts. The frozen Proposal Controller produces 294 upward proposals, defining the prespecified primary population for the second-stage confirmation.
On these examples, the frozen global MN-CSA comparator reaches , whereas the frozen Candidate Utility Router reaches . The paired improvement is , with paired bootstrap 95% CI and two-sided exact McNemar . The frozen proposal controller’s 294 upward examples comprise 12 with , 24 with , and 258 with ; candidate outcomes are generated for all 14 non-identity candidates and frozen before router evaluation, yielding 4,116 candidate rows.
The improvement reflects both additional repairs and fewer destructive interventions. Relative to identity, the frozen global comparator produces 132 repairs and 27 breaks, whereas the Candidate Utility Router produces 138 repairs and 16 breaks. All 17 paired discordant outcomes favor the Candidate Utility Router (17 router-only versus 0 comparator-only).
The final policy also does not collapse to one intervention. Across the 294 upward proposals, it selects identity on 60 examples, MN on 134, and SA on 100, and uses every nonzero gain in the frozen candidate grid. Together with the preceding direction–gain crossover, this untouched result supports the bounded policy claim: even after a coarse cardinality action has been proposed, deciding whether, along which direction, and how strongly to intervene remains an example-dependent utility problem.
| Policy | Correct | Accuracy | Repairs | Breaks |
| Identity | 73 | 24.83% | – | – |
| Frozen global MN | 178 | 60.54% | 132 | 27 |
| Frozen Utility Router | 195 | 66.33% | 138 | 16 |
| Candidate oracle | 217 | 73.81% | – | – |
Table 18. Untouched Utility Router final confirmation on the prespecified upward population.
Relative to the global comparator, the router captures 17 of the 39 available oracle-headroom examples, or 43.6% of the remaining gap to the candidate oracle. Gain selections are (identity) 60, 0.5: 29, 0.75: 7, 1: 16, 1.25: 20, 1.5: 18, 1.75: 22, and 2: 122.
Whole-Head Robustness and Matched-Budget Geometry Diagnostics
The original Utility Router confirmation is intentionally interpreted within its frozen low-rank candidate family. After that final population had already been observed, we ran a separate diagnostic in which whole-head intervention received its own per-example gain router trained only on SA-dev development data. The frozen adaptive whole-head router reaches , compared with for the original low-rank router. The post-hoc difference is with paired bootstrap 95% CI and exact McNemar (20 whole-only versus 7 low-rank-only outcomes). The adaptive whole protocol produces 150 repairs and 15 breaks versus 138 repairs and 16 breaks for the low-rank router. Because the final population had already been used in prior AROMA analysis, this comparison is a robustness diagnostic and is never described as pristine confirmation.
To separate geometry from candidate budget and router capacity, a second post-hoc experiment gives MN and whole-head families identical eight-candidate budgets (identity plus seven gains), identical 47 sample features, identical 191-dimensional generic candidate-conditioned router design, and matched displacement norm . The result does not yield a stable intrinsic geometry ranking:
| Population | MN | Whole | 95% CI | ||
| Development OOD | 160 | 66.25% | 62.50% | ||
| Final diagnostic | 294 | 65.65% | 66.67% |
Table 19. Matched-budget MN versus whole-head diagnostic. Positive denotes Whole–MN. Neither population supports a stable geometry ordering.
Exact McNemar -values are 0.2379 and 0.6476, respectively. The adaptive-whole advantage therefore cannot be interpreted as proof that full-head geometry is intrinsically superior; geometry, gain range, and routing interact. This diagnostic directly motivates prospective unified geometry routing.

Figure 8. (a) Routing advantage across evaluation stages. (b) Final repairs increase while breaks fall. (c) The router uses all three decision types. (d) Adaptive gain selection. Paired wins: 17 router-only / 0 global-only. Utility-aware routing confirms adaptive control within the frozen low-rank candidate family. (a) The router improves over the fold-selected global comparator in grouped OOF development and the frozen global comparator in external validation and the untouched final population; only the last stage is the primary confirmatory test. (b) On the final , the advantage combines six additional repairs with eleven fewer breaks, yielding 17 router-only versus 0 global-only paired wins. (c–d) The learned policy does not collapse to a single method or gain: it selects identity, MN, and SA and uses every nonzero gain in the frozen grid. Final primary effect: , 95% CI , exact McNemar .
Unified Geometry Router
UGR uses the frozen Stage-1 Proposal Controller and invokes Stage 2 only for . For each invoked sample, the frozen candidate set is identity plus MN, SA, and WHOLE crossed with gains , giving 43 candidates per sample. The 47 sample features are the 39 frozen Proposal Controller features, selected proposal action/score, and six mechanistic summaries. Nine candidate channels plus their interactions yield 479 design features. All routers use StandardScaler plus L2 LogisticRegression with grouped five-fold splitting by sample. MN and SA retain the frozen Top-4 directions, with displacement magnitude for gain in the grid
For WHOLE, the matched finite-gain rule is , so controls the same proposal-relative scaling variable across geometry families. The design therefore has dimensions, and deterministic ties are broken toward IDENTITY, smaller gain, MN, SA, then WHOLE.
| Stage / policy | Correct | Accuracy | Repairs | Breaks | Oracle | |
| Dev low-rank restricted | 654 | 450 | 68.81% | 290 | 37 | 80.58% |
| Dev whole restricted | 654 | 450 | 68.81% | 287 | 34 | 80.58% |
| Dev unified | 654 | 470 | 71.87% | 305 | 32 | 83.18% |
| Final identity | 633 | 195 | 30.81% | – | – | – |
| Final whole restricted | 633 | 449 | 70.93% | 282 | 28 | 80.88% |
| Final unified | 633 | 463 | 73.14% | 298 | 30 | 84.36% |
Table 20. UGR development OOF and untouched final results. The selected restricted comparator is whole-head restricted because restricted accuracies tie in development but whole-head has fewer breaks.
The development unified-minus-selected-restricted difference is +3.058 . On untouched final data, unified-minus-whole is +2.212 with 95% paired bootstrap CI [+0.316, +4.265] , exact McNemar = 0.0384773, and discordance 27 unified-only versus 13 whole-only. The archived final selection distribution is WHOLE 311, MN 214, SA 43, and IDENTITY 65. These frequencies are descriptive and are not used to attribute the paired UGR gain to any individual geometry family. The full-head restricted router selects WHOLE on 549 and IDENTITY on 84. Complete per-gain selection distributions are archived rather than expanded into the main paper; the final frozen grid is used without post-hoc extension.
The final candidate generator completed all 633 samples (27,219 candidate rows) before a stale development-size family-count validator terminated the process. All 633 complete 43-candidate records were already written and were audited for exact sample order, exact candidate order, and exact family counts before deterministic flattening into the final candidate-outcome table; no inference was rerun and no outcome, feature, router, comparator, or candidate family was modified.

Figure 9a. Triangles are candidate-oracle upper bounds. Dev = +3.06 ; final = +2.21 .

Figure 9b. R/U denote restricted/unified. Selection shares are descriptive: they show that the unified policy does not degenerate to a single family.
Figure 9. Prospective unified geometry routing improves the strongest development-selected restricted policy. (a) Development OOF and untouched-final exact accuracy for the prospectively selected full-head restricted comparator and UGR; triangles show candidate-oracle upper bounds. The paired final gain is +2.212 (95% CI [+0.316, +4.265] ; exact McNemar = 0.0385), with 27 unified-only versus 13 restricted-only correct examples. (b) Selection shares show that the unified router uses the prespecified intervention families non-degenerately. These shares are descriptive rather than an attribution of the paired improvement; the primary claim is the frozen comparison in (a).
E Multi-Actuator Replication in Llama
The original geometry/routing pipeline deliberately concentrates on L18H13, but this leaves open whether its finite controllability is an idiosyncratic property of the selected head. The multi-actuator experiment revisits the complete historical Llama candidate pool after the primary L18H13 mechanism/control pipeline has already been established. The seven coordinates are L33H1, L3H4, L18H13, L8H30, L3H11, L13H11, and L33H21. All seven are evaluated on the same fresh cohort with and no head-specific tuning. Baseline outputs are identical across runs. The raw outcome archive contains 900 rows per head and is frozen before the primary endpoint is computed.
Six of seven candidates satisfy the prespecified bidirectional replication criterion after Holm correction across 14 directional tests: L33H1, L3H4, L18H13, L3H11, L13H11, and L33H21. This includes five of the six non-primary candidates. L8H30 does not replicate: attenuation consistency is 50.67% and amplification consistency is 47.00%, both near chance and in the wrong mean-shift regime. In contrast, the replicated heads span a wide effect-strength range, from L3H11 (64.0% attenuation; 57.67% amplification) to L18H13 (91.67%; 92.67%). Thus the actuator phenomenon is not unique to L18H13, while historical candidacy alone does not guarantee stable finite control.
| Head | Att. | Amp. | Strict | Mean shift (D/U) | Rep. | ||
| L33H1 | 85.33% | 86.00% | 76.67% | yes | |||
| L3H4 | 79.67% | 69.67% | 56.67% | yes | |||
| L18H13 | 91.67% | 92.67% | 89.00% | yes | |||
| L8H30 | 50.67% | 0.8625 | 47.00% | 0.8637 | 24.67% | no | |
| L3H11 | 64.00% | 57.67% | 0.0139 | 40.00% | yes | ||
| L13H11 | 71.33% | 72.67% | 49.67% | yes | |||
| L33H21 | 61.00% | 67.33% | 40.00% | yes |
Table 21. Fresh-cohort D1 multi-actuator endpoint. “Att.” and “Amp.” are directional consistencies for and ; reported -values are Holm-adjusted across all 14 head-by-direction tests. Replication requires both adjusted directional tests to pass.
The endpoint yields 6/7 replicated heads and 5/6 replicated non-primary heads. The result should not be read as an unbiased prevalence estimate over all 256 cross-attention heads because the seven-head pool was historically screened. Its role is narrower and stronger: the primary L18H13 phenomenon survives a prospectively fixed within-model robustness check across the complete historical candidate set, while L8H30 supplies a negative control against treating candidacy as guaranteed finite controllability. The D1 inference runner is frozen at commit c9f7d15; seven-head raw outcomes and manifests/multi_actuator_d1_v1/raw_outcomes_freeze_v1.json are frozen at commit 5f57ab6. Primary endpoint artifacts are outputs/proc_count_multi_actuator_d1_v1/analysis/primary_endpoint_v1.csv and primary_endpoint_v1.json.

Figure 10. Bidirectional actuator behavior generalizes across the frozen historical candidate pool. (a) Attenuation () and amplification () directional consistency for all seven candidates on the same fresh cohort; the dashed line marks 0.5. Six heads pass both one-sided exact directional tests after Holm correction across the 14 head-by-direction tests. (b) The stricter per-example ordering varies substantially across heads and is not itself the replication criterion. L8H30 provides a useful negative case: historical causal candidacy does not automatically imply stable finite controllability.
F IDEFICS Cross-Model Replication
Actuator and Scoring
Cross-model experiments use HuggingFaceM4/idefics-9b-instruct at revision 7065afb8a5bb03015700b315f6751dc3dd4e16eb. The model exposes eight gated cross-attention blocks with 32 heads per block and 128-dimensional head slices, for 256 screened cross-attention heads. Formal intervention is applied to the B2H28 slice at the input to the block-2 cross-attention output projection. Because decimal numeral strings are multi-token under the IDEFICS tokenizer, formal expected numeral is computed from complete teacher-forced sequence log-likelihoods over strings 0–15, including the end-of-utterance terminator; token-length normalization is not used.
The held-out Q7 actuator confirmation contains examples, balanced with 30 examples for each count 1–10. For B2H28, attenuation gives 246/300 downward shifts (82.00%), amplification gives 248/300 upward shifts (82.67%), and strict ordering holds on 203/300 (67.67%). The corresponding one-sided exact binomial -values are and , both Holm-significant. A pre-registered auxiliary “strong” threshold required at least 70% strict ordering; the observed 67.67% therefore misses that auxiliary threshold by 2.33 percentage points while still satisfying the protocol’s bidirectional replication criterion.
IDEFICS Sensitivity Geometry and Robustness
On a disjoint geometry cohort (), the raw uncentered Top-4 explains 0.913626 of sensitivity trace (, ), and the centered Top-4 explains 0.839657 (, ). The raw mean-gradient direction accounts for 0.559881 of trace. Against 10,000 random rank-4 subspaces, no null replicate reaches the raw or centered observed concentration (empirical for each). The mean projected activation-energy share of the raw Top-4 is only 0.041375, with mean , again separating sensitivity concentration from activation energy.
| Model / estimand | Raw Top-4 | Raw centered | Unit Top-4 | Unit centered |
| Llama L18H13 | 0.898229 | 0.867775 | 0.835115 | 0.650435 |
| IDEFICS B2H28 | 0.913626 | 0.839657 | 0.899015 | 0.784893 |
Table 22. Raw and unit-gradient sensitivity robustness in the two model-specific actuator coordinates. Unit normalization changes the estimand by giving every example equal gradient norm; it is therefore reported as robustness, not as a replacement for the frozen raw analysis.
For Llama, the raw mean-gradient trace share is 0.232831 and the mean unit-gradient share is 0.563789; for IDEFICS the corresponding values are 0.559881 and 0.614526. Thus the large raw cross-model difference in shared-direction concentration is substantially reduced after equalizing per-example gradient magnitude. We do not interpret the normalized analysis as proving that gradient magnitude “causes” the raw difference; it shows that the contrast is estimand-sensitive. The leading subspaces remain strongly low-dimensional, but exact basis membership is not invariant under normalization, particularly in lower modes.
A separate activation-scale audit finds B2H28 median pooled norm 0.457, ranking 214/256 globally by median norm but 18/32 within cross-attention block B2; its median norm is about 0.89 times the per-image 256-head median. The head is therefore globally low-scale but locally typical within its block rather than near-dead. Because projected energy is normalized within the head, this scale audit does not itself explain the 4.14% Top-4 activation-energy share.
IDEFICS Equal-Norm MN-CSA Confirmation
The primary cross-model finite-control test uses the frozen raw uncentered Top-4 and the same magnitude-matched operator as Eq. (1) on a fresh cohort. Table 23 summarizes the result.
| Metric | Down | Up | Strict |
| MN-CSA consistency | 65.33% | 66.00% | 37.67% |
| Whole-head consistency | 85.33% | 88.00% | 74.00% |
| Mean MN-CSA shift | 0.01336 | 0.01269 | – |
| Mean whole-head shift | 0.03300 | 0.03702 | – |
| Effect retention | 40.47% | 34.28% | – |
| One-sided exact |
Table 23. IDEFICS equal-norm rank-4 MN-CSA on a fresh cohort. “Retention” is the ratio of mean MN-CSA directional shift to the same-run whole-head directional shift.
Both directional tests remain significant after Holm correction. This stage establishes partial causal transfer at matched displacement magnitude before any gain calibration is introduced.
IDEFICS Rank and Gain Development Record
The v2 development sequence is completely disjoint from the equal-norm confirmation and from the final confirmation. A screen cohort uses replicates 18–19 (); a development-validation cohort uses replicates 20–23 (); the untouched final cohort uses replicates 24–29 (); replicates 30–39 () remain unused reserve data. All four splits have zero overlap with prior IDEFICS formal cohorts.
Table 24. IDEFICS rank screen at on development-only screen100. Sensitivity share rises monotonically, whereas balanced finite causal shift peaks at rank 4.
| Sens. share | Down cons. | Up cons. | Balanced shift | Down/Up retention | |
| 2 | 85.32% | 62% | 65% | 0.01289 | 40.69% / 40.50% |
| 4 | 91.36% | 63% | 68% | 0.01417 | 44.71% / 43.37% |
| 8 | 96.12% | 63% | 63% | 0.00898 | 28.35% / 39.73% |
| 16 | 98.36% | 61% | 61% | 0.00845 | 26.66% / 25.79% |
Table 24.
The top two ranks () then enter a bounded gain screen with . The predeclared primary ranking metric is balanced mean shift, with minimum directional consistency and strict ordering as secondary criteria. The top two configurations are () and (). On the independent dev200 set, the ordering reverses: () has balanced shift 0.02344 versus 0.02275 for rank 2 and is selected for final confirmation. Its dev200 directional consistency is 73.0%/74.0%, with 73.69%/73.62% effect retention. No further parameter search is performed after this selection.
IDEFICS Untouched Gain-Calibrated Final Confirmation
The final cohort contains examples from replicates 24–29 and is first evaluated only after is selected on dev200. The frozen raw Top-4 basis explains 91.36% of raw sensitivity trace.
Table 25. IDEFICS untouched final confirmation for the fixed gain-calibrated rank-4 intervention ().
| Metric | Down | Up |
| MN directional consistency | 79.67% | 79.67% |
| Whole-head directional consistency | 84.00% | 84.33% |
| MN mean shift | 0.02555 | 0.02880 |
| Whole-head mean shift | 0.03151 | 0.03534 |
| Effect retention | 81.09% | 81.49% |
| Retention 95% CI | [71.21, 92.15]% | [71.90, 92.15]% |
| Directional exact | ||
| MN–whole mean difference | −0.00596 | −0.00654 |
| MN–whole 95% CI | [−0.00969, −0.00230] | [−0.01063, −0.00257] |
Table 25.
Strict ordering holds on 187/300 = 62.33%, versus 211/300 = 70.33% for whole-head intervention. Both directional exact tests survive Holm correction. The gain-calibrated edit therefore closes most of the mean-effect gap but does not eliminate it. Because , these retention ratios quantify recovered whole-head behavioral effect under a stronger low-rank displacement, not equal-norm efficiency. The fixed low-rank controller is directionally consistent on 239/300 examples in each direction, and the paired mean MN-minus-whole differences remain negative with CIs excluding zero, so the result is substantial recovery, not parity. The rank screen also rules out a simple “capture more spectral mass” explanation and motivates treating finite control as a separate behavioral property (Figure 11). The clean cross-model conclusion is therefore structural: low-rank causal geometry replicates, while finite gain remains model-specific.

Figure 11. Cross-model replication and held-out gain calibration in IDEFICS-9B-Instruct. (a) Independently screened B2H28 shows held-out bidirectional actuator behavior. (b) Sensitivity trace rises monotonically with rank (point labels), but finite causal steering peaks at rank 4 and deteriorates at ranks 8 and 16. (c) On the development-only screen set, larger gain substantially increases balanced low-rank steering; only the top two configurations are carried forward. (d) Independent dev validation selects , before the final cohort is touched. On the untouched final set, the fixed low-rank intervention reaches 79.67% directional consistency in both directions and recovers about 81% of same-run whole-head mean shift. Because , this is a gain-calibrated, not equal-norm, comparison.
Natural-Image Transfer and the HoloCount Boundary
Geometry Transfer, Zero-Shot Policy Failure, and Recalibration
Frozen geometry transfers more readily than policy. We first test whether the Llama low-rank control geometry discovered on procedural data remains useful on natural images. On an image-disjoint 4,000-example TallyQA confirmation set, the frozen rank-4 basis and MN-CSA intervention rule are transferred without natural-domain refitting. Using the same frozen natural-domain Proposal Controller for both whole-head and low-rank actuation, baseline exact-count accuracy is 64.975%, whole-head reaches 67.075%, and MN-CSA reaches 66.375%. Thus frozen low-rank geometry retains positive natural-domain utility (+1.40 pp over baseline; 95% CI [+0.875, +1.925] pp), although it remains −0.70 pp below same-run whole-head actuation (95% CI [−1.15, −0.25] pp). This result establishes geometry transfer, not a universal low-rank advantage.
The synthetic Proposal Controller does not transfer zero-shot. On a separate frozen 4,000-example TallyQA set, the synthetic Proposal Controller decreases exact-count accuracy from 64.85% to 61.875% (−2.975 pp), producing 51 repairs and 170 breaks. Yet a per-example oracle over the same actuator and coarse action set reaches 71.575% (+6.725 pp over baseline), showing that substantial conditional repair capacity remains even when the transferred policy is harmful. Post-hoc diagnostics show that this failure coincides with a reversal in the natural error regime, loss of score–utility alignment, and feature-support shift. Support clipping changes controller decisions but does not recover accuracy, indicating a conditional-utility shift rather than a single preprocessing artifact.
Natural-domain recalibration restores Stage-1 utility. Keeping the actuator, coarse action set, feature family, model class, and regularization fixed, we relearn only the first-stage utility mapping. After freezing a natural-domain Proposal Controller calibrated on natural examples, evaluation on a new 4,000-example question–image- and image-ID-disjoint TallyQA set improves exact-count accuracy from 64.925% to 67.050% (+2.125 pp; 95% CI [+1.50, +2.775] pp; exact McNemar ), with 130 repairs and 45 breaks. The action distribution is selective: 3,200/4,000 examples (80%) receive identity. This is policy sparsity, not evidence that 80% of latent states are intrinsically uncontrollable.
| Action | Accuracy | Change from baseline |
| Baseline / | 64.85% | 0.00 |
| 67.20% | +2.35 | |
| 63.10% | −1.75 | |
| 62.15% | −2.70 | |
| 57.725% | −7.125 | |
| Per-example oracle | 71.575% | +6.725 |
Table 26. Natural fixed actions on the 4,000-example TallyQA development/forensic sample.
Among the 269 oracle-repairable natural errors, the closest successful action is for 149 cases, for 19, for 32, and for 69. The frozen synthetic controller captures only 51/269 (18.96%) of these opportunities.
| Domain | Accuracy change | Intervention rate | Score–utility Spearman |
| v2 OOF development | +8.30 | 33.30% | |
| v3 frozen confirmation | +8.10 | 32.65% | |
| TallyQA frozen zero-shot | −2.975 | 65.275% |
Table 27. Controller behavior across domains.
| Subset | Baseline | Post | Repairs | Breaks | ||
| Simple | 2000 | 78.05% | 79.15% | +1.10 | 49 | 27 |
| Complex | 2000 | 51.80% | 54.95% | +3.15 | 81 | 18 |
| Overall | 4000 | 64.925% | 67.050% | +2.125 | 130 | 45 |
Table 28. Final natural confirmation by subset.
| Count | Fraction | |
| 0 | 508 | 12.70% |
| 1 | 3200 | 80.00% |
| 1.5 | 32 | 0.80% |
| 2 | 122 | 3.05% |
| 4 | 138 | 3.45% |
Table 29. Final natural-controller action distribution.

Figure 12. Transfer separates geometry from deployment policy. (a) Frozen Llama rank-4 geometry retains positive natural-domain utility, though below whole-head control. (b) The synthetic router fails zero-shot on a separate TallyQA set even while the same actuator/action family retains large oracle headroom. (c) Relearning only the lightweight utility map is sample-efficient; is frozen as an efficiency elbow before a new confirmation set. (d) HoloCount preserves large oracle headroom while the transferred policy is essentially neutral, exposing a stronger routing/readout boundary.
Hierarchical Natural UGR
Natural Stage-2 development begins from the upward proposals selected by the frozen natural-domain Proposal Controller. The candidate geometry/gain space is identical in concept to procedural UGR, but the policy architecture

Figure 13. Diagnosing limited zero-shot policy transfer. (a) The frozen synthetic router improves procedural v3 accuracy by pp but changes zero-shot TallyQA accuracy by pp. The post hoc natural action oracle reaches 71.575% ( pp over the 64.85% baseline), indicating retained actuator headroom. (b) Baseline errors shift from 93.76% undercounts and 6.24% overcounts on synthetic v2 to 21.12% undercounts and 78.88% overcounts on natural TallyQA. (c) Selected-score/realized-utility Spearman correlation is positive on v2 and v3 ( and 0.411) but near zero and slightly negative on natural data (). (d) Without fitting or tuning parameters, SPUC-v1 clips standardized features to empirical v2 training support. Clipping affects 39.825% of samples; accuracy remains 61.875%, while intervention rate changes from 65.275% to 63.275%. Support violation contributes to extreme extrapolation but does not explain the accuracy decrease, indicating a conditional-utility shift.

Figure 14. Lightweight natural-domain calibration restores routing utility. (a) With the actuator and routing model family fixed, mean development gain rises from pp at to pp at and then largely saturates. The selected is a sample-efficiency elbow, not the maximum-gain point; error bars denote across-run standard deviations. (b) Frozen evaluation on a new 4,000-example image-disjoint TallyQA set improves accuracy from 64.925% to 67.050% ( pp; paired 95% CI [, ] pp), with 130 repairs, 45 breaks, and net repairs (exact McNemar ). (c) Both prespecified subsets improve: Simple by pp and Complex by pp. (d) The policy retains identity () for 80.0% of examples and applies non-identity actions to 20.0%.
is treated as a domain-specific development choice. Table 30 records the sequence that led to the final hierarchical factorization.
Table 30. Natural UGR Stage-2 development. Flat and pairwise variants are retained as negative/near-neutral development results; only the hierarchical gate-plus-ranker is carried into the final freeze.
| Variant | Whole | Unified | U–W | |
| Flat candidate classifier | 292 | 38.014% | 35.959% | −2.055 |
| Pairwise ranker | 292 | 36.986% | 36.301% | −0.685 |
| Hierarchical gate + ranker | 292 | 34.247% | 35.616% | +1.370 |
Table 30.
For the selected hierarchical design, Whole produces 12 repairs and 0 breaks and Unified 16 repairs and 0 breaks on development. The gate threshold is 0.5; logistic models use and the frozen 47-D GT-free sample representation. These choices, the Whole/Unified candidate families, and the complete 14-gain grid are frozen before final-cohort evaluation.
The untouched final cohort contains 8,000 TallyQA examples, balanced 4,000 simple/4,000 complex, with zero question, image-ID, and pixel overlap against the earlier natural data. The frozen Proposal Controller emits 990 proposals at , 6,379 at , 64 at , 271 at , and 296 at , yielding the prespecified upward Stage-2 population . For every Stage-2 example, all 43 candidates (IDENTITY plus 14 each for MN, SA, and WHOLE) are executed, for 27,133 candidate outcomes.
Table 31. Natural UGR v2 untouched final result on the prespecified Stage-2 population. Oracles are computed from the exhaustive candidate archive.
| Policy | Correct | Accuracy | Repairs | Breaks | Oracle |
| Identity | 201 | 31.854% | – | – | – |
| Hierarchical Whole | 211 | 33.439% | 16 | 6 | 43.106% |
| Hierarchical Unified | 223 | 35.341% | 26 | 4 | 49.287% |
Table 31.
Unified minus Whole is +1.902 with paired bootstrap 95% CI [+0.792, +3.170] , exact two-sided McNemar , and paired discordance 14 Unified-only versus 2 Whole-only. Whole intervenes on 48/631 (7.61%) and Unified on 52/631 (8.24%). Unified selections are IDENTITY 579, WHOLE 27, MN 20, and SA 5; its selected gains are 0.5 on one intervention, 3.25 on one, and 4.0 on 50. The selection frequencies are descriptive and are not used to attribute the paired improvement to any one family.
Table 32. Natural final candidate-oracle capacity and subgroup routing results. Subgroup differences are descriptive; no interaction claim is made.
| Oracle / subgroup | Value | Difference |
| MN oracle | 44.216% | +12.361 vs identity |
| SA oracle | 40.095% | +8.241 vs identity |
| Low-rank oracle (MN/SA) | 46.751% | +14.897 vs identity |
| Whole oracle | 43.106% | +11.852 vs identity |
| Unified oracle | 49.287% | +17.433 vs identity |
| Complex () | 32.444% → 34.889% | +2.444 |
| Simple () | 35.912% → 36.464% | +0.552 |
Table 32.
Because the Whole and Unified pipelines are identical on the remaining 7,369 examples, the final Stage-2 advantage corresponds mechanically to a 12/8000 = +0.150 pairwise difference over the full cohort. We retain the conditional comparison as the primary endpoint because the scientific question is which geometry/gain should refine an already-frozen upward proposal.
Action-space headroom is complementary. The exhaustive candidate archive separates available capacity from learned realization. The Whole oracle reaches 272/631=43.106%, whereas the Unified oracle reaches 311/631=49.287%, a +6.181 capacity advantage. Individual-family oracles are 44.216% for MN, 40.095% for SA, and 46.751% for the combined low-rank family. Thus Whole is not a sufficient universal geometry on natural images, but neither is any low-rank family globally dominant; the evidence instead supports conditional access to complementary actions. The learned router realizes +1.902 of this incremental Unified-vs-Whole oracle gap without changing the frozen candidate set.
B200 numerical-drift amendment. During exhaustive candidate execution, one sample showed an archived baseline prediction of 6 but a live B200 prediction of 5. Rather than disabling the check, the complete 631-sample population was replayed. Baseline discrete predictions matched on 630/631 examples (one flip, 0.158%), and Stage-1 actions matched on 629/631, with the two action changes occurring at decision boundaries. The frozen Stage-1 population, selected proposal actions, baseline reference, and both routers were left unchanged; only the replay-verified baseline discrepancy was whitelisted while any new mismatch remained fatal. This is recorded as a hardware numerical-compatibility amendment, not a result-dependent policy change.

Figure 15. Figure 15. Natural hierarchical UGR converts complementary action-space capacity into an untouched gain. On the prespecified Stage-2 population from a new 8,000-example TallyQA cohort, frozen Unified routing improves over frozen Whole-only routing by (95% CI [, ] ; exact McNemar ; 14:2 paired discordance). The unified candidate oracle adds over the Whole oracle. The realized policy is sparse (52/631 interventions), increases repairs while reducing breaks, and does not collapse to WHOLE: 48.1% of its non-identity selections use MN or SA.
HoloCount External Boundary
The HoloCount external run contains 2,480 questions and 2,056 unique image hashes over 20 subsets. The frozen TallyQA-adapted controller changes 46.6935% baseline accuracy to 46.6532% (), with 72 repairs and 73 breaks. The action oracle reaches 58.9516% () and identifies 304 repairable baseline errors. Fixed-action changes are for , for , for , and for . Grouped OOF training with the same 39-feature StandardScalerRidge family also yields approximately null gain and near-zero score–utility alignment. The supported conclusion is therefore a boundary: the actuator/action family retains substantial conditional repair capacity, but neither the transferred policy nor the tested linear representation/readout family recovers it.
H Off-Target Stress Test and Finite-Step Response Diagnostics
GQA Off-Target Stress Test
The counting experiments establish target-task controllability, but an inference-time intervention should also be examined for collateral changes outside the behavior used to discover it. We therefore freeze a non-counting GQA stress test on the first 1,000 baseline-correct examples in candidate order and compare L18H13 MN against WHOLE under seven matched finite displacement budgets. Because the population is baseline-correct by construction, the endpoint is break burden: the fraction of geometry-budget conditions that turn a correct non-counting answer into an error.
Across the 7,000 matched conditions per family, WHOLE produces 101 breaks (1.4429%) and MN produces 83 (1.1857%), an aggregate MN−WHOLE difference of . The paired bootstrap 95% CI is [,] . Six of seven individual budget levels favor MN; the per-budget differences in MN−WHOLE break rate are , , , , , , from through 2.0. A post-freeze 1,000,000-draw paired sign-flip analysis gives two-sided , and repeated bootstrap seeds leave the confidence interval essentially unchanged. We therefore make a bounded claim: both geometries preserve the large majority of baseline-correct non-counting behavior, with a stable directional trend toward lower MN collateral burden, but the global superiority of MN is not statistically established by this cohort.
This stress test complements rather than replaces the target-task specificity controls. The Bottom-4/random-subspace experiments ask whether the learned low-rank directions are behaviorally privileged for counting; GQA asks whether a matched finite edit perturbs already-correct behavior outside the discovery task. Together they separate target efficacy from off-target preservation.
| 0.50 | 0.75 | 1.00 | 1.25 | 1.50 | 1.75 | 2.00 | Aggregate | |
| MN−WHOLE break rate (pp) | +0.60 | -0.10 | -0.40 | -0.60 | -0.20 | -0.40 | -0.70 | -0.257 |
Table 33. GQA off-target break-burden summary. Difference is MN−WHOLE; negative values favor MN.
MN has 18 fewer breaks than WHOLE. A 200,000-repetition bootstrap gives ; five additional 100,000-repetition seeds reproduce essentially the same interval and . Thus Monte Carlo instability is not driving the interval, but sampling uncertainty remains.
Each stage of the C2 chain (baseline population, matched-intervention execution protocol, raw outcomes, endpoint analysis, and post-freeze robustness analysis) is frozen under a separate commit before the next stage begins.
Finite-Step Response on Natural Data
The confirmed results above establish that AROMA can control cardinality-related behavior and that deployment utility must be routed conditionally. We next ask a narrower post-hoc question: what distinguishes useful finite movement from destructive finite movement? These analyses use already-generated outcomes and do not modify any frozen policy.

Figure 16. Finite-step response geometry explains selective repair and failure. (a) AROMA intervenes selectively rather than globally. (b) Larger expected-numeral movement creates both repair opportunity and break risk. (c) Repairs are strongly concentrated in target-aligned movement directions. Target alignment is a post-hoc diagnostic using ground truth and is not available at inference time.
| Dataset | Active / total | Active repair | Active break | Repair: small/ med./large | Break: small/ med./large | Target-aligned / away repair |
| TallyQA | 800 / 4000 (20.00%) | 16.25% | 5.63% | 5.24 / 19.55 / 23.97% | 0.75 / 4.51 / 11.61% | 32.17% / 0.49% |
| HoloCount | 880 / 2480 (35.48%) | 8.18% | 8.30% | 1.36 / 12.29 / 10.92% | 1.70 / 10.24 / 12.97% | 22.62% / 0.99% |
Table 34. Post-hoc finite-step response diagnostics on natural data. “Active” means nonzero observed expected-numeral movement; on the final TallyQA controller this coincides with non-identity selection. Magnitude bins are tertiles within active examples. Target alignment is defined using ground truth and is therefore diagnostic only, not deployable.
Selectivity transfers more readily than utility. On the final TallyQA confirmation, the 3,200 inactive identity examples are substantially easier: baseline and post-intervention accuracy are both 2402/3200 = 75.06%. The 800 active examples are much harder, with baseline accuracy 195/800 = 24.38%; after natural-domain recalibration their accuracy rises to 280/800 = 35.00%, reflecting 130 repairs against 45 breaks. On HoloCount, 1,600 zero-movement examples likewise have higher baseline accuracy (885/1600 = 55.31%) than the 880 active examples (273/880 = 31.02%). However, active accuracy changes only to 272/880 = 30.91% because repairs and breaks nearly cancel (72 versus 73). Thus the transferred policy still concentrates intervention on harder cases, but its action utility no longer transfers. The mean active is 0.282094 on TallyQA and 0.684911 on HoloCount.
Movement magnitude exposes an opportunity–risk trade-off. Among active natural examples, we partition into within-dataset tertiles. On TallyQA, repair rates rise from 5.24% to 19.55% to 23.97% across small/medium/large movement, while break rates simultaneously rise from 0.75% to 4.51% to 11.61%. On HoloCount, repair rises from 1.36% to 12.29% from small to medium movement but then falls to 10.92% in the largest bin, while break rate continues to increase from 1.70% to 10.24% to 12.97%. The external dataset therefore exposes an overshoot/instability boundary: stronger movement creates more opportunity for state change, not guaranteed improvement.
Repairs concentrate in target-aligned expected-numeral movement. For analysis only, we call an active baseline error target-aligned when the intervention reduces expected-numeral distance to the ground truth, , and away when it increases that distance. On TallyQA, 129/401 target-aligned active baseline errors are repaired (32.17%) versus 1/204 away cases (0.49%). On HoloCount, the corresponding repair rates are 69/305=22.62% and 3/302=0.99%. This diagnostic is intentionally not treated as a deployable routing feature: it uses and therefore cannot be available at inference time. Its value is explanatory—the discrete repairs produced by AROMA are strongly concentrated among interventions whose continuous expected-numeral movement is directionally consistent with the required correction.
| Dataset | Bin | Mean | Repair | Break | |
| TallyQA | small | 267 | 0.04150 | 5.24% | 0.75% |
| TallyQA | medium | 266 | 0.16862 | 19.55% | 4.51% |
| TallyQA | large | 267 | 0.63573 | 23.97% | 11.61% |
| HoloCount | small | 294 | 0.06950 | 1.36% | 1.70% |
| HoloCount | medium | 293 | 0.31308 | 12.29% | 10.24% |
| HoloCount | large | 293 | 1.67425 | 10.92% | 12.97% |
Table 35. Natural active-response magnitude tertiles. Repair and break rates are fractions of all active examples in each magnitude bin, not conditional on baseline error/correctness.
SA-dev Finite-Gain Response Curves
Synthetic response curves show the same finite-gain tension. On the 500-example SA-dev cohort, increasing gain causes repairs and breaks to accumulate together rather than producing a single monotonic “more is better” regime. For example, MN accuracy increases from 50.8% at identity to 55.6% at gain 8, but the same sweep grows from 0/0 repairs/breaks at identity to 107/83 at gain 8; SA reaches 56.6% with 99/70 repairs/breaks, while WHOLE reaches 59.6% with 91/47. These development diagnostics help explain why the final system routes gain and geometry instead of selecting a universally maximal displacement.
| Gain | MN Accuracy | MN Mean | MN Repairs | MN Breaks | SA Accuracy | SA Mean | SA Repairs | SA Breaks | WHOLE Accuracy | WHOLE Mean | WHOLE Repairs | WHOLE Breaks |
| 1.0 | 50.8% | 0.000000 | 0 | 0 | 50.8% | 0.000000 | 0 | 0 | 50.8% | 0.000000 | 0 | 0 |
| 1.5 | 51.6% | -0.047078 | 12 | 8 | 51.4% | -0.107031 | 11 | 8 | 51.4% | -0.024922 | 13 | 10 |
| 2.0 | 52.0% | -0.072813 | 21 | 15 | 52.2% | -0.195453 | 20 | 13 | 50.8% | -0.063250 | 20 | 20 |
| 3.0 | 51.4% | -0.088031 | 40 | 37 | 51.2% | -0.336531 | 33 | 31 | 49.6% | -0.198234 | 32 | 38 |
| 4.0 | 54.2% | -0.068422 | 64 | 47 | 54.4% | -0.441625 | 57 | 39 | 49.4% | -0.395469 | 36 | 43 |
| 6.0 | 54.6% | +0.168969 | 83 | 64 | 55.2% | -0.489297 | 79 | 57 | 55.0% | -0.718328 | 67 | 46 |
| 8.0 | 55.6% | +0.708812 | 107 | 83 | 56.6% | -0.136937 | 99 | 70 | 59.6% | -0.846797 | 91 | 47 |
Table 36. SA-dev finite-gain response curve (). is mean post-minus-baseline expected numeral. Repairs and breaks are counted relative to identity.
Finite-Step Gradient-Drift Diagnostic
The gradient-drift diagnostic recomputes the gradient of correct-answer negative log-likelihood at the input to the L18H13 cross-attention output projection. The loss used by the diagnostic is
and the head-slice gradient is pooled across token positions. A fixed supplied direction is then applied with displacement proportional to , and the gradient is recomputed after the intervention. We report cosine with the identity-state gradient. This design probes drift of local objective geometry under a finite step; because it is not the complete deployed candidate-selection process, it is not used as direct proof of MN/SA routing.
| Direction label | 1.0 | 1.5 | 2.0 | 3.0 | 4.0 | 6.0 | 8.0 |
| MN mean | 1.000 | 0.918 | 0.866 | 0.734 | 0.629 | 0.418 | 0.136 |
| MN median | 1.000 | 0.995 | 0.981 | 0.930 | 0.857 | 0.621 | 0.129 |
| SA mean | 1.000 | 0.910 | 0.855 | 0.737 | 0.621 | 0.407 | 0.146 |
| SA median | 1.000 | 0.995 | 0.981 | 0.944 | 0.892 | 0.716 | 0.296 |
Table 37. Finite-step gradient drift on SA-dev. Values are cosine similarity between the post-intervention correct-answer-loss gradient and the identity-state gradient.
For gains above identity, mean gradient norms remain on the order of rather than collapsing to zero: approximately 0.000385–0.000438 for the MN-labeled runs and 0.000277–0.000406 for the SA-labeled runs over the recorded gain range. The central observation is therefore rotation of the local gradient direction, not disappearance of local sensitivity. We interpret this result narrowly: local correct-answer-loss geometry changes substantially along large finite interventions, so a direction that is locally informative need not remain locally equivalent after a large step. It is supporting evidence for the local-versus-finite distinction, not a direct proof of the deployed MN/SA routing mechanism.
Full-Population Deployment Accounting and External Steering Controls
What Is Confirmed and What Is Reconstructed
The pre-frozen primary targeted-deployment comparison remains runner-up + MEA versus frozen old Unified on the same prespecified upward-proposal Stage-2 population. Its paired outcomes (244 versus 217 correct, 29 MEA-only versus two Unified-only, exact ) are untouched by the supplementary analyses. The full-cohort reconstruction is derived after outcome execution from the identical locked per-example outputs, and the external controls below were developed after the prospective Stage-2 outcomes were available.
Identity-fallback estimand and integrity. Let denote the untouched natural cohort; its frozen upward-proposal branch, and the frozen APPLY set. Let be the archived unmodified model prediction and the pre-frozen MEA prediction on APPLY. We define an identity-fallback selective intervention policy on ,
This definition intentionally retains identity for all non-APPLY examples, including non-upward Stage-1 proposals; it should not be conflated with an execution of every non-upward Proposal Controller action. The frozen cohort exactly matches the 630 upward proposals in Stage 1, all 630 Stage-1/Stage-2 identity predictions agree, and the Stage-2 runner-up + MEA prediction equals identity on all 567 KEEP examples. Therefore the full-population correct count is uniquely determined by the archive:
The active gate executes 63/7,992 interventions (0.788% of the population). Relative to identity, MEA yields 51 repairs and four breaks, or 47 net correct responses, and exact two-sided paired McNemar from the frozen 51:4 discordance. This inference is about selective prediction accuracy, not inference-time efficiency: feature extraction, proposal, and gate execution costs are not measured by the intervention coverage rate.
Matched-Gate Output Control
The frozen APPLY population is strongly enriched for zero-count questions: 51/63 (80.95%) true answers are zero and 12/63 are nonzero. As an explicit simple control, retain the same AROMA gate and the same archived identity predictions on KEEP, but emit the numeral zero on APPLY. This policy is correct on 51/63 APPLY examples and 5,253/7,992 full-cohort examples (65.728%), compared with 52/63 and 5,254/7,992 (65.741%) for targeted MEA. There are five MEA-only and four zero-only correct outcomes (exact two-sided McNemar ). On APPLY, MEA is correct on 47/51 true-zero examples and 5/12 nonzero examples, whereas the zero policy is correct on 51/51 and 0/12. The deployment gain over identity is concentrated in zero-count repair. The matched-budget and oracle-target experiments evaluate the control law on endpoints that do not depend on this target prior.
| Prediction policy | APPLY correct | Full correct | Full accuracy |
| Unmodified identity | 5/63 | 5,207 | 65.1527% |
| Old Unified with matched gate | 25/63 | 5,227 | 65.4029% |
| Gate-conditional zero | 51/63 | 5,253 | 65.7282% |
| Frozen runner-up + MEA | 52/63 | 5,254 | 65.7407% |
Table 38. Audited identity-fallback policies across and the same frozen APPLY set. Gate-conditional zero is a post-confirmation stress control.
Two Distinct GrAInS-Related External Controls
GrAInS [36] derives steering directions from gradient-based attributions, token ablation, and representation contrasts. We study a counting-adapted GrAInS-inspired method and a separate minimal multimodal compatibility port; neither is an official reimplementation validated on the original GrAInS model and task. All comparisons use the identical fixed Stage-2 question set, and both external configurations are fixed without using test ground truth as an inference feature. However, development and baseline selection occur after AROMA’s prospective test outcomes are known, so these are supplemental comparisons.
Counting-adapted GrAInS-inspired steering. The earlier adapted implementation uses a separate fit40/cal48 development partition, task-adapted numerical contrastive objectives and token ablation, and a calibrated steering direction/gain on the counting task. Under the same intervention set, it predicts 33/63 correctly, compared with MEA’s 52/63. With identical archived identity fallback on KEEP, the GrAInS-inspired policy is correct on 225/630 (35.714%) and targeted MEA on 244/630 (38.730%): a +3.016 pp matched-gate difference, 23 MEA-only versus four GrAInS-only correct outcomes, exact two-sided McNemar . When the adapted method instead steers all 630 Stage-2 examples, it reaches 247/630 (39.206%), versus MEA’s 244/630 (38.730%); the direct global-policy paired discordance is 33 MEA-only versus 36 GrAInS-only ().
Image-preserving GrAInS minimal port. A second method follows the source-level attribution–ablation–representation-difference design more closely while adapting its image-conditioned forward pass to Llama-3.2-11B-Vision-Instruct. At the official GrAInS revision 8ad78db, the compatibility port uses zero-embedding-baseline contrastive Integrated Gradients with 20 Gauss–Legendre steps, separate positive/negative signed token attribution, PAD/UNK token ablation, L31 representation differences, and principal-component analysis of 80 difference vectors constructed from 40 isolated development pairs. A documented pre-fit amendment uses Top- (the official function-supported default), because the example-script would ablate every eligible text token in the short counting prompts; the median eligible-token ablation rate becomes 13.64%. A disjoint CAL48 selects over by the preferred smaller-gain tie break (both 22/48 correct), while keeping direction, layer, attribution, and selection rule fixed. GrAInS-style sequence-norm-preserving steering is retained. On the frozen Stage-2 set, the calibrated port reaches 201/630 (31.905%), or 198/630 (31.429%) under the same gate and BF16 identity fallback, with 6/63 correct on APPLY. Its own FP32 identity attains 198/630 (31.429%); calibrated steering versus that FP32 identity has exact paired . The uncalibrated demonstration reaches 92/630 (14.603%). The FP32 port and archived BF16 AROMA identity disagree on 17/630 discrete outputs.
| Policy / intervention scope | Global correct / 630 | APPLY correct / 63 |
| Identity (BF16, no steering) | 197 (31.270%) | 5 (7.937%) |
| Frozen AROMA old Unified | 217 (34.444%) | 25 (39.683%) |
| AROMA runner-up + MEA | 244 (38.730%) | 52 (82.540%) |
| GrAInS-inspired, counting-adapted, same-gate | 225 (35.714%) | 33 (52.381%) |
| GrAInS-inspired, counting-adapted, global | 247 (39.206%) | 33 (52.381%) |
| GrAInS minimal port, same-gate | 198 (31.429%) | 6 (9.524%) |
| GrAInS minimal port, global () | 201 (31.905%) | 6 (9.524%) |
| GrAInS minimal port, global () | 92 (14.603%) | 3 (4.762%) |
| Always zero, Stage-2 global | 307 (48.730%) | 51 (80.952%) |
Table 39. Supplemental external steering comparisons on the fixed Stage-2 cohort. “Same-gate” always retains the archived BF16 identity prediction on the 567 KEEP examples, whereas “global” steers all 630. The APPLY cohort (63 samples) is chosen by AROMA’s gate, not by an external-method-independent randomization.
Interpretive hierarchy. The strictest prospective deployment evidence remains MEA versus the already-frozen old Unified policy on the untouched Stage-2 cohort. The oracle-target/matched-budget experiments identify the value of iterative re-linearization conditional on supplied targets and displacement norms. The later GrAInS-family comparisons show that fixed-direction steering, adaptation, and deployment scope can yield very different accuracies. Finally, the gate-conditional zero control shows that, on this cohort, target construction and gating carry most of the deployment gain. This separation of control capacity, target construction, and deployment policy is the intended scientific interpretation of AROMA.
Extended Discussion, Scope of Claims, and Related Work
Extended Discussion
Coverage, magnitude, and scientific interpretation. The untouched Stage-2 improvement is substantial within its frozen selection regime, and it remains positive and statistically supported over the complete natural cohort even though only 0.788% of inputs are edited. In contrast, high oracle reachability and matched-budget re-linearization benefits isolate an actuator-control capability that aggregate accuracy alone cannot resolve. Together with the global GrAInS-inspired and gate-conditional-zero controls, this argues for reporting mechanistic reachability, policy-level deployment utility, and population-level gain as separate endpoints.
Local sensitivity is not finite control. Across Llama and IDEFICS, cardinality sensitivity is strongly concentrated in a low-dimensional subspace, yet that subspace contains comparatively little activation energy. This mismatch is operational rather than merely spectral: preserving most local sensitivity does not by itself yield a finite edit of useful magnitude. Magnitude normalization repairs one failure mode, but the IDEFICS rank screen further shows that capturing more sensitivity variance does not necessarily improve behavioral steering. The finite-step diagnostics point in the same direction: as gain increases, the correct-answer-loss gradient rotates away from its identity-state orientation even though its norm remains nonzero. Taken together, these results distinguish a locally informative sensitivity geometry from a behaviorally validated intervention geometry. AROMA therefore treats spectral structure as a proposal for control, not as control evidence by itself.
Controllability is not utility. Causal screening establishes that an internal component can move the target behavior; it does not determine whether a particular finite intervention will improve correctness on a particular example. This distinction appears repeatedly. Direction–gain orderings cross, a prospective sensitivity-only gain rule fails to outperform the learned candidates, and larger finite movement creates both repair opportunities and break risk. The natural-domain diagnostics sharpen the same point: intervention is concentrated on harder examples, yet selective targeting alone is insufficient when action utility shifts, as on HoloCount. Likewise, target-aligned expected-numeral movement strongly concentrates repairs, but this quantity depends on the ground-truth answer and is therefore explanatory rather than deployable. These results motivate routing candidate correctness rather than responsiveness: a useful actuator still requires a policy that decides whether, how, and how strongly to intervene.
Geometry belongs inside the policy rather than being fixed globally. The low-rank Utility Router first shows that direction and gain should be selected conditionally even within a fixed intervention family. Matched-budget MN, SA, and WHOLE trajectories then show no stable intrinsic geometry winner once finite gain is calibrated. The later UGR experiments turn this ambiguity into a prospective policy question. On procedural counting, Frozen Unified routing improves the strongest development-selected restricted policy by +2.21 pp on untouched data. Under natural-domain shift, the same principle survives only after policy refactorization: the Unified action space exposes an additional +6.181 pp of oracle capacity over Whole-only control, and the frozen hierarchical router converts part of that capacity into a +1.902 pp paired final gain. The important conclusion is therefore not that MN, SA, or WHOLE is globally superior, but that intervention geometry itself is an example-conditional control variable. Family-selection frequencies support non-degenerate use of the enlarged action space, but they are descriptive rather than causal attributions of the aggregate gain.
Domain shift separates abstention calibration from action ranking. Natural-domain transfer reveals that mechanism, geometry, and policy do not generalize as a single unit. Frozen low-rank geometry retains positive utility on TallyQA, whereas the synthetic Proposal Controller becomes harmful until its utility mapping is recalibrated. At Stage 2, a flat unified classifier and pairwise ranking alone fail to exploit the enlarged natural action space reliably; successful transfer requires a hierarchical factorization that first decides whether to intervene and only then ranks non-identity candidates. After this architecture is selected on development data and frozen, it produces a statistically supported gain on the untouched natural Stage-2 population. IDEFICS provides a complementary form of generalization: the actuator–geometry–gain hierarchy independently reappears without transferring Llama coordinates. HoloCount, however, remains a boundary in which intervention headroom persists while the tested policy/readout does not recover positive utility. Generalization is therefore factorized rather than binary: causal capacity can persist even when gain, abstention calibration, action ranking, or readout requires new domain-specific learning.
WHERE, WHICH GEOMETRY, WHEN, WHICH TARGET, and HOW are separate decisions. The routing and targeted-control experiments now form one coherent decomposition. Causal localization supplies WHERE; MN, SA, and WHOLE routing select geometry and gain; the hierarchical gate supplies WHEN; runner-up targeting supplies WHICH TARGET; and MEA supplies HOW TO REACH. Keeping these roles separate clarifies why apparently contradictory results coexist: no geometry is globally best after gain calibration, the same actuator can be useful or destructive under different targets, and a strong target controller can coexist with a sparse deployment gate.
Target quality is now the dominant residual bottleneck on the active branch. The prospective APPLY set sharpens the diagnosis: target selection is correct on 80.95% of examples, assigned-target reachability is 96.83%, and all 51/51 correct targets are reached. The execution controller therefore leaves almost no observed residual error once given a correct target on this cohort. Improving target construction is consequently higher-leverage than making MEA more aggressive under the current protocol.
Negative results are part of the causal argument. The observed transfer and target-selection failures identify which stages of the control pipeline require domain-specific calibration. Together with the successful held-out confirmations, they support treating actuator capacity, target selection, and deployment policy as distinct components. The upward-only v1 rule is especially informative: because MEA reliably reaches even wrong targets, it separates target quality from actuator capacity.
The experimental chronology is part of the scientific claim. Mechanistic control is especially vulnerable to optimistic selection when localization, geometry, gain, and routing are tuned on the same outcomes (Makelov et al., 2024). AROMA therefore separates causal screening, geometry construction, rank-matched specificity controls, development-only calibration, grouped OOF router selection, explicitly labeled post-hoc diagnostics, and untouched confirmation after protocol closure. This chronology is not merely procedural bookkeeping: it determines which conclusions are prospective, which are diagnostic, and which remain unresolved. The resulting contribution is therefore a reproducible path from causal mechanism discovery to adaptive control. The final targeted-deployment chain makes this explicit: retrospective failure diagnosis OOF target-policy support frozen runner-up rule new untouched natural cohort frozen gate/reference/target plan prospective outcome execution.
Scope of Claims
The evidence supports four findings: the actuator–low-rank–geometry–finite-control pattern independently replicates in two VLMs; procedural UGR and domain-developed hierarchical Natural UGR each receive untouched prospective confirmation on upward proposals; ground-truth-free runner-up targeting plus frozen-gate MEA receives a separate untouched natural deployment confirmation in Llama; and model/domain-specific gain, target policy, and routing calibration are separate problems. The scope of these findings is bounded in three respects. (i) Model specificity: the exact actuator identity, basis vectors, and configuration are model-specific, subspaces are not transferred coordinate-wise from Llama into IDEFICS, and causal intervention rather than observational attention identifies the mechanism. (ii) Domain and policy coverage: policies and procedural UGR parameters are not transferred zero-shot across natural datasets, Llama MN-CSA transfer is established on the reported frozen confirmations, and second-stage routing is evaluated on upward proposals in Llama rather than on downward proposals, HoloCount, or IDEFICS. (iii) Generality: the results concern visual counting and the current linear/hierarchical routing representations.
Closest VLM Intervention Work
Table 40 summarizes the control object emphasized by nearby VLM intervention work rather than scoring methods by checkmarks. The comparison is deliberately descriptive: different papers optimize different behaviors, models, and evaluation regimes.
| Method | Localization / intervention object | Conditional control emphasis | Evaluation emphasis |
| NOTICE ([7]) | causally localized multimodal components | diagnostic localization | corruption-based mechanism analysis |
| V-SEAM ([29]) | causal visual-semantic components | intervention after attribution | editing / attention modulation |
| SteerVLM ([22]) | hidden activation dimensions | lightweight adaptive steering | robustness / off-target behavior |
| CAST ([12]) | steering vectors | conditional activation steering | condition-dependent deployment |
| Rudman et al. ([20]) | hallucination-relevant mechanisms | causal intervention | prompt-induced hallucination analysis |
| VIB-Probe ([34]) | probe-derived hallucination signal | mitigation after detection | hallucination correction |
| HONES ([30]) | head/neuron causal structure | neuron-level steering | multi-task causal attribution |
| AROMA | validated causal actuator → low-rank control subspace | geometry/gain routing plus explicit target selection and iterative target execution | untouched mechanism, routing, targeted-deployment, transfer, and off-target boundaries |
Table 40. Descriptive positioning against closely related VLM intervention work. The table records each method’s main intervention/control emphasis; it is not a quality ranking.
From steering directions to closed-loop control. Most activation-steering pipelines couple a direction and magnitude into one intervention object. AROMA instead treats intervention geometry, finite gain, target selection, target execution, and deployment gating as separately falsifiable stages. The distinction matters empirically: a target can be highly reachable and still destroy task utility, while a frozen gate plus a prospectively confirmed target policy can convert the same actuator into a useful controller. This mechanism–target–policy decomposition is the main conceptual difference from treating steering as a fixed vector edit.
Reproducibility and Provenance
Every formal stage is tied to an immutable protocol or result artifact. The record includes the exact model revision, Git freeze/result commits, controller and Top-4 basis hashes, null/random seeds, random-subspace basis inventory, method fingerprints, and dataset manifests. The successful environments use Python 3.11.16, PyTorch 2.11.0+cu128, Transformers 5.17.0, Accelerate 1.15.0, NumPy 2.4.6, Pandas 3.0.5, SciPy 1.17.1, scikit-learn 1.9.0, Pillow 12.3.0, PyYAML 6.0.3, tqdm 4.70.0, Matplotlib 3.11.1, and joblib 1.6.0 on CUDA 12.8. Runs were executed on Blackwell GPUs including B200 and RTX PRO 6000 Server Edition; per-run metadata records the actual device. The cross-model IDEFiCS runs use the pinned revision 7065afb8a5bb003015700b315ef6751dcs3dd4e16eb. The frozen IDEFiCS Top-4 basis, its source gradient matrix, and the v2 split manifests are archived with SHA256 checksums.
Frozen runners include pre-inference audits for controller identity, model revision, manifests, image hashes, feature leakage, ground-truth isolation, identity interventions, Git ancestry, and method fingerprints. Resume functionality is accepted only for technical interruption and rejects incompatible partial artifacts. The final v4 set was generated only after the CSA algorithm was frozen; an independent pre-inference audit verified 2,000 unique sample IDs, 2,000 unique seeds, 2,000 unique image SHA256 hashes, exact count/condition balance, and zero seed or image-hash overlap with v1/v2/v3. The generator recorded 38 objective collision retries under the prespecified seed/pixel-uniqueness rule; no sample was filtered or replaced using model behavior. Where archived action matrices exist, the appendix artifacts are sufficient to regenerate reported tables and plots without rerunning the VLM.
Consolidated Result Ledger
| Experiment | Baseline | Post/Oracle | ||
| Synthetic v2 OOF router | 1000 | 50.30 | 58.60 | +8.30 |
| Frozen v3 whole-head confirmation | 2000 | 49.95 | 58.05 | +8.10 |
| Final untouched v4 MN-CSA | 2000 | 50.25 | 58.25 | +8.00 |
| Experiment | Baseline | Post/Oracle | ||
| v4 whole-head reference | 2000 | 50.25 | 57.25 | +7.00 |
| Utility Router OOF (upward) | 160 | 62.50 | 67.50 | +5.00 |
| Utility Router frozen external (upward) | 156 | 66.67 | 71.15 | +4.487 |
| Utility Router untouched final (upward) | 294 | 60.54 | 66.33 | +5.782 |
| Procedural UGR untouched final (upward) | 633 | 70.93 | 73.14 | +2.212 |
| Natural hierarchical UGR final (upward) | 631 | 33.439 | 35.341 | +1.902 |
| Runner-up + MEA v2 (retrospective) | 631 | 31.854 | 37.084 | +5.230 |
| Runner-up + MEA v2 (prospective) | 630 | 31.270 | 38.730 | +7.460 |
| TallyQA zero-shot synthetic router | 4000 | 64.85 | 61.875 | -2.975 |
| TallyQA action oracle | 4000 | 64.85 | 71.575 | +6.725 |
| Frozen = 500 TallyQA confirmation | 4000 | 64.925 | 67.050 | +2.125 |
| HoloCount transferred router | 2480 | 46.694 | 46.653 | -0.040 |
| HoloCount action oracle | 2480 | 46.694 | 58.952 | +12.258 |
Table 41. Consolidated result ledger. Development, confirmatory, oracle, and external-boundary rows are intentionally reported together here for auditability; their evidence classes remain distinct in the main text.
| Comparison | Reference | Alternative | Evidence class | ||
| Utility Router final | 294 | Global MN 60.54 | Low-rank router 66.33 | +5.782 pp | Untouched confirmatory |
| Adaptive whole challenge | 294 | Low-rank 66.33 | Whole 70.75 | +4.422 pp | Post-hoc diagnostic |
| Matched budget OOF | 160 | MN 66.25 | Whole 62.50 | −3.75 pp | Development diagnostic |
| Matched budget final | 294 | MN 65.65 | Whole 66.67 | +1.02 pp | Post-hoc diagnostic |
| Procedural UGR development OOF | 654 | Whole restricted 68.81 | Unified 71.87 | +3.058 pp | Development / OOF |
| Procedural UGR untouched final | 633 | Whole restricted 70.93 | Unified 73.14 | +2.212 pp | Prospective confirmatory |
| Natural hierarchical UGR dev | 292 | Whole 34.247 | Unified 35.616 | +1.370 pp | Development |
| Natural hierarchical UGR final | 631 | Whole 33.439 | Unified 35.341 | +1.902 pp | Prospective confirmatory |
| Targeted MEA-v2 final | 630 | Old Unified 34.444 | Runner-up+MEA 38.730 | +4.286 pp | Prospective confirmatory |
Table 42. Geometry-routing comparison ledger. Post-hoc diagnostic rows are separated from the prospective UGR final test.
| Experiment | Endpoint | Result | |
| E1 local response | 160 | local calibration | : Pearson .9936, Spearman .9990, sign 100%; median |
| Oracle MEA | 112 | strict target reachability | 91/112=81.25% within 20 steps |
| Frozen-direction matched budget | 112 | target reachability | 17/112=15.18%; MEA-only 74, frozen-only 0 |
| Activation-aligned matched budget | 112 | target reachability | 1/112=0.89%; MEA-only 90, activation-only 0 |
| Deployable v1 forced target | 623 | targetable assigned-target reachability / utility | 66.93% target reached but −18.542 pp archived utility; 17 repairs / 134 breaks |
| Runner-up target dev | 30 | APPLY target quality | 24/30=80.0% target=>GT; 15/16=93.75% among old repairs |
| Runner-up + MEA v2 | 52 | APPLY controller execution | 50/52=96.15% target reached; 37/37 correct targets reached |
| Runner-up + MEA v2 prospective | 63 | APPLY target quality / execution | 51/63=80.95% target=>GT; 61/63=96.83% reached; 51/51 correct targets reached |
Table 43. Principled local-control extension ledger. These metrics are not all exact-count accuracy and therefore are kept separate from the main accuracy ledger.
| Experiment | Primary endpoint | Result |
| Llama D1 multi-actuator () | Bidirectional replication after Holm across 14 directional tests | 6/7 heads replicate; 5/6 non-primary; L8H30 is the negative case |
| GQA off-target () | Aggregate paired break burden across seven matched budgets | WHOLE 1.4429%, MN 1.1857%; MN−WHOLE −0.2571 , 95% CI [−0.6429, +0.1286] ; sign-flip |
Table 44. Robustness ledger for experiments that are not exact-count accuracy comparisons.
| Stage | Down cons. | Up cons. | Mean-effect retention (D/U) | |
| Held-out whole-head actuator confirmation | 300 | 82.00% | 82.67% | – |
| Equal-norm rank-4 MN-CSA | 300 | 65.33% | 66.00% | 40.47% / 34.28% |
| Dev-selected , (dev200) | 200 | 73.00% | 74.00% | 73.69% / 73.62% |
| Untouched final , | 300 | 79.67% | 79.67% | 81.09% / 81.49% |
Table 45. Cross-model IDEFICS causal-control ledger. These rows report directional-control endpoints rather than exact-count accuracy and therefore are kept separate from the accuracy ledger above.
References
- [1]Manoj Acharya, Kushal Kafle, and Christopher Kanan. Tal-lyQA: Answering Complex Counting Questions. In AAAI, pages 8076–8084, 2019.
- [2]Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, Roman Ring, Eliza Rutherford, Serkan Cabi, Tengda Han, Zhitao Gong, Sina Samangooei, Marianne Monteiro, Jacob Menick, Sebastian Borgeaud, Andrew Brock, Aida Nematzadeh, Sahand Sharifzadeh, Nikolaj Binkowski, Ricardo Barreira, Oriol Vinyals, Andrew Zisserman, and Karen Simonyan. Flamingo: a Visual Language Model for Few-Shot Learning. In NeurIPS, 2022.
- [4]Liwei Che, Zhiyu Xue, Yihao Quan, Benlin Liu, Zeru Shi, Michelle Hurst, Jacob Feldman, Ruixiang Tang, Ranjay Krishna, and Vladimir Pavlovic. Counting Circuits: Mecha-nistic Interpretability of Visual Reasoning in Large Vision-Language Models. arXiv:2603.18523, 2026.arxiv.org/abs/2603.18523
- [5]Jinhong Deng, Limeng Qiao, and Guanglu Wan. Holo-Count: A Holistic Visual Counting Benchmark for MLLMs. arXiv:2607.06420, 2026.arxiv.org/abs/2607.06420
- [6]Ahmed Oumar El-Shangiti, Abzal Nurgazy, Hilal AlQ uabeh, Nikolai Rozanov, and Kentaro Inui. The Count Is There, but Misaligned: Understanding and Correcting Counting Failures in VLMs. arXiv:2607.09544, 2026.
- [7]Michal Golovanevsky, William Rudman, Vedant Palit, Carsten Eickhoff, and Ritambhara Singh. What Do VLMS NOTICE? A Mechanistic Interpretability Pipeline for Gaussian-Noise-free Text-Image Corruption and Evaluation. In NAACL, pages 11462–11482, 2025.
- [8]Yash Goyal, Tejas Khot, Douglas Summers-Stay, Dhruv Batra, and Devi Parikh. Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering. In CVPR, 2017.
- [9]Drew A. Hudson and Christopher D. Manning. GQA: A New Dataset for Real-World Visual Reasoning and Compositional Question Answering. In CVPR, 2019.arxiv.org/abs/1902.09506
- [11]Ranjay Krishna, Yuke Zhu, Oliver Groth, Justin Johnson, Kenji Hata, Joshua Kravitz, Stephanie Chen, Yannis Kalantidis, Li-Jia Li, David A. Shamma, Michael S. Bernstein, and Li Fei-Fei. Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations. International Journal of Computer Vision, 123:32–73, 2017.arxiv.org/abs/1602.07332
- [12]Bruce W. Lee, Inkıt Padhi, Karthikeyan Natesan Ramamurthy, Erik Mehling, Pierre Dognin, Manish Nagireddy, and Amit Dhurandhar. Programming Refusal with Conditional Activation Steering. In ICLR, 2025.arxiv.org/abs/2409.05907
- [14]Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. Locating and Editing Factual Associations in GPT. In NeurIPS, 2022.arxiv.org/abs/2202.05262
- [15]Meta. Llama 3.2 Vision Model Card: Llama-3.2-11B-Vision-Instruct. Hugging Face model documentation, 2024.
- [16]Judea Pearl. Causality: Models, Reasoning and Inference. Cambridge University Press, second edition, 2009.
- [17]Muhammad Fetrat Qharabagh, Mohammadreza Ghofrani, and Kimon Fountoulakis. VLM-Count: Enhancing the Counting Ability of Large Vision-Language Models. Transactions on Machine Learning Research, 2026.
- [18]Yifei Qian, Zhongliang Guo, Bowen Deng, Chun Tong Lei, Shuai Zhao, Chun Pong Lau, Xiaopeng Hong, and Michael P. Pound. T2ICount: Enhancing Cross-modal Understanding for Zero-Shot Counting. In CVPR, pages 25336–25345, 2025.arxiv.org/abs/2502.20625
- [20]William Rudman, Michal Golovanevsky, Dana Arad, Yonatan Belinkov, Carsten Eickhoff, Ritambhara Singh, and Kyle Mahowald. Mechanisms of Prompt-Induced Hallucination in Vision–Language Models. In ACL, pages 41894–41912, 2026.
- [21]Saurav Sengupta, Nazanin Moradinasab, Jiebei Liu, and Donald E. Brown. Can Vision-Language Models Count? A Synthetic Benchmark and Analysis of Attention-Based Interventions. arXiv:2511.17722, 2025.
- [22]Anushka Sivakumar, Andrew Zhang, Zaber Ibn Abdul Hakim, and Chris Thomas. SteerVLM: Robust Model Control through Lightweight Activation Steering for Vision Language Models. In Findings of EMNLP, pages 23640–23665, 2025.
- [23]Niklas Stoehr, Kevin Du, Vėsturinn Snæbjarnarson, Robert West, Ryan Cotterell, and Aaron Schein. Activation Scaling for Steering and Interpreting Language Models. In Findings of EMNLP, pages 8189–8200, 2024.
- [24]Jayant Sravan Tamarapalli, Rynaa Grover, Nilay Pande, and Sahiti Yerramilli. CountQA: How Well Do MLLMs Count in the Wild? arXiv:2508.06885, 2025.
- [25]Alberto Testolin, Kuinan Hou, and Marco Zorzi. Visual Enumeration Remains Challenging for Multimodal Generative AI. PLOS ONE, 20(9):e0331566, 2025.DOI
- [26]Alexander Trott, Caiming Xiong, and Richard Socher. Interpretable Counting for Visual Question Answering. In ICLR, 2018.arxiv.org/abs/1712.08697
- [27]Alexander Matt Turner, Lisa Thiergart, Gavin Leech, David Udell, Juan J. Vazquez, Ulisse Mini, and Monte MacDiarmid. Steering Language Models With Activation Engineering. arXiv:2308.10248, 2023.arxiv.org/abs/2308.10248
- [28]Jesse Vig, Sebastian Gehrmann, Yonatan Belinkov, Sharon Qian, Daniel Nevo, Yaron Singer, and Stuart Shieber. Investigating Gender Bias in Language Models Using Causal Mediation Analysis. In NeurIPS, 2020.
- [29]Qidong Wang, Junjie Hu, and Ming Jiang. V-SEAM: Visual Semantic Editing and Attention Modulating for Causal Interpretability of Vision-Language Models. In EMNLP, pages 17396–17420, 2025.arxiv.org/abs/2509.14837
- [30]Qidong Wang, Junjie Hu, and Ming Jiang. From Heads to Neurons: Causal Attribution and Steering in Multi-Task Vision–Language Models. In Findings of ACL, pages 36151–36175, 2026.
- [31]Jiaxi Yang, Shicheng Liu, Yuchen Yang, and Dongwon Lee. Steering to Say No: Configurable Refusal via Activation Steering in Vision Language Models. arXiv:2602.07013, 2026.arxiv.org/abs/2602.07013
- [32]Jianghao Yin, Qin Chen, Kedi Chen, Jie Zhou, Xingjian Wu, and Liang He. Dynamic Multimodal Activation Steering for Hallucination Mitigation in Large Vision-Language Models. In ICLR, 2026.arxiv.org/abs/2602.21704
- [33]Yan Zhang, Jonathon Hare, and Adam Prügel-Bennett. Learning to Count Objects in Natural Images for Visual Question Answering. In ICLR, 2018.arxiv.org/abs/1802.05766
- [34]Feiran Zhang, Yixin Wu, Zhenghua Wang, Xiaohua Wang, Changze Lv, Xuanjing Huang, and Xiaoqiang Zheng. VIB-Probe: Detecting and Mitigating Hallucinations in Vision-Language Models via Variational Information Bottleneck. In ACL, pages 23509–23521, 2026.
- [35]Andy Zou, Long Phan, Sarah Chen, James Campbell, Phillip Guo, Richard Ren, Alexander Pan, Xuwang Yin, Mantas Mazeika, Ann-Kathrin Dombrowski, Shashwat Goel, Nathaniel Li, Michael J. Byun, Zifan Wang, Alex Mallen, Steven Basart, Sanmi Koyejo, Dawn Song, Matt Fredrikson, J. Zico Kolter, and Dan Hendrycks. Representation Engineering: A Top-Down Approach to AI Transparency. arXiv:2310.01405, 2023.arxiv.org/abs/2310.01405
- [36]Duy Nguyen, Archiki Prasad, Elias Stengel-Eskin, and Mohit Bansal. GrAlNs: Gradient-based Attribution for Inference-Time Steering of LLMs and VLMs. arXiv:2507.18043, 2025.arxiv.org/abs/2507.18043



