betaFeedback

WorksComputer Science

From Concepts to Actual Causes: Toward Closing the Concept-Malfunction Gap in Robot Foundation Models

Abstract

Post-incident analysis of a robot foundation model poses a singular causal question: which internal components caused this failure, and to what degree? Mechanistic interpretability (MI), however, establishes general causal claims by decoding the concepts a policy has learned, yet knowing which concepts are encoded is insufficient for identifying the actual cause of a specific incident. In this position paper, we analyze this concept-malfunction gap by posing fault isolation in the framework of actual causality. Specifically, we formalize a fault-isolation query as a tuple of intervened internal components, counterfactual values, contingencies, and read-outs, and lay out conditions for it to be well-specified. Through this lens, we identify shortcomings of current MI approaches: they assume fixed contingencies and read out only immediate effects, letting backups obscure actual causes. We demonstrate on a toy example that these shortcomings can lead to conflicting verdicts. Lastly, based on the well-specification conditions, we outline four open research tasks toward closing the gap.

Introduction

As robot foundation models (RFMs)—e.g., vision-language-action models (VLAs) [1, 2, 3] and world action models (WAMs) [4]—enter spaces shared with humans, their failures can become serious incidents. Recent legislation such as the EU AI Act requires providers of high-risk AI systems, which can include autonomous robots, to investigate and report such incidents and implement corrective actions [5]. Since retraining the entire model is costly and may not even resolve the underlying fault, efficient compliance requires localizing the fault for targeted repair, which comes down to one question: which internal components actually caused the failure, and to what degree?

The closest toolkit for analyzing the inner workings of a foundation model is mechanistic interpretability (MI) [6, 7, 8]. However, MI is primarily designed to establish general causal claims across a distribution [9], whereas an individual incident calls for a singular one. This discrepancy creates a concept–malfunction gap: MI reveals which concepts a policy encodes, not which components caused the malfunction. The closed loop of RFMs widens this gap: unlike in language models, internal errors propagate through the environment and compound over time.

In this position paper, we argue that the concept–malfunction gap stems primarily from misspecified causal queries rather than deficient methods. To re-pose these queries, we ground fault isolation in the framework of actual causality [10, 11], assuming that the incident can be replayed in a deterministic simulator to compute the counterfactuals it requires.1 Our contributions are threefold: we (1) formally define a fault-isolation query QQ (§ 2); (2) diagnose how existing MI methods instantiate QQ via a toy example (§ 3); and (3) outline four open research tasks toward fault isolation in RFMs (§ 4).

The Science of Physical AI Safety (SPAIS) Workshop at the 10th Conference on Robot Learning (CoRL 2026), Austin TX, USA.

Fault Isolation as Actual Causation

In this section, we formalize fault isolation with actual causality [11] by treating an RFM’s closed-loop rollout as a structural causal model and introducing the fault-isolation query QQ.

Robot foundation models. We model an RFM (VLA or WAM) as a policy πθ\pi_\theta that maps an observation history o≤to_{\leq t} (e.g., images and proprioception) and an instruction ℓ\ell to an action chunk At∈RH×dA_t \in\mathbb{R}^{H \times d} consisting of HH future dd-dimensional actions (either discrete [1, 2] or continuous [3]), and additional future states for WAMs. We decompose the computation of πθ\pi_\theta at step tt into components Zt=(Zt1,…,Ztn)Z_t = (Z_t^1, \ldots, Z_t^n), the intermediate units an investigator can intervene on and blame, e.g., attention-head outputs, MLP neurons, dictionary latents, or a WAM’s predicted-state latents.

Incidents and fault isolation. The exogenous context uu (initial state, seed, instruction), a deterministic simulator, and πθ\pi_\theta determine the rollout τ\tau. A failure predicate φ(τ)∈{0,1}\varphi(\tau) \in\{0,1\} encodes the incident, e.g., the gripper coming within a safety distance of a human hand, or a grasped knife being dropped. Since each action shapes later observations, a component affects φ\varphi directly through its action and indirectly through the environment. With V={Zti}t,iV = \{Z_t^i\}_{t,i} collecting all components, fault isolation asks which subset X⊆VX \subseteq V caused φ(τ)=1\varphi(\tau) = 1 in this rollout, not how the policy behaves on average.

Counterfactuals and contingencies. Actual causality builds on but-for dependence: had XX taken another value, would the failure have been averted? To make this precise, we treat the unrolled rollout as a structural causal model MM [10] over VV and φ\varphi, with equations given by πθ\pi_\theta and the simulator. Under MM, an intervention [X←x′][X \leftarrow x'] sets XX to x′x' and re-runs the rollout, and (M,u)⊨[X←x′] ψ(M,u) \models[X \leftarrow x']\ \psi denotes that ψ\psi holds in the resulting counterfactual rollout; X=xX=x is thus a but-for cause if (M,u)⊨[X←x′] ¬φ(M,u) \models[X \leftarrow x']\ \neg\varphi for some x′x'. Yet a genuine cause may fail to be a but-for cause when a backup component would have produced the failure anyway (e.g., the Hydra effect [15]). Actual causality therefore also tests dependence under a contingency, which holds such backups at their observed values.2

Definition 1 (actual cause). Following the modified definition of Halpern [16], an assignment X=xX=x is an actual cause of φ\varphi in context uu if all three conditions hold:

AC1 (factuality) X=xX=x and φ\varphi both hold in the observed rollout;

AC2 (counterfactual dependence) for some contingency W⊆V∖XW \subseteq V \setminus X with observed values w∗w^* and some alternative value x′x', (M,u)⊨[X←x′,W←w∗]¬φ(M,u) \models[X \leftarrow x', W \leftarrow w^*]\neg\varphi, i.e., the failure would have been averted had XX been x′x' and WW not resp ected;

AC3 (minimality) no strict subset of XX satisfies AC1 and AC2.

Such WW is called a witness [11], and but-for causation is the case W=∅W = \varnothing. Holding WW at w∗w^* amounts to activation patching [17], where factual activations are used during the counterfactual rollout.

Definition 2 (degree of responsibility). Following Chockler and Halpern [18], as adapted to the modified definition [11 Ch. 6], the degree of responsibility of an assignment Z=zZ=z with Z∈VZ \in V for φ\varphi is 1/k1/k, where kk is the minimum of ∣X∣+∣W∣|X| + |W| over all actual causes X=xX=x that include Z=zZ=z and their witnesses WW; it is 0 if no such cause exists.

Definition 1 is binary: it cannot distinguish a cause with one backup from a cause with ten, nor rank candidate causes for repair. Definition 2 grades them, as kk counts the components that must be changed (XX) or held fixed (WW) for φ\varphi to depend on ZZ. A single-variable but-for cause thus has responsibility 1, whereas a cause with one or ten backups has 1/21/2 or 1/111/11. A component changed or held at several steps counts as a single variable, both in kk and in AC3.

Definition 3 (fault-isolation query). A fault-isolation query Q=⟨X,x′,W,Y⟩Q = \langle X, x', W, Y\rangle consists of intervened components X⊆VX \subseteq V, a counterfactual value x′x', a contingency W⊆V∖XW \subseteq V \setminus X, and a read-out YY; it tests whether YY changes under [X←x′,W←w∗][X \leftarrow x', W \leftarrow w^*]. We call QQ well-specified for an RFM incident if all four conditions hold, and mispecified otherwise:

WS1 (temporal scope) XX may include components at any step;

WS2 (realizability) x′x' can be induced by some observation history;

WS3 (searched contingency) WW is searched per incident, not fixed in advance;

WS4 (rollout outcome) YY is φ\varphi evaluated on the counterfactual rollout.

A misspecified QQ can reach a wrong verdict: a single-step XX misses faults that accumulate over steps (WS1); an unrealizable x′x' may avert φ\varphi merely by driving the policy off-manifold (WS2; cf. normality [19]); a fixed WW may hide causes that have backups, or misgrade them (WS3); and a per-step read-out can miss effects mediated by the environment (WS4).

Handling backups in RFMs. (i) Loop backups and realizability: in RFMs, the policy itself can undo a repair at later steps (e.g., a gripper stopped short of its goal simply moves toward it again), so a witness must also freeze such later reactions. A cause revealed this way is a persistent fault: repairing it alone will not prevent the failure. Such witnesses, however, may force combinations the policy would never produce (e.g., a slowed robot whose drive stays idle), so their verdicts weaken as the counterfactual drifts from the factual rollout. (ii) Granularity and responsibility: responsibility (Definition 2) counts how many components must be frozen, which is meaningful only if each backup is one component; sparse autoencoder latents [20] approximate this better than individual neurons.

The Concept–Malfunction Gap in Current MI Methods

In this section, we view existing MI methods as instantiations of QQ, check whether they are well-specified, and illustrate how their misspecifications may lead to conflicting verdicts via an example.

Existing MI methods as instantiations of QQ. We examine six MI method families and summarize in Table 1 how each instantiates QQ. None yields a well-specified query. Violating only WS1 or WS3 would merely miss causes, but every intervening family also violates WS2, so even its reported causes may be spurious. Specifically, they violate temporal scope (WS1) by intervening within a single step or only on the input (except steering); realizability (WS2) by using counterfactual values that the incident may never produce, such as masked images, other inputs [17], or steered states; searched contingency (WS3) by fixing WW in advance or averaging it over inputs, thus ignoring the incident’s backups; and rollout outcome (WS4) by reading out a per-step surrogate such as the action AtA_t (except visual attribution and steering), which misses what the loop does next.3

Condition →WS1WS2WS3WS4
Method ↓scope of Xrealizability of x′flexibility of Whorizon of Y
Well-specifiedacross stepsrealizablesearched per incidentfailure φ\varphi, full rollout
Probing / SAEs [7, 8, 20]✗ no intervention✗ —✗ —✗ latent value, one step
Visual attribution [6]✗ input image only✗ masking✗ ∅\varnothing✓ action or rollout
Activation / path patching [21, 17]✗ one step✗ other input✗ ∅\varnothing or paths✗ logits, one step
Causal scrubbing [22]✗ one step✗ resampling✗ set by hypothesis✗ loss, one step
Activation steering [23, 24, 25]✓ all steps✗ steering✗ ∅\varnothing✓ rollout behavior
Causal abstraction [9]✗ one step✗ source input✗ averaged over inputs✗ high-level var., one step
Open research tasksT1: finding faultsT2: certifying x′x'T3: scaling contingency searchT1: finding faults
T4: validating fault isolation (all conditions jointly)

Table 1. The requirements of a well-specified query (top), existing MI methods as instantiations of Q (middle), and open research tasks toward meeting them (bottom). ✓/✗: met/not met.

A toy example. In our scenario, a policy drives a robot toward a human 2 m away at vref=1v_{\mathrm{ref}} = 1 m/s (Figure 1a; simulated at 10 Hz for 5 s), and the incident φ=1\varphi= 1 occurs if the robot comes within 0.3 m of the human. The policy sets the acceleration at=−2ht(1−st)+gta_t = -2h_t(1-s_t) + g_t through three components: a hazard latent ht=1h_t = 1 that fires when a person is detected triggering braking; a suppressor st∈[0,1]s_t \in[0,1] that blocks this braking while a task is active; and a target-reaching drive gt=4max⁡(0,vref−vt)g_t = 4\max(0, v_{\mathrm{ref}} - v_t) that accelerates whenever the robot falls below its reference speed. By design, the suppressor is always on (st=1s_t = 1), so the robot never brakes or triggers the drive (gt=0g_t = 0), breaching at 1.7 s. The suppressor ss is thus an actual cause: a sound method should blame ss, not the hazard latent hh (active, but blocked by ss) or the drive gg (idle throughout). Yet gg acts as a backup; even if ss were repaired, gg would keep driving the robot toward the human until it breaches, so a fix must address both ss and gg.

Toy example, single-pass patching, and well-specified query

Figure 1. (a) A toy example: the suppressor ss (actual cause) blocks braking on the hazard latent hh, and the drive gg (its backup) re-accelerates the robot through the loop. (b) Single-pass patching reads out only the immediate effect of the intervention; (c) the well-specified query, with contingency g←0g \leftarrow0, correctly identifies s=1s=1 as an actual cause.

Conflicting verdicts. We now demonstrate that applying the MI methods in Table 1 yields conflicting verdicts. First, probing finds the hazard encoded in hh, yet cannot explain why the policy failed to brake even though it “saw” the human. Visual attribution sees no change when the human is masked, as ss already blocks braking regardless of hh, and in any case blames pixels rather than internal3: Some of these gaps are among the open problems in MI identified by Sharkey et al. [26]: single-step analysis (WS1), backup behavior (WS3), and direct-effect-only read-outs (WS4). components. Next, single-pass activation patching of s0←0s_0 \leftarrow0 (Figure 1b) initially yields full braking (∣Δa0∣=2 m/s2|\Delta a_0| = 2\,\mathrm{m/s}^2), but a counterfactual rollout reveals that the breach still occurs, only delayed by 0.1 s (WS1, WS4). Patching ss at all steps also fails to isolate ss as a cause, as the backup gg settles the robot at 0.5 m/s, still resulting in a breach at 3.3 s (WS3); causal scrubbing and causal abstraction, which also fix WW, inherit this blind spot. Lastly, steering s←1−αs \leftarrow1-\alpha finds no realizable α≤1\alpha\le1 that averts φ\varphi, since gg always restores the robot’s speed (WS3). Only the well-specified query on ss, freezing gg (Figure 1c), recovers the ground truth: s=1s=1 is an actual cause with responsibility 1/21/2 (X={s}X=\{s\}, W={g}W=\{g\}), albeit via an off-manifold witness (§ 2).

Open Research Tasks

In this section, we identify four open research tasks, each arising from meeting a well-specification condition of QQ (Definition 3) at the scale of real RFMs (Table 1, bottom).

Task 1: Finding candidate faults (WS1, WS4). An investigator must decide which components to test as faults (XX), as testing every component is intractable. As modern RFMs generate continuous action chunks [3], this may require an action lens, a counterpart of the logit lens [27] that projects components onto problematic actions; its candidates must then be tested in counterfactual rollouts.

Task 2: Certifying realizable counterfactuals (WS2). In an RFM, a patched activation may not be induced by any observation history. Alternatively, activation steering along a direction δ\delta separating safe from unsafe behavior (x′=x+αδx^\prime=x+\alpha\delta) makes the counterfactual more likely to stay within the policy’s representations, but offers no guarantee that any observation history actually induces it.

Task 3: Scaling contingency search (WS3). Given a candidate fault, its contingency (WW) must be found. In an RFM, this means searching among millions of latents, including dormant ones. As deciding actual causation is DPD^P-complete even for explicit models [16], candidate backups must be shortlisted, e.g., by attribution [28].

Task 4: Validating fault isolation. Judging whether a fault was correctly isolated requires knowing which components caused the incident. Naturally trained RFMs rarely offer this, as no fault is built in by design. Planting faults with known causes and backups, e.g., via interchange intervention training [29], could provide ground truth, but planted faults must be as realistic as natural ones.

Conclusion

In this paper, we addressed the concept–malfunction gap in RFMs: the discrepancy between the concepts that interpretability research aims to uncover and the actual causes of a specific malfunction. Formalizing fault isolation with actual causality revealed that the gap stems primarily from how existing MI methods specify the causal query rather than from their tools. Specifically, they assume fixed contingencies and read out only immediate effects, which lets backup components affect the counterfactual rollout and obscure actual causes. Answering well-specified queries at the scale of real RFMs still requires new tools, for which we outlined four open research tasks.

Limitations. As this is a position paper, we provide no experiments on real RFMs: our argument rests on a toy example and assumes a faithful simulator, leaving empirical validation to future work.

Acknowledgments

This work was supported by National IT Industry Promotion Agency (NIPA) grant funded by the Korea government(MSIT) (Development of Physical Intelligence Action Models and an Integrated Framework, No. RS-2026-25621677).

References

  1. [1]B. Zitkovich, T. Yu, S. Xu, P. Xu, T. Xiao, E. Xia, J. Wu, P. Wohlhart, et al. RT-2: Vision-language-action models transfer web knowledge to robotic control. In Conference on Robot Learning (CoRL), pages 2165–2183, 2023.arxiv.org/abs/2307.15818
  2. [2]M. J. Kim, K. Pertsch, S. Karamcheti, T. Xiao, A. Balakrishna, S. Nair, R. Rafailov, E. Foster, et al. OpenVLA: An open-source vision-language-action model. In Conference on Robot Learning (CoRL), pages 2679–2713, 2024.arxiv.org/abs/2406.09246
  3. [3]K. Black, N. Brown, D. Driess, A. Esmail, M. Equi, C. Finn, N. Fusai, L. Groom, et al. π₀: A vision-language-action flow model for general robot control. In Robotics: Science and Systems (RSS), 2025.DOI
  4. [4]S. Ye, Y. Ge, K. Zheng, S. Gao, S. Yu, G. Kurian, S. Indupuru, Y. L. Tan, et al. World action models are zero-shot policies. arXiv preprint arXiv:2602.15922, 2026.arxiv.org/abs/2602.15922
  5. [5]European Parliament and Council of the European Union. Regulation (EU) 2024/1689 laying down harmonised rules on artificial intelligence (Artificial Intelligence Act). Official Journal of the European Union, L 2024/1689, 2024. URL https://eur-lex.europa.eu/eli/reg/2024/1689/oj.
  6. [6]H. Zhang, M. Xu, A. Dhafer, S. Yue, H. Dong, and Z. D. Hao. Embodied interpretability: Linking causal understanding to generalization in vision-language-action models. In International Conference on Machine Learning (ICML), 2026.arxiv.org/abs/2605.00321
  7. [7]H. Mai, B. Zhu, and T. Do. Beyond task success: Behavioral and representational diagnostics for WAM and VLA. arXiv preprint arXiv:2606.01095, 2026.arxiv.org/abs/2606.01095
  8. [8]A. Swann, L. McGranahan, H. Buurmeijer, M. Kennedy, III, and M. Schwager. Sparse autoencoders reveal interpretable and steerable features in VLA models. In Conference on Robot Learning (CoRL), 2026.arxiv.org/abs/2603.19183
  9. [9]A. Geiger, D. Ibeling, A. Zur, M. Chaudhary, S. Chauhan, J. Huang, A. Arora, Z. Wu, et al. Causal abstraction: A theoretical foundation for mechanistic interpretability. Journal of Machine Learning Research, 26(83):1–64, 2025.
  10. [10]J. Y. Halpern and J. Pearl. Causes and explanations: A structural-model approach. Part I: Causes. The British Journal for the Philosophy of Science, 56(4):843–887, 2005.
  11. [11]J. Y. Halpern. Actual Causality. MIT Press, 2016.
  12. [12]R. Isermann. Model-based fault-detection and diagnosis—status and applications. Annual Reviews in Control, 29(1):71–85, 2005.
  13. [13]I. Beer, S. Ben-David, H. Chockler, A. Orni, and R. Trefler. Explaining counterexamples using causality. In International Conference on Computer Aided Verification (CAV), pages 94–108, 2009.
  14. [14]S. Triantafyllou, A. Singla, and G. Radanovic. Actual causality and responsibility attribution in decentralized partially observable Markov decision processes. In AAAI/ACM Conference on AI, Ethics, and Society (AIES), pages 739–752, 2022.arxiv.org/abs/2204.00302
  15. [15]T. McGrath, M. Rahtz, J. Kramár, V. Mikulik, and S. Legg. The Hydra effect: Emergent self-repair in language model computations. arXiv preprint arXiv:2307.15771, 2023.arxiv.org/abs/2307.15771
  16. [16]J. Y. Halpern. A modification of the Halpern–Pearl definition of causality. In International Joint Conference on Artificial Intelligence (IJCAI), pages 3022–3033, 2015.DOI
  17. [17]F. Zhang and N. Nanda. Towards best practices of activation patching in language models: Metrics and methods. In International Conference on Learning Representations (ICLR), 2024.arxiv.org/abs/2309.16042
  18. [18]H. Chockler and J. Y. Halpern. Responsibility and blame: A structural-model approach. Journal of Artificial Intelligence Research, 22:93–115, 2004.
  19. [19]J. Y. Halpern and C. Hitchcock. Graded causation and defaults. The British Journal for the Philosophy of Science, 66(2):413–457, 2015.
  20. [20]R. Huben, H. Cunningham, L. R. Smith, A. Ewart, and L. Sharkey. Sparse autoencoders find highly interpretable features in language models. In International Conference on Learning Representations (ICLR), 2024.arxiv.org/abs/2309.08600
  21. [21]K. Wang, A. Variengien, A. Conmy, B. Shlegeris, and J. Steinhardt. Interpretability in the wild: A circuit for indirect object identification in GPT-2 small. In International Conference on Learning Representations (ICLR), 2023.arxiv.org/abs/2211.00593
  22. [22]L. Chan, A. Garriga-Alonso, N. Goldowsky-Dill, R. Greenblatt, J. Nitishinskaya, A. Radhakrishnan, B. Shlegeris, and N. Thomas. Causal scrubbing: A method for rigorously testing interpretability hypotheses. AI Alignment Forum, 2022. URL https://www.alignmentforum.org/posts/JvZhhzycHu2Yd57RN.
  23. [23]A. M. Turner, L. Thiergart, G. Leech, D. Udell, J. J. Vazquez, U. Mini, and M. MacDiarmid. Steering language models with activation engineering. arXiv preprint arXiv:2308.10248, 2023.arxiv.org/abs/2308.10248
  24. [24]N. Rimsky, N. Gabrieli, J. Schulz, M. Tong, E. Hubinger, and A. M. Turner. Steering Llama 2 via contrastive activation addition. In Annual Meeting of the Association for Computational Linguistics (ACL), pages 15504–15522, 2024.arxiv.org/abs/2312.06681
  25. [25]B. Håon, K. Stocking, I. Chuang, and C. Tomlin. Mechanistic interpretability for steering vision-language-action models. In Conference on Robot Learning (CoRL), volume 305 of Proceedings of Machine Learning Research, pages 2743–2762, 2025.arxiv.org/abs/2509.00328
  26. [26]L. Sharkey, B. Chughtai, J. Batson, J. Lindsey, J. Wu, L. Bushnaq, N. Goldowsky-Dill, S. Heimer, sheim, et al. Open problems in mechanistic interpretability. Transactions on Machine Learning Research (TMLR), 2025.arxiv.org/abs/2501.16496
  27. [27]nostalgebraist. Interpreting GPT: The logit lens. LessWrong, 2020. URL https://www.lesswrong.com/posts/AcKRB8wDpdaN6v6ru.
  28. [28]A. Syed, C. Rager, and A. Conmy. Attribution patching outperforms automated circuit discovery. In BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP (BlackboxNLP), pages 407–416, 2024.arxiv.org/abs/2310.10348
  29. [29]A. Geiger, Z. Wu, H. Lu, J. Rozner, E. Kreiss, T. Icard, N. Goodman, and C. Potts. Inducing causal structure for interpretable neural networks. In International Conference on Machine Learning (ICML), pages 7324–7338, 2022.arxiv.org/abs/2112.00826

Paper details

vlnc:W_as78rkku
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 9

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents