WorksComputer Science
From Concepts to Actual Causes: Toward Closing the Concept-Malfunction Gap in Robot Foundation Models
Abstract
Post-incident analysis of a robot foundation model poses a singular causal question: which internal components caused this failure, and to what degree? Mechanistic interpretability (MI), however, establishes general causal claims by decoding the concepts a policy has learned, yet knowing which concepts are encoded is insufficient for identifying the actual cause of a specific incident. In this position paper, we analyze this concept-malfunction gap by posing fault isolation in the framework of actual causality. Specifically, we formalize a fault-isolation query as a tuple of intervened internal components, counterfactual values, contingencies, and read-outs, and lay out conditions for it to be well-specified. Through this lens, we identify shortcomings of current MI approaches: they assume fixed contingencies and read out only immediate effects, letting backups obscure actual causes. We demonstrate on a toy example that these shortcomings can lead to conflicting verdicts. Lastly, based on the well-specification conditions, we outline four open research tasks toward closing the gap.
Introduction
As robot foundation models (RFMs)—e.g., vision-language-action models (VLAs) [1, 2, 3] and world action models (WAMs) [4]—enter spaces shared with humans, their failures can become serious incidents. Recent legislation such as the EU AI Act requires providers of high-risk AI systems, which can include autonomous robots, to investigate and report such incidents and implement corrective actions [5]. Since retraining the entire model is costly and may not even resolve the underlying fault, efficient compliance requires localizing the fault for targeted repair, which comes down to one question: which internal components actually caused the failure, and to what degree?
The closest toolkit for analyzing the inner workings of a foundation model is mechanistic interpretability (MI) [6, 7, 8]. However, MI is primarily designed to establish general causal claims across a distribution [9], whereas an individual incident calls for a singular one. This discrepancy creates a concept–malfunction gap: MI reveals which concepts a policy encodes, not which components caused the malfunction. The closed loop of RFMs widens this gap: unlike in language models, internal errors propagate through the environment and compound over time.
In this position paper, we argue that the concept–malfunction gap stems primarily from misspecified causal queries rather than deficient methods. To re-pose these queries, we ground fault isolation in the framework of actual causality [10, 11], assuming that the incident can be replayed in a deterministic simulator to compute the counterfactuals it requires.1 Our contributions are threefold: we (1) formally define a fault-isolation query (§ 2); (2) diagnose how existing MI methods instantiate via a toy example (§ 3); and (3) outline four open research tasks toward fault isolation in RFMs (§ 4).
The Science of Physical AI Safety (SPAIS) Workshop at the 10th Conference on Robot Learning (CoRL 2026), Austin TX, USA.
Fault Isolation as Actual Causation
In this section, we formalize fault isolation with actual causality [11] by treating an RFM’s closed-loop rollout as a structural causal model and introducing the fault-isolation query .
Robot foundation models. We model an RFM (VLA or WAM) as a policy that maps an observation history (e.g., images and proprioception) and an instruction to an action chunk consisting of future -dimensional actions (either discrete [1, 2] or continuous [3]), and additional future states for WAMs. We decompose the computation of at step into components , the intermediate units an investigator can intervene on and blame, e.g., attention-head outputs, MLP neurons, dictionary latents, or a WAM’s predicted-state latents.
Incidents and fault isolation. The exogenous context (initial state, seed, instruction), a deterministic simulator, and determine the rollout . A failure predicate encodes the incident, e.g., the gripper coming within a safety distance of a human hand, or a grasped knife being dropped. Since each action shapes later observations, a component affects directly through its action and indirectly through the environment. With collecting all components, fault isolation asks which subset caused in this rollout, not how the policy behaves on average.
Counterfactuals and contingencies. Actual causality builds on but-for dependence: had taken another value, would the failure have been averted? To make this precise, we treat the unrolled rollout as a structural causal model [10] over and , with equations given by and the simulator. Under , an intervention sets to and re-runs the rollout, and denotes that holds in the resulting counterfactual rollout; is thus a but-for cause if for some . Yet a genuine cause may fail to be a but-for cause when a backup component would have produced the failure anyway (e.g., the Hydra effect [15]). Actual causality therefore also tests dependence under a contingency, which holds such backups at their observed values.2
Definition 1 (actual cause). Following the modified definition of Halpern [16], an assignment is an actual cause of in context if all three conditions hold:
AC1 (factuality) and both hold in the observed rollout;
AC2 (counterfactual dependence) for some contingency with observed values and some alternative value , , i.e., the failure would have been averted had been and not resp ected;
AC3 (minimality) no strict subset of satisfies AC1 and AC2.
Such is called a witness [11], and but-for causation is the case . Holding at amounts to activation patching [17], where factual activations are used during the counterfactual rollout.
Definition 2 (degree of responsibility). Following Chockler and Halpern [18], as adapted to the modified definition [11 Ch. 6], the degree of responsibility of an assignment with for is , where is the minimum of over all actual causes that include and their witnesses ; it is 0 if no such cause exists.
Definition 1 is binary: it cannot distinguish a cause with one backup from a cause with ten, nor rank candidate causes for repair. Definition 2 grades them, as counts the components that must be changed () or held fixed () for to depend on . A single-variable but-for cause thus has responsibility 1, whereas a cause with one or ten backups has or . A component changed or held at several steps counts as a single variable, both in and in AC3.
Definition 3 (fault-isolation query). A fault-isolation query consists of intervened components , a counterfactual value , a contingency , and a read-out ; it tests whether changes under . We call well-specified for an RFM incident if all four conditions hold, and mispecified otherwise:
WS1 (temporal scope) may include components at any step;
WS2 (realizability) can be induced by some observation history;
WS3 (searched contingency) is searched per incident, not fixed in advance;
WS4 (rollout outcome) is evaluated on the counterfactual rollout.
A misspecified can reach a wrong verdict: a single-step misses faults that accumulate over steps (WS1); an unrealizable may avert merely by driving the policy off-manifold (WS2; cf. normality [19]); a fixed may hide causes that have backups, or misgrade them (WS3); and a per-step read-out can miss effects mediated by the environment (WS4).
Handling backups in RFMs. (i) Loop backups and realizability: in RFMs, the policy itself can undo a repair at later steps (e.g., a gripper stopped short of its goal simply moves toward it again), so a witness must also freeze such later reactions. A cause revealed this way is a persistent fault: repairing it alone will not prevent the failure. Such witnesses, however, may force combinations the policy would never produce (e.g., a slowed robot whose drive stays idle), so their verdicts weaken as the counterfactual drifts from the factual rollout. (ii) Granularity and responsibility: responsibility (Definition 2) counts how many components must be frozen, which is meaningful only if each backup is one component; sparse autoencoder latents [20] approximate this better than individual neurons.
The Concept–Malfunction Gap in Current MI Methods
In this section, we view existing MI methods as instantiations of , check whether they are well-specified, and illustrate how their misspecifications may lead to conflicting verdicts via an example.
Existing MI methods as instantiations of . We examine six MI method families and summarize in Table 1 how each instantiates . None yields a well-specified query. Violating only WS1 or WS3 would merely miss causes, but every intervening family also violates WS2, so even its reported causes may be spurious. Specifically, they violate temporal scope (WS1) by intervening within a single step or only on the input (except steering); realizability (WS2) by using counterfactual values that the incident may never produce, such as masked images, other inputs [17], or steered states; searched contingency (WS3) by fixing in advance or averaging it over inputs, thus ignoring the incident’s backups; and rollout outcome (WS4) by reading out a per-step surrogate such as the action (except visual attribution and steering), which misses what the loop does next.3
| Condition → | WS1 | WS2 | WS3 | WS4 |
| Method ↓ | scope of X | realizability of x′ | flexibility of W | horizon of Y |
| Well-specified | across steps | realizable | searched per incident | failure , full rollout |
| Probing / SAEs [7, 8, 20] | ✗ no intervention | ✗ — | ✗ — | ✗ latent value, one step |
| Visual attribution [6] | ✗ input image only | ✗ masking | ✗ | ✓ action or rollout |
| Activation / path patching [21, 17] | ✗ one step | ✗ other input | ✗ or paths | ✗ logits, one step |
| Causal scrubbing [22] | ✗ one step | ✗ resampling | ✗ set by hypothesis | ✗ loss, one step |
| Activation steering [23, 24, 25] | ✓ all steps | ✗ steering | ✗ | ✓ rollout behavior |
| Causal abstraction [9] | ✗ one step | ✗ source input | ✗ averaged over inputs | ✗ high-level var., one step |
| Open research tasks | T1: finding faults | T2: certifying | T3: scaling contingency search | T1: finding faults |
| T4: validating fault isolation (all conditions jointly) |
Table 1. The requirements of a well-specified query (top), existing MI methods as instantiations of Q (middle), and open research tasks toward meeting them (bottom). ✓/✗: met/not met.
A toy example. In our scenario, a policy drives a robot toward a human 2 m away at m/s (Figure 1a; simulated at 10 Hz for 5 s), and the incident occurs if the robot comes within 0.3 m of the human. The policy sets the acceleration through three components: a hazard latent that fires when a person is detected triggering braking; a suppressor that blocks this braking while a task is active; and a target-reaching drive that accelerates whenever the robot falls below its reference speed. By design, the suppressor is always on (), so the robot never brakes or triggers the drive (), breaching at 1.7 s. The suppressor is thus an actual cause: a sound method should blame , not the hazard latent (active, but blocked by ) or the drive (idle throughout). Yet acts as a backup; even if were repaired, would keep driving the robot toward the human until it breaches, so a fix must address both and .

Figure 1. (a) A toy example: the suppressor (actual cause) blocks braking on the hazard latent , and the drive (its backup) re-accelerates the robot through the loop. (b) Single-pass patching reads out only the immediate effect of the intervention; (c) the well-specified query, with contingency , correctly identifies as an actual cause.
Conflicting verdicts. We now demonstrate that applying the MI methods in Table 1 yields conflicting verdicts. First, probing finds the hazard encoded in , yet cannot explain why the policy failed to brake even though it “saw” the human. Visual attribution sees no change when the human is masked, as already blocks braking regardless of , and in any case blames pixels rather than internal3: Some of these gaps are among the open problems in MI identified by Sharkey et al. [26]: single-step analysis (WS1), backup behavior (WS3), and direct-effect-only read-outs (WS4). components. Next, single-pass activation patching of (Figure 1b) initially yields full braking (), but a counterfactual rollout reveals that the breach still occurs, only delayed by 0.1 s (WS1, WS4). Patching at all steps also fails to isolate as a cause, as the backup settles the robot at 0.5 m/s, still resulting in a breach at 3.3 s (WS3); causal scrubbing and causal abstraction, which also fix , inherit this blind spot. Lastly, steering finds no realizable that averts , since always restores the robot’s speed (WS3). Only the well-specified query on , freezing (Figure 1c), recovers the ground truth: is an actual cause with responsibility (, ), albeit via an off-manifold witness (§ 2).
Open Research Tasks
In this section, we identify four open research tasks, each arising from meeting a well-specification condition of (Definition 3) at the scale of real RFMs (Table 1, bottom).
Task 1: Finding candidate faults (WS1, WS4). An investigator must decide which components to test as faults (), as testing every component is intractable. As modern RFMs generate continuous action chunks [3], this may require an action lens, a counterpart of the logit lens [27] that projects components onto problematic actions; its candidates must then be tested in counterfactual rollouts.
Task 2: Certifying realizable counterfactuals (WS2). In an RFM, a patched activation may not be induced by any observation history. Alternatively, activation steering along a direction separating safe from unsafe behavior () makes the counterfactual more likely to stay within the policy’s representations, but offers no guarantee that any observation history actually induces it.
Task 3: Scaling contingency search (WS3). Given a candidate fault, its contingency () must be found. In an RFM, this means searching among millions of latents, including dormant ones. As deciding actual causation is -complete even for explicit models [16], candidate backups must be shortlisted, e.g., by attribution [28].
Task 4: Validating fault isolation. Judging whether a fault was correctly isolated requires knowing which components caused the incident. Naturally trained RFMs rarely offer this, as no fault is built in by design. Planting faults with known causes and backups, e.g., via interchange intervention training [29], could provide ground truth, but planted faults must be as realistic as natural ones.
Conclusion
In this paper, we addressed the concept–malfunction gap in RFMs: the discrepancy between the concepts that interpretability research aims to uncover and the actual causes of a specific malfunction. Formalizing fault isolation with actual causality revealed that the gap stems primarily from how existing MI methods specify the causal query rather than from their tools. Specifically, they assume fixed contingencies and read out only immediate effects, which lets backup components affect the counterfactual rollout and obscure actual causes. Answering well-specified queries at the scale of real RFMs still requires new tools, for which we outlined four open research tasks.
Limitations. As this is a position paper, we provide no experiments on real RFMs: our argument rests on a toy example and assumes a faithful simulator, leaving empirical validation to future work.
Acknowledgments
This work was supported by National IT Industry Promotion Agency (NIPA) grant funded by the Korea government(MSIT) (Development of Physical Intelligence Action Models and an Integrated Framework, No. RS-2026-25621677).
References
- [1]B. Zitkovich, T. Yu, S. Xu, P. Xu, T. Xiao, E. Xia, J. Wu, P. Wohlhart, et al. RT-2: Vision-language-action models transfer web knowledge to robotic control. In Conference on Robot Learning (CoRL), pages 2165–2183, 2023.arxiv.org/abs/2307.15818
- [2]M. J. Kim, K. Pertsch, S. Karamcheti, T. Xiao, A. Balakrishna, S. Nair, R. Rafailov, E. Foster, et al. OpenVLA: An open-source vision-language-action model. In Conference on Robot Learning (CoRL), pages 2679–2713, 2024.arxiv.org/abs/2406.09246
- [3]K. Black, N. Brown, D. Driess, A. Esmail, M. Equi, C. Finn, N. Fusai, L. Groom, et al. π₀: A vision-language-action flow model for general robot control. In Robotics: Science and Systems (RSS), 2025.DOI
- [4]S. Ye, Y. Ge, K. Zheng, S. Gao, S. Yu, G. Kurian, S. Indupuru, Y. L. Tan, et al. World action models are zero-shot policies. arXiv preprint arXiv:2602.15922, 2026.arxiv.org/abs/2602.15922
- [5]European Parliament and Council of the European Union. Regulation (EU) 2024/1689 laying down harmonised rules on artificial intelligence (Artificial Intelligence Act). Official Journal of the European Union, L 2024/1689, 2024. URL https://eur-lex.europa.eu/eli/reg/2024/1689/oj.
- [6]H. Zhang, M. Xu, A. Dhafer, S. Yue, H. Dong, and Z. D. Hao. Embodied interpretability: Linking causal understanding to generalization in vision-language-action models. In International Conference on Machine Learning (ICML), 2026.arxiv.org/abs/2605.00321
- [7]H. Mai, B. Zhu, and T. Do. Beyond task success: Behavioral and representational diagnostics for WAM and VLA. arXiv preprint arXiv:2606.01095, 2026.arxiv.org/abs/2606.01095
- [8]A. Swann, L. McGranahan, H. Buurmeijer, M. Kennedy, III, and M. Schwager. Sparse autoencoders reveal interpretable and steerable features in VLA models. In Conference on Robot Learning (CoRL), 2026.arxiv.org/abs/2603.19183
- [9]A. Geiger, D. Ibeling, A. Zur, M. Chaudhary, S. Chauhan, J. Huang, A. Arora, Z. Wu, et al. Causal abstraction: A theoretical foundation for mechanistic interpretability. Journal of Machine Learning Research, 26(83):1–64, 2025.
- [10]J. Y. Halpern and J. Pearl. Causes and explanations: A structural-model approach. Part I: Causes. The British Journal for the Philosophy of Science, 56(4):843–887, 2005.
- [11]J. Y. Halpern. Actual Causality. MIT Press, 2016.
- [12]R. Isermann. Model-based fault-detection and diagnosis—status and applications. Annual Reviews in Control, 29(1):71–85, 2005.
- [13]I. Beer, S. Ben-David, H. Chockler, A. Orni, and R. Trefler. Explaining counterexamples using causality. In International Conference on Computer Aided Verification (CAV), pages 94–108, 2009.
- [14]S. Triantafyllou, A. Singla, and G. Radanovic. Actual causality and responsibility attribution in decentralized partially observable Markov decision processes. In AAAI/ACM Conference on AI, Ethics, and Society (AIES), pages 739–752, 2022.arxiv.org/abs/2204.00302
- [15]T. McGrath, M. Rahtz, J. Kramár, V. Mikulik, and S. Legg. The Hydra effect: Emergent self-repair in language model computations. arXiv preprint arXiv:2307.15771, 2023.arxiv.org/abs/2307.15771
- [16]J. Y. Halpern. A modification of the Halpern–Pearl definition of causality. In International Joint Conference on Artificial Intelligence (IJCAI), pages 3022–3033, 2015.DOI
- [17]F. Zhang and N. Nanda. Towards best practices of activation patching in language models: Metrics and methods. In International Conference on Learning Representations (ICLR), 2024.arxiv.org/abs/2309.16042
- [18]H. Chockler and J. Y. Halpern. Responsibility and blame: A structural-model approach. Journal of Artificial Intelligence Research, 22:93–115, 2004.
- [19]J. Y. Halpern and C. Hitchcock. Graded causation and defaults. The British Journal for the Philosophy of Science, 66(2):413–457, 2015.
- [20]R. Huben, H. Cunningham, L. R. Smith, A. Ewart, and L. Sharkey. Sparse autoencoders find highly interpretable features in language models. In International Conference on Learning Representations (ICLR), 2024.arxiv.org/abs/2309.08600
- [21]K. Wang, A. Variengien, A. Conmy, B. Shlegeris, and J. Steinhardt. Interpretability in the wild: A circuit for indirect object identification in GPT-2 small. In International Conference on Learning Representations (ICLR), 2023.arxiv.org/abs/2211.00593
- [22]L. Chan, A. Garriga-Alonso, N. Goldowsky-Dill, R. Greenblatt, J. Nitishinskaya, A. Radhakrishnan, B. Shlegeris, and N. Thomas. Causal scrubbing: A method for rigorously testing interpretability hypotheses. AI Alignment Forum, 2022. URL https://www.alignmentforum.org/posts/JvZhhzycHu2Yd57RN.
- [23]A. M. Turner, L. Thiergart, G. Leech, D. Udell, J. J. Vazquez, U. Mini, and M. MacDiarmid. Steering language models with activation engineering. arXiv preprint arXiv:2308.10248, 2023.arxiv.org/abs/2308.10248
- [24]N. Rimsky, N. Gabrieli, J. Schulz, M. Tong, E. Hubinger, and A. M. Turner. Steering Llama 2 via contrastive activation addition. In Annual Meeting of the Association for Computational Linguistics (ACL), pages 15504–15522, 2024.arxiv.org/abs/2312.06681
- [25]B. Håon, K. Stocking, I. Chuang, and C. Tomlin. Mechanistic interpretability for steering vision-language-action models. In Conference on Robot Learning (CoRL), volume 305 of Proceedings of Machine Learning Research, pages 2743–2762, 2025.arxiv.org/abs/2509.00328
- [26]L. Sharkey, B. Chughtai, J. Batson, J. Lindsey, J. Wu, L. Bushnaq, N. Goldowsky-Dill, S. Heimer, sheim, et al. Open problems in mechanistic interpretability. Transactions on Machine Learning Research (TMLR), 2025.arxiv.org/abs/2501.16496
- [27]nostalgebraist. Interpreting GPT: The logit lens. LessWrong, 2020. URL https://www.lesswrong.com/posts/AcKRB8wDpdaN6v6ru.
- [28]A. Syed, C. Rager, and A. Conmy. Attribution patching outperforms automated circuit discovery. In BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP (BlackboxNLP), pages 407–416, 2024.arxiv.org/abs/2310.10348
- [29]A. Geiger, Z. Wu, H. Lu, J. Rozner, E. Kreiss, T. Icard, N. Goodman, and C. Potts. Inducing causal structure for interpretable neural networks. In International Conference on Machine Learning (ICML), pages 7324–7338, 2022.arxiv.org/abs/2112.00826