WorksComputer Science
Do VLAs Know They Will Fail? Interpretable Success and Failure Patterns in Physical AI Systems
Abstract
Vision-Language-Action (VLA) models have emerged as powerful general-purpose robot policies, but their successes and failures remain difficult to diagnose and understand. In this work, we show that sparse autoencoder (SAE) representations of OpenVLA activations contain outcome-predictive information and can be used to predict task failure before the robot reaches an unrecoverable state. Moreover, to provide insights into what common behaviors may contribute to these outcomes, we identify individual SAE features whose strong activations are associated with specific stages of success or failure, and uncover recurring behavioral patterns by localizing temporal intervals where these features exhibit significant activation changes. We further develop a VLM-based interpretation pipeline to generate behavioral descriptions of these patterns and validate their correspondence to feature activations on held-out intervals.
Introduction
Vision-language-action (VLA) models have achieved strong generalist performance by scaling model capacity and training data [1, 2], but their internal representations make it difficult to diagnose why a rollout succeeds or fails. Recent work shows that these representations anticipate failure. SAFE learns a multitask failure detector from internal VLA features and uses conformal prediction to trade detection accuracy against warning time [3]. Mahato and Ren show that near-term failure is linearly decodable from OpenVLA activations under visual distribution shift [4]. FPC-VLA instead predicts failures with an external VLM supervisor [5]. These works report warning time relative to the end of the episode. The problem is that every action an agent takes changes how easy the remaining task is to complete; thus, these approaches may trigger an alarm after the task has become unrecoverable. We therefore build on this line of research by measuring lead time against a point of no return, a timestep after which the only reachable states by branched policies are failures.
More importantly, we aim to move beyond predicting outcomes toward identifying the behavioral patterns strongly associated with success or failure. Sparse autoencoders (SAEs) provide a promising approach by decomposing dense, polysemantic activations into sparse, potentially interpretable features [6, 7]. Recent work has applied this technique to VLAs. Hao et al. [8] identify interpretable, steerable directions in OpenVLA. Swann et al. [9] find SAE features for motion and semantics, some of which causally steer behavior. Event-Grounded SAEs tie features to behaviorally salient events in closed-loop rollouts [10]. These works characterize features by what they represent or control. We ask a complementary question: which sparse features track whether a rollout will succeed, and what behavior do they correspond to?
We show that SAE features preserve the outcome-predictive information in OpenVLA activations, and that, evaluated against the point of no return, the predictor separates eventual failures from successes while the failures are still recoverable. We then select individual features by their association with trajectory outcomes and automatically localize the critical intervals where they emerge. Finally, we describe these intervals with a VLM and validate the descriptions on held-out clips via detection scoring [11, 12]. Together, these steps connect early failure warning to interpretable, failure-relevant behavior in VLA policies. An extended discussion of related work is in Appendix § 5.
Experimental Results
Problem Setup
We run OpenVLA-7B [2] fine-tuned on LIBERO-10 [13] greedily from the 50 benchmark initial states of each of the ten tasks, giving 500 rollouts that end either in success or in failure at a 520-step timeout (51.6% success; 95 trajectories held out for evaluation). Let denote the layer-0 residual-stream activation at timestep of trajectory , averaged over image tokens. While dense activations may contain outcome-predictive information, their polysemanticity makes it difficult to identify the specific behavioral patterns associated with success or failure [6, 7]. We therefore use a TopK sparse autoencoder (2,048 features, ) to decompose them into sparse feature activations . We ask whether and predict the eventual outcome, and whether individual SAE features reveal interpretable success- and failure-related behavior (task, splits, and SAE details are in Appendix § 6).
Online Outcome Prediction
We first test whether the raw activations and SAE features contain information predictive of the eventual trajectory outcome. Specifically, at each timestep , we train a logistic regressor using either the raw activations or SAE features to predict the final outcome of an ongoing trajectory, taking its predicted failure probability as . To reduce transient activation noise, we consider activations within a rolling window of steps rather than single-step activations. The predictor input is the rolling mean raw activation or the rolling mean SAE representation . As successful trajectories tend to terminate earlier, the activations at large predominantly come from failed trajectories, causing severe label imbalance and making high accuracy easy to achieve. We therefore compare our predictor with a majority baseline that takes no input and always predicts the majority outcome label at each timestep .
Figure 1(a) shows that raw activations and SAE features achieve similar accuracy and outperform the majority baseline at early-to-mid timesteps. Their AUROCs are also nearly identical (0.80 and 0.81, respectively, over all held-out frames), remaining above chance from the first 80 steps (0.68) through steps 240–319 (0.75). These results indicate that SAE features preserve most of the outcome-predictive information in the original OpenVLA activations.

Figure 1. For late timesteps, almost all trajectories are failures, so low performance is most relevant. (a) Raw activations and SAE features outperform the majority baseline at early timesteps. (b) Both representations achieve above-chance AUROC, indicating that they retain outcome-predictive information. (c) For held-out failures aligned at their point of no return (PNR, red line), predicted failure probability rises as estimated recoverability declines.
Recoverability-grounded early warning. Predicting failure before an episode ends does not necessarily provide an actionable warning, as the robot may have already reached an unrecoverable state. To evaluate whether our predictor identifies failures while recovery is still possible, we estimate state recoverability by restoring saved simulator states and sampling policy continuations. We define the point of no return (PNR) as the earliest probe after which all subsequent probes have estimated recoverability . An alarm is triggered when for two consecutive evaluations and is actionable if it occurs before the PNR.
Early warning performance. Figure 1(c) illustrate a trade-off between prediction accuracy and actionability: although accuracy generally increases over time, recoverability declines, leaving fewer opportunities for intervention. Before the estimated PNR, the SAE predictor distinguishes eventual failures from successes with an AUROC of 0.71 (95% CI [0.62, 0.79]), compared with 0.65 [0.57, 0.74] for raw activations. Table 1 reports the early warning performance under different alarm thresholds. As increases from 0.5 to 0.9, the false-alarm rate decreases from 64% to 14%, but actionable detection also drops from 88% to 33%, while the median lead time decreases from 190 to 70 steps. These results highlight the trade-off between reducing false alarms and providing sufficiently early warnings.
| False alarms | Actionable | Lead | |
| 0.5 | 0.64 | 0.88 | 190 |
| 0.7 | 0.25 | 0.55 | 160 |
| 0.9 | 0.14 | 0.33 | 70 |
Table 1. Alarms relative to the point of no return (held-out). False alarms: fraction of successful trajectories that raise an alarm. Actionable: fraction of failures alarmed before their PNR. Lead: median steps from alarm to PNR for actionable alarms.
Discovering Outcome-Associated SAE Features
Next, we seek SAE features whose activations may associate with successful or failed outcomes, potentially revealing specific success- or failure-related behavioral patterns. For feature , we define the rolling mean activation as , and the trajectory score as the maximum rolling mean activation, . We hypothesize that some features exhibit strong indications of success or failure when highly activated. We therefore select the top- trajectories by trajectory score for each feature and compute their success rate among these top- trajectories. We use in our experiments. Table 2 shows a selection of features with the lowest and highest success rates (see full results in Table 4), which supports our hypothesis that strong activation is predictive of the trajectory outcome: trajectories with high scores for feature 1578 are almost always successful (28 of the top 30), whereas those with high scores for feature 480 are almost always failures (29 of the top 30, spanning four tasks). This suggests that individual SAE features may correspond to specific success- or failure-related behavioral patterns.
| Feature | Success rate | Automatically inferred pattern | AUROC | |
| 480 | 3.33% | In Task 9, arm moves right from the stove toward the moka pot handle without holding the pot | 0.72 | 0.26 |
| 467 | 3.33% | In Task 1, gripper hovers close to the moka pot handle without moving the pot | 0.84 | 0.22 |
| 850 | 3.33% | In Task 1, open gripper approaching the not-yet-grasped middle moka pot | 0.42 | −0.53 |
| 1578 | 93.33% | In Task 4, carrying the grasped task object left toward the basket | 0.89 | 0.04 |
| 1671 | 86.67% | In Task 3, carrying the grasped green-and-red-label can left and placing it in the basket | 1.00 | 0.17 |
| All 12 feature–task pairs (261 clips; shuffled-description AUROC 0.58) | 0.76 | 0.17 |
Table 2. Representative outcome-associated SAE features and their automatically inferred behavioral patterns. Success rates are based on each feature’s top 30 trajectories. AUROC evaluates descriptions on held-out clips; is the difference from shuffled-description AUROC. Task descriptions are provided in Table 3.
| Relation | Feature | Success rate | # Unique Tasks | Active trajectories |
| Failure | 480 | 3.33% | 4 | 303 |
| Failure | 850 | 3.33% | 1 | 164 |
| Failure | 35 | 3.33% | 1 | 346 |
| Failure | 467 | 3.33% | 1 | 240 |
| Failure | 685 | 6.67% | 5 | 367 |
| Success | 1578 | 93.33% | 3 | 235 |
| Success | 748 | 90.00% | 1 | 176 |
| Success | 132 | 90.00% | 1 | 132 |
| Success | 1671 | 86.67% | 2 | 188 |
| Success | 1528 | 86.67% | 1 | 230 |
Table 4. SAE features with the strongest outcome associations. For each feature, the success rate is computed over the top- trajectories. We show features with the lowest and highest success rates, corresponding to failure- and success-associated features, respectively.
Localizing Outcome-Associated Behavioral Patterns
We first inspect the temporal activation patterns of outcome-associated SAE features to understand their behavioral significance. As shown in Figure 2, feature 480’s activation rises sharply and subsequently remains elevated. This pattern is consistent across its strongest-activating failed trajectories (see Figure 4 in Appendix). Manual inspection reveals that high activation coincides with the robot remaining nearly stationary near the target object. We observe similar associations for other features, with high activations corresponding to specific failure states in failed trajectories or particular stages of task execution in successful trajectories.
Figure 2. Activation traces over time and automatically detected critical intervals for the failed trajectories with the strongest activations of feature 480 in Task 1: "Put both moka pots on the stove". It shows five uniformly sampled visualization frames above the full activation trace.
Figure 4. Activation traces over time and automatically detected critical intervals for three failed trajectories with the strongest activations of feature 480 in Task 1. It shows five uniformly sampled visualization frames above the full activation trace.
Although high activation may indicate a failure state even before the episode terminates, we are more interested in the events leading into these states, as they may offer insights into potential causal mechanisms underlying success or failure and inform future policy improvements. We therefore detect the strongest sustained rise in each feature trace, rather than simply selecting regions of high activation, and retain one naturally bounded critical interval per trajectory (See details in Appendix § 8). Figure 4 visualizes the automatically selected intervals for the three failure trajectories that have the strongest feature 480 activations in Task 1 (put both moka pots on the stove). Despite differences in timing, all three intervals show a similar pattern: the robot reaches the middle moka pot and remains in contact with it, but fails to transport it onto the stove. These observations suggest that the detected activation rises can consistently localize behavioral events that may contribute to failure-related states.
Automatic Interpretation and Validation
Manual inspection is useful for validation but does not scale. We therefore develop an automatic interpretation pipeline that uses a VLM to describe behavioral patterns in critical intervals (See details in Appendix § 7.§ 1). Table 2 presents two representative automatic interpretations: failure-related feature 480 in Task 9 (turn on the stove and put the moka pot on it) and success-related feature 1671 in Task 3 (put both the alphabet soup and the tomato sauce in the basket). Feature 480 in Task 9 most clearly captures a stalled approach to a moka pot that never moves with the gripper, whereas feature 1671 consistently captures transporting a grasped can into the basket. The explanations may point to events that contribute to the subsequent failure or success and thus provide candidate causal explanations for the final outcome. More interpretation results and analysis are provided in Table 5 in Appendix § 7.§ 1. We validate the generated descriptions using detection scoring [12] on held-out clips. Across 12 feature-task pairs, descriptions achieve a mean AUROC of 0.76, compared to 0.58 for shuffled descriptions (Appendix § 7.§ 2).
| Relation | Feature | Task | Automatically inferred pattern | Outcome relation | AUROC | |
| Failure | 35 | 1 | Empty arm swings right into the workspace before touching any object | ordinary task progress | 0.53 | 0.46 |
| Failure | 467 | 1 | Gripper hovers close to the moka pot handle without moving the pot | unclear | 0.84 | 0.22 |
| Failure | 480 | 1 | Stalled, unlifted, or pushing grasp attempt after closing in on the moka pot | early indicator | 0.56 | 0.06 |
| Failure | 480 | 8 | End effector holding or touching the yellow-and-white mug and moving it toward the open microwave | ordinary task progress | 0.65 | 0.15 |
| Failure | 480 | 9 | Arm moves right from the stove toward the moka pot handle / frying pan area without holding the pot | early indicator | 0.72 | 0.26 |
| Failure | 850 | 1 | Open gripper approaching the not-yet-grasped middle moka pot | ordinary task progress | 0.42 | |
| Success | 132 | 0 | Placing the grasped book into the back compartment of the caddy and releasing it | ordinary task progress | 0.97 | 0.05 |
| Success | 748 | 0 | Lifting the grasped book and carrying it toward the back compartment | ordinary task progress | 0.88 | 0.65 |
| Success | 1578 | 3 | Carrying the grasped blue-and-yellow can left to the basket and placing it inside | ordinary task progress | 0.78 | 0.06 |
| Success | 1578 | 4 | Carrying the grasped task object left toward the basket | ordinary task progress | 0.89 | 0.04 |
| Success | 1671 | 2 | Delivering the grasped blue-and-yellow can into the basket (can at or inside the basket opening) | ordinary task progress | 0.86 | 0.50 |
| Success | 1671 | 3 | Carrying the grasped green-and-red-label can left and placing it in the basket | ordinary task progress | 1.00 | 0.17 |
Table 5. Automatic interpretations of the four highest-ranked failure-related and success-related SAE features and their reproduction-test scores on held-out clips: AUROC of a judge VLM that sees only the description, and the difference to a shuffled description of another feature on the same clips. Each explanation uses the three strongest task-specific critical clips and one independently sampled successful demonstration. The outcome-relation category is produced by the external VLM. Task numbers follow Table 3.
Conclusion
This work presents a novel framework that goes beyond early failure prediction to automatically discover and interpret outcome-associated behavioral patterns in VLA models. We show that SAE features preserve outcome-predictive information in OpenVLA activations, enabling failure prediction before the robot reaches an estimated unrecoverable state. We further identify individual SAE features strongly associated with successful or failed outcomes and uncover recurring behavioral patterns by localizing critical activation intervals. Our automatic VLM-based interpretation pipeline generates behavioral descriptions of these patterns, which we validate against feature activations on held-out clips. Together, these results connect early failure prediction with interpretable analysis of outcome-associated behaviors, providing a foundation for investigating potential failure mechanisms, developing targeted safety interventions, and exploring interpretable features as reward signals for future policy training.
Acknowledgments
We want to thank Dr. Kenneth Marino and the students of the University of Utah’s Spring 2026 CS 6960 course on multimodal llm agents for their feedback at the start of the project.
Related Work
Failure Prediction for Vision-Language-Action Models As vision-language-action (VLA) models such as RT-2 [1], OpenVLA [2], and [14] increasingly serve as general-purpose robot policies, recent work has explored whether their internal representations can provide early signals of task failure. SAFE [3] learns a failure detector directly from internal VLA features and demonstrates multitask failure detection, using conformal prediction to trade off detection accuracy and warning time. More recently, Mahato and Ren [4] showed that OpenVLA feedforward activations contain linearly decodable signals of near-term failure under visual distribution shifts, demonstrating that internal activations can provide early warning signals. FPC-VLA [5] instead uses an external VLM-based supervisor at manipulation keyframes to predict failures and generate corrective actions. These works establish that failures can be anticipated from either internal VLA representations or external observations. Our work complements this direction by asking whether the internal signals predictive of trajectory outcomes can be decomposed into sparse, individually interpretable features that reveal the behavioral events associated with success and failure. Unlike these works, which report warning time relative to the end of an episode, we also measure it relative to a simulator-estimated point after which the failure can no longer be avoided, because a warning is not early if it arrives after the failure has become unavoidable.
Sparse and Mechanistic Interpretability of VLA Policies Sparse autoencoders (SAEs) have emerged as a tool for decomposing dense neural representations into sparse and potentially interpretable features [6, 7]. Recent work has extended these techniques to VLA policies. Swann et al. [9] train SAEs on VLA hidden activations and identify both episode-specific features and generalizable motion or semantic features, demonstrating through intervention that some individual SAE features can causally steer robot behavior. Event-Grounded SAEs [10] further ground SAE features in behaviorally salient events derived from robot trajectories, using visual, state, and temporal cues to associate internal features with closed-loop behavior and evaluate them through interventions. These results demonstrate that SAE features can expose behaviorally meaningful structure within VLA representations. Automatic feature descriptions are commonly validated by detection scoring, in which a model that sees only the description predicts held-out activations [11, 12]; Gur-Arieh et al. [15] distinguish such input-centric descriptions from output-centric descriptions of a feature’s effect on the model’s output. In contrast, we specifically connect sparse features to trajectory outcomes: we first identify SAE features whose activation is strongly associated with success or failure, then localize critical temporal intervals in which these features emerge and use a VLM to automatically interpret the corresponding behavioral events.
Experimental Setup Details
Dataset and Rollout Collection
We study OpenVLA [2] rollouts on multiple robot manipulation tasks. Each rollout terminates either when the task is successfully completed or when it reaches the maximum number of environment steps, in which case it is considered a failure. We use OpenVLA-7B fine-tuned on LIBERO-10 [13] and run it greedily from the 50 fixed benchmark initial states of each of the ten tasks (500 rollouts, at most 520 steps after 10 settling steps), recording each camera frame and the full simulator state. Overall, 51.6% of rollouts succeed, ranging from 12% to 80% across tasks (Table 3; tasks are numbered alphabetically by instruction). A hash of the trajectory identifier selects 95 held-out trajectories (51 failures, 44 successes), and the remaining trajectories are split into 324 fit and 81 calibration trajectories; all results are reported on the held-out set unless stated otherwise. Let denote the layer-0 residual-stream activation of OpenVLA at timestep of trajectory , averaged over the image tokens.
| Task | Instruction | Success |
| 0 | pick up the book and place it in the back compartment of the caddy | 80% |
| 1 | put both moka pots on the stove | 12% |
| 2 | put both the alphabet soup and the cream cheese box in the basket | 56% |
| 3 | put both the alphabet soup and the tomato sauce in the basket | 54% |
| 4 | put both the cream cheese box and the butter in the basket | 76% |
| 5 | put the black bowl in the bottom drawer of the cabinet and close it | 46% |
| 6 | put the white mug on the left plate and put the yellow and white mug on the right plate | 50% |
| 7 | put the white mug on the plate and put the chocolate pudding to the right of the plate | 44% |
| 8 | put the yellow and white mug in the microwave and close it | 36% |
| 9 | turn on the stove and put the moka pot on it | 62% |
Table 3. LIBERO-10 tasks (numbered alphabetically by instruction) and the success rate of OpenVLA over the 50 benchmark initial states.
Sparse Autoencoder
We aim to understand the behavioral patterns encoded in these activations, but dense representations can be difficult to interpret because individual dimensions may represent multiple features due to superposition and polysemanticity [6, 7]. We therefore use a top- sparse autoencoder (SAE) to decompose into sparse feature activations . Specifically, the SAE encoder maps to feature activations . We retain only the top- activations to obtain the sparse representation , and reconstruct the original activation as . This associates each feature with a specific activation direction that may correspond to a particular behavioral pattern, providing a more interpretable unit of analysis. We use 2,048 features and , and train the SAE on mean-centered activations from all 500 rollouts. Our goal is to determine whether and contain outcome-associated information, and whether individual SAE features reveal interpretable success- and failure-related behavioral patterns.
Additional Experimental Results
Full Automatic Interpretation Results
Table 5 summarizes the automatic explanations for the four highest-ranked failure-related features (480, 850, 35, and 467) and the four highest-ranked success-related features (1578, 748, 132, and 1671) across every task with at least three eligible primary examples. The results reveal both useful task-specific semantics and important limitations. Feature 480 in Tasks 1 and 9 most clearly captures a stalled approach to a moka pot that never moves with the gripper, whereas feature 1671 consistently captures transporting a grasped can into the basket. The explanations may point to an event that contributes to the subsequent failure state and thus provides a candidate causal explanation for failure. In contrast, feature 132 represents the final book-placement stage, while feature 480 in Task 8 is broader, describing the mug held near the open microwave rather than a stalled grasp, which shows that a feature’s meaning can depend on the task. These features are better interpreted as markers of successful or failed outcomes rather than their causes.
Figure 3. (a) Randomly selected failed trajectory. (b) High-activation successful trajectory. Temporal activation of success-related feature 1671 on Task 3.
Explanation Validations
Table 5 lists the per-pair scores. Across the 12 pairs (261 clips), the mean AUROC is 0.76 (95% CI [0.64, 0.87], bootstrap over pairs and clips; permutation ) and the mean Spearman correlation with activation is 0.46. Descriptions outperform shuffled descriptions on 11 of 12 pairs (mean AUROC = 0.17, 95% CI over clips [0.10, 0.24]; sign test ), although resampling pairs as well widens this interval to [−0.01, 0.35], so specificity varies across features. Individual descriptions are uneven: the description of feature 850 is anti-correlated with its activation (AUROC 0.42), and several transport descriptions are accurate but not specific, because other features on the same task describe similar transport (e.g. feature 132: 0.97 versus 0.92 shuffled). With 24 clips per pair, an individual AUROC has a confidence interval of roughly ±0.2; we therefore treat the description methodology as validated in aggregate and individual descriptions as hypotheses.
Selection of Critical Activation Intervals
Given the activation sequence of an SAE feature along a trajectory, we identify one critical activation interval that captures the most salient transition into a strongly activated state. This is intended to exclude isolated activation spikes and to retain sufficient visual context before and after the transition.
We first normalize the activation sequence by its 90th percentile over non-zero activations and apply Gaussian smoothing. For each time step , we then compute a local activation contrast:
where is the smoothed normalized activation and is a temporal horizon. In our experiments, environment steps. A high value of therefore indicates that the feature becomes consistently more active after , rather than exhibiting only a single-frame spike.
Candidate rising events are obtained from prominent local maxima of . The boundaries of each event are determined by extending from its peak while the local contrast remains above a relative support threshold. We additionally include 15 steps of temporal context on both sides of the detected transition, so that the resulting clip contains part of the preceding low-activation state, the activation rise, and the subsequent high-activation state.
Nearby candidate events separated by fewer than 25 steps are merged. This makes the detector robust to short activation drops or intermittent zero activations within the same underlying event.
If multiple event groups remain, we retain the group with the largest aggregate prominence. Thus, exactly one critical interval is selected for each trajectory. All detector parameters are fixed across features, tasks, and trajectory outcomes. Outcome labels are not used during interval selection.
Recoverability Estimation
This appendix details the recoverability estimate and point of no return (PNR) used to evaluate early warning in § 2.2.
Recoverability Estimator
OpenVLA conditions only on the current frame and instruction, so the recorded simulator state determines the policy’s future. We estimate the recoverability under the policy with temperature-1 sampling of its action tokens by restoring and running stochastic continuations for the remaining steps:
Sampling is necessary because the recorded rollouts are collected with greedy decoding (Appendix § 6), which would reproduce the recorded outcome.
Probe Placement and Point of No Return
Probes are placed at steps around the predictor’s alarm and around abrupt rises in its failure log-odds in all 51 held-out failures (838 probes, 3,509 branch rollouts). For a failed trajectory with probes , the PNR is the earliest probe after which every probe has ; it is left-censored if this already holds at the first probe. The predictor raises an alarm once for two consecutive evaluations, and an alarm is actionable if it precedes the PNR.
For the before-PNR evaluation in § 2.2, each failure is restricted to frames up to its last still-recoverable probe and compared with held-out successful trajectories at matched times. This uses the 25 failures with an uncensored PNR before step 420; later PNRs are excluded because late probes can fail simply because the remaining step budget runs out (Appendix § 9.4).
Short-Term Spikes in Predicted Failure Probability
Short-term spikes in are less informative than its gradual rise: most spikes (83%) occur after has already fallen to at most 0.1, and those occurring earlier show no larger drop in than matched non-spike intervals (difference , 95% CI ). With four branches per probe, smaller effects cannot be excluded. We also note that our methodology for detecting spikes, qualitatively from human analysis, seems to exclude important meso-trends over the course of tens of timesteps that predict human-derived PNR values. We leave extending this pipeline to account for this confound to future analysis.
Limitations of the Recoverability Estimate
is an imperfect instrument. Branches inherit the remaining step budget, so late probes can fail because time runs out (branch success is 0.2% with fewer than 100 steps left versus 18–23% with 200, regardless of actual simulator state), which is why we exclude PNRs after step 420. Temperature sampling perturbs the policy in a phase-dependent way: in one rollout the greedy policy was stuck against the microwave roof from about step 207, but sampled branches occasionally shook free and delayed the measured PNR, while during first motions or object transport sampled branches drop objects more often than the greedy policy, which lowers . With four branches per probe a single estimate is coarse (for the 95% interval is roughly ), and measures recoverability for this policy rather than physical recoverability. We therefore read these results as estimates that support qualitative conclusions, not precise failure times. We hope to extend this methodology in the future to involve more branches and further development of the recoverability estimation system.
References
- [1]B. Zitkovich, T. Yu, S. Xu, P. Xu, T. Xiao, F. Xia, J. Wu, P. Wohlhart, S. Welker, A. Wahid, Q. Vuong, V. Vanhoucke, H. Tran, R. Sorićut, A. Singh, J. Singh, P. Sermanet, P. R. Sanketi, G. Salazar, M. S. Ryoo, et al. Rt-2: Vision-language-action models transfer web knowledge to robotic control. arXiv preprint arXiv:2307.15818, 2023. URL https://arxiv.org/abs/2307.15818.
- [2]M. J. Kim, K. Pertsch, S. Karamcheti, T. Xiao, A. Balakrishna, S. Nair, R. Rafailov, E. P. Foster, P. R. Sanketi, Q. Vuong, T. Kollar, B. Burchfiel, R. Tedrake, D. Sadigh, S. Levine, P. Liang, and C. Finn. Openvla: An open-source vision-language-action model. In Conference on Robot Learning (CoRL), 2024. URL https://openreview.net/forum?id=ZMnD6QZAE6. arXiv:2406.09246.arxiv.org/abs/2406.09246
- [3]Q. Gu, Y. Ju, S. Sun, I. Gilitschenski, H. Nishimura, M. Itkina, and F. Shkurti. Safe: Multi-task failure detection for vision-language-action models. In Advances in Neural Information Processing Systems, 2025.DOI
- [4]D. T. Mahato and R. Ren. Early warning signals for openvla failure under visual distribution shift. arXiv preprint arXiv:2606.29699, 2026.arxiv.org/abs/2606.29699
- [5]Y. Yang, Z. Duan, T. Xie, F. Cao, P. Shen, P. Song, C. Zhao, P. Jin, G. Sun, S. Xu, Y. You, and J. Liu. Fpc-vla: A vision-language-action framework with a supervisor for failure prediction and correction. Expert Systems with Applications, 316:131742, 2026. doi:10.1016/j.eswa.2026.131742.DOI
- [6]T. Bricken et al. Towards monosemanticity: Decomposing language models with dictionary learning. Transformer Circuits Thread, 2023. URL https://transformer-circuits.pub/2023/monosemantic-features/index.html. Anthropic.
- [7]H. Cunningham, A. Ewart, L. Riggs, R. Huben, and L. Sharkey. Sparse autoencoders find highly interpretable features in language models. arXiv preprint arXiv:2309.08600, 2023. URL https://arxiv.org/abs/2309.08600.
- [8]B. Håon, K. Stocking, I. Chuang, and C. Tomlin. Mechanistic interpretability for steering vision-language-action models. arXiv preprint arXiv:2509.00328, 2025. URL https://arxiv.org/abs/2509.00328.
- [9]A. Swann, L. McGranahan, H. Buurmeijer, M. Kennedy III, and M. Schwager. Sparse autoencoders reveal interpretable and steerable features in vla models. In Conference on Robot Learning (CoRL), 2026.
- [10]X. Jin, A. Chatterjee, P. Kumar, and R. Paleja. Event-grounded sparse autoencoders for vision-language-action policies. In Advances in Neural Information Processing Systems, 2026.
- [11]S. Bills, N. Cammarata, D. Mossing, H. Tillman, L. Gao, G. Goh, I. Sutskever, J. Leike, J. Wu, and W. Saunders. Language models can explain neurons in language models, 2023. URL https://openaipublic.blob.core.windows.net/neuron-explainer/paper/index.html.
- [12]G. S. Paulo, A. T. Mallen, C. Juang, and N. Belrose. Automatically interpreting millions of features in large language models. In Proceedings of the 42nd International Conference on Machine Learning, volume 267 of Proceedings of Machine Learning Research, pages 48393–48421. PMLR, 2025. URL https://proceedings.mlr.press/v267/paulo25a.html.arxiv.org/abs/2410.13928


