Should the Action Expert Train the VLM? Action-Token Supervision and Gradient Routing in Mixture-of-Transformer VLAs

Introduction

Pretrained vision–language models (VLMs) bring broad perceptual, semantic, and reasoning capabilities to embodied control. Robotic control, however, requires precise continuous outputs at frequencies poorly suited to autoregressive decoding with large VLMs. Many recent vision–language–action (VLA) models therefore use Mixture-of-Transformers-style architectures [23], pairing an autoregressive VLM with a smaller diffusion or flow-matching action expert [2]. The VLM processes visual observations and language instructions, while the expert efficiently generates continuous action chunks conditioned on the VLM’s per-layer key–value (K/V) representations [34].

This architecture leaves open a consequential training decision: should the continuous-action loss train only the expert, or also the VLM that produces its conditioning K/V cache? Allowing expert gradients to update the VLM (coupling) enables co-adaptation and can teach the backbone which information its representations must preserve for precise control. Yet it also exposes pretrained visual and linguistic features to gradients from a newly introduced, often randomly initialized expert, which may degrade the VLM’s semantic and world understanding.

Popular approaches therefore block the action expert’s gradients from entering the VLM (insulation), either through stop-gradient or frozen-backbone training [6, 34]. However, full-parameter fine-tuning on a narrow robot training distribution can degrade pretrained capabilities regardless of gradient routing [11]. When vision–language co-training is also used to mitigate forgetting, it is unclear whether insulation provides additional protection.

Insulation is also rarely evaluated in isolation. Blocking expert gradients removes one source of action-relevant learning signal, but such systems typically supply another: they train the VLM to predict discrete action tokens [33], which directly supervise the behavior the expert models. Co-training on general vision–question-answering (VQA) and chain-of-thought examples may preserve the VLM’s semantic capabilities, but it does not explicitly encourage representations relevant to control. In the absence of action-token supervision, insulation may therefore eliminate the only objective that directly adapts the VLM for trajectory prediction. Crucially, defining an analogous discrete proxy for continuous prediction targets is a domain-dependent design choice whose effectiveness must be established empirically. It is therefore unclear whether the reported benefits of insulation arise from gradient blocking itself or depend on this auxiliary supervision.

Prior work has not isolated this interaction. Knowledge Insulation reports benefits from insulated training while supervising the VLM with FAST action tokens and vision–language objectives, but does not evaluate insulation without FAST supervision [6, 33]. In contrast, Wall-OSS-0.5 reports stronger action performance from coupled training even when discrete-action and multimodal objectives are retained [39]. Because these systems also differ in training recipes, initialization, data, and optimization, their seemingly opposing results cannot isolate the contributions of action-token supervision and expert-to-VLM gradient flow in downstream control or determine how the two mechanisms interact.

Contributions. In this work, we study how a VLA’s representations are made action-relevant, and whether the two available signals—the expert’s gradient and discrete action-token supervision—substitute for one another. We cross action-token supervision with expert-to-VLM gradient flow in autonomous driving and robot manipulation, under both joint training and training schedules in which one component is adapted first. Within each training schedule and action-token setting, routing comparisons hold data, architecture, optimization, and semantic co-training fixed. In open-loop driving evaluations, the two routing choices are nearly indistinguishable with action-token supervision; without it, coupling reduces trajectory-prediction error by roughly 25%. This pattern holds across training schedules and in- and out-of-distribution evaluations. Under joint training, coupling improves closed-loop performance regardless of supervision: driving route progress increases by approximately 17% with action tokens and 40% without them, while manipulation task progress improves under both settings. Together, these results identify action-token supervision as a factor previously entangled with gradient routing and show that VLM insulation is a conditional choice, not a default training recipe for training VLAs.

Related Work

Vision–language–action architectures. Early VLA policies incorporated control into pretrained vision–language models (VLMs) by predicting discretized actions autoregressively [3, 16]. Many recent systems instead pair a VLM with a specialized continuous-action module. $\pi_{0}$ introduced a flow-matching expert conditioned on the VLM’s per-layer key–value (K/V) cache, an architecture retained or adapted by $\pi_{0.5}$, Wall-OSS-0.5, and MolmoAct2 [2, 7, 34, 39]. SmolVLA, GR00T N1, Qwen-RobotManip, CogACT, and DexVLA use related flow- or diffusion-based action decoders with different interfaces to the VLM [21, 28, 35, 36, 40]. These systems instantiate a broader Mixture-of-Transformers pattern in which modality-specific computation communicates through attention [23].

Many autonomous-driving VLAs follow similar designs. EMMA, OpenDriveVLA, and AutoVLA generate future trajectories autoregressively as textual coordinates or discrete tokens, whereas Alpamayo-R1 combines autoregressive physical reasoning and trajectory prediction with a continuous action expert [12, 30, 44, 45]. Related architectures combine autoregressive reasoning with continuous generation for images, video, and other physical-AI modalities [31, 42]. Despite their growing adoption, how continuous experts shape the autoregressive representations through the K/V cache they condition on remains poorly understood. We study this question in a controlled setting.

Action representation and backbone supervision. Action representation determines both how a policy is executed and how control supervision reaches the VLM. Discrete action tokens train the backbone through next-token prediction, while diffusion and flow matching directly model continuous, potentially multimodal action distributions [2, 5]. FAST compresses action chunks into discrete tokens using a discrete cosine transform, OAT learns compact action tokens whose prefixes decode into progressively refined action chunks, OpenVLA-OFT predicts normalized continuous action chunks in parallel through an MLP head, and HybridVLA combines autoregressive and diffusion-based action prediction [17, 25, 26, 33].

Backbone supervision also affects whether pretrained capabilities survive VLA adaptation. End-to-end fine-tuning can distort useful features, motivating parameter-efficient adaptation, language-aligned action representations, and continued vision–language co-training [9, 11, 19, 41]. These approaches address how the VLM is trained, but leave unresolved a key interaction: whether autoregressive supervision over a discrete representation of the continuous action target affects the benefit of allowing gradients from the continuous expert to flow into the VLM backbone.

Gradient routing and training recipes. $\pi_{0.5}$, MolmoAct2, and Alpamayo-R1 adopt Knowledge Insulation: they train the continuous action expert without propagating its gradients into the VLM backbone, while updating the VLM through autoregressive action-token and semantic objectives [6, 7, 30, 34]. Within this framework, $\pi_{0.5}$ and MolmoAct2 use a two-stage schedule. The VLM is first adapted through autoregressive prediction of tokenized actions and semantic targets and the continuous expert is introduced afterward. Variants of this staged, insulated training recipe are common in recent VLAs; Table 1 summarizes representative works that insulate or freeze the VLM during action-expert training. By contrast, Wall-OSS-0.5 favors coupled optimization, allowing gradients from the continuous expert to update the VLM while retaining discrete-action and multimodal objectives [39]. Although these works motivate different gradient-routing choices at a shared VLM–expert interface, their results are not directly comparable because they also differ in training recipes, initialization, and data.

Policy	VQA Predicted	Discrete Actions Predicted
Autonomous Driving
Alpamayo-R1 [30]	$\checkmark$	$\checkmark$
Qwen-Drive-1.0 [43]	$\checkmark$	–
SpanVLA [46]	–	$\checkmark$
UniDriveVLA [22]	$\checkmark$	–
Manipulation
$\pi_{0.5}$ [34]	$\checkmark$	$\checkmark$
MolmoAct2 [7]	$\checkmark$	$\checkmark$

The growing use of this hybrid autoregressive–continuous MoT framework has not resolved how models should be trained. $\pi_{0.7}$ scales a FAST-supervised version of the recipe; MolmoAct2 changes its gradient routing between broad post-training and embodiment-specific fine-tuning; and APT finds that action-expert preparation changes the relative performance of insulated and coupled training [7, 13, 37]. Together with the contrasting Wall-OSS-0.5 results, these findings suggest that the preferred gradient routing depends on the surrounding training recipe. Related multimodal-pretraining studies similarly show that modalities can reinforce or interfere with one another depending on parameter sharing and training order [10]. Prior work therefore establishes gradient insulation as a consequential design choice, but does not isolate its interaction with autoregressive action supervision. We vary these decisions independently while holding semantic supervision and the VLM–expert architecture fixed.

Methodology

In this work, we test whether supervision on discrete future-action tokens changes the effect of propagating gradients from a continuous action expert into the VLM. We first study this question in autonomous driving, using the Alpamayo VLA architecture [30] and the hybrid autoregressive and flow-matching recipe used by $\pi_{0.5}$ and Knowledge Insulation [6, 34]. We vary future-action-token supervision and expert-to-VLM gradient flow independently, while holding the architecture and semantic supervision fixed.

Mixture-of-Transformers VLA. The driving model pairs a Qwen3-VL-2B-Instruct [38] backbone with a same-width, non-causal flow-matching action expert. At each transformer layer, the expert attends to key–value representations produced by the corresponding VLM layer, while the VLM does not attend to the expert’s continuous action inputs. The expert predicts $K$ future control pairs, each comprising acceleration and curvature. Starting from the current ego state, these controls are integrated using a discretized unicycle model to recover position and heading waypoints for low-level control (see A.1 for further details). We retain this architecture for every experiment.

Training includes trajectory examples with and without reasoning annotations. When present, the VLM predicts a concise, scene-grounded rationale identifying relevant road users or scene features and the intended driving response, followed by the future-trajectory start token. In action-token-supervised conditions, the VLM also predicts the subsequent future-trajectory (action) tokens. At evaluation, the VLM always generates a rationale and stops decoding at the start token. The action expert then predicts continuous controls conditioned on the resulting VLM K/V cache. Thus, future-action tokens are never decoded at evaluation and, when supervised, serve only as an auxiliary training signal shaping VLM representations.

The VLM is trained on the standard cross-entropy (CE) objective, while the expert is trained through rectified flow matching. The joint objective is $\mathcal{L} = \mathcal{L}_{\mathrm{CE}} + \mathcal{L}_{\mathrm{flow}}$, where

We sample $b\sim\operatorname{Beta}(1.5,1.0)$, set $t=0.999(1-b)$, and sample $\epsilon\sim\mathcal{N}(0,I)$. Parameters $\theta$ and $\phi$ belong to the VLM and action expert, respectively; $x$ is the current timestep’s visual observations, language context, and past trajectory history, $y$ is the autoregressive target sequence, and $a$ is the continuous trajectory. The term $\widetilde{H}_{\theta}$ denotes the VLM K/V representations supplied to the expert. For coupling, $\widetilde{H}_{\theta}=H_{\theta}$; for insulation, $\widetilde{H}_{\theta}=\operatorname{sg}(H_{\theta})$, where $\operatorname{sg}$ passes the representations unchanged to the expert but blocks gradients from the flow loss to the VLM [6]. The set $\mathcal{I}_{\mathrm{base}}$ contains always-supervised token, including VQA answers, driving rationales, and future-trajectory delimiters, while $\mathcal{I}_{\mathrm{action}}$ contains future-trajectory value tokens. The indicator $\mathbf{1}_{\mathrm{action}}$ enables or disables direct CE supervision on those value tokens. Both loss terms have unit weight.

Crucially, the discrete action tokens and continuous expert encode the same future trajectory. Disabling action-token supervision masks CE only on future-action values, which remain teacher-forced; trajectory delimiters remain supervised, and past-trajectory tokens remain in the input context. We fix $N$ at the pre-mask count of supervised token positions, preserving the weight of other CE terms, and exclude future-action token states from the expert’s K/V context.

Gradient routing. Gradient routing changes only the backward pass through the layerwise K/V interface. In the coupled condition, the flow loss propagates through this interface and updates both the expert and the VLM. In the insulated condition, these gradients are stopped before entering the VLM. Stop-gradient leaves the forward computation unchanged for fixed parameters, so both conditions use the same expert conditioning mechanism and flow-matching target. Figure 2 summarizes which target signals update the VLM in each condition. VQA, reasoning, and structural-token supervision remain active in every condition; past-trajectory tokens remain in the input context.

Training schedules. To test whether training order changes the interaction between action-token supervision and gradient routing, we evaluate all four conditions under three training schedules (see Figure 2). In concurrent training, the pretrained VLM and a randomly initialized expert are trained together from the outset. In expert-first training, motivated by APT’s expert-preparation strategy [37], the expert is trained for 100k steps against a frozen VLM before joint training begins. Neither action-token supervision nor gradient routing affects this first stage because the VLM is frozen. In VLM-first training, reflecting the staged adaptation used by $\pi_{0.5}$ and MolmoAct2 [7, 34], the VLM is trained alone for 100k steps before a newly initialized expert is introduced. Action-token supervision varies during this first stage, while gradient routing applies only after the expert is introduced. Experts share the same initialization across conditions.

All joint-training stages use the same mixture of trajectory and vision–language batches, with trajectory-bearing batches constituting approximately 48.5% of the data. The VLM receives an update on every joint-training step, whereas the expert receives one only on trajectory-bearing steps. During the VLM-first warm-up, tokens-off models use only vision–language data, whereas tokens-on models also receive trajectory data. During the expert-first warm-up, every step updates the expert because it uses trajectory data exclusively. Thus, matching total optimizer steps across histories does not equalize component-level update budgets; additional details appear in Appendix § 6.

We compare checkpoints after 300k total optimizer steps, when validation performance had stabilized. Each schedule has the same total step budget: 300k concurrent steps or a 100k single-component stage followed by 200k joint steps. Within each schedule, the four conditions share the starting pretrained checkpoint, expert initialization, and optimizer schedule. Batch sampling is matched between routing conditions within each action-token setting.

Our main experiments evaluate these interventions in autonomous driving. In § 4.3, we repeat the concurrent-training experiment in closed-loop robot manipulation to test whether the findings extend beyond driving. Architecture, action-space, optimization, data, and flow-matching details appear in Appendix § 6.

Experiments

In this section, we ask three questions: (Q1) Does action-token supervision change the most performative gradient routing mechanism? (Q2) Does pretraining the VLM or expert first change that answer? (Q3) Are findings consistent across application domains? We address these questions in the three results subsections below.

Open-loop driving. We evaluate on the publicly released Physical AI Autonomous Vehicles dataset, specifically its nominal (named “Public Golden”) and out-of-distribution (“OOD”) splits, which contain 1,434 and 273 scenes, respectively [32]. For each example, the policy samples six candidate trajectories. Our primary metric is average displacement error (ADE), the mean Euclidean distance between predicted and ground-truth positions over the future trajectory. We report ADE for the first sampled candidate. Minimum ADE (minADE) is the lowest ADE among the six candidates and serves as an oracle selection metric.

Closed-loop driving. We evaluate in AlpaSim, a closed-loop simulator with realistic sensor observations, vehicle dynamics, and traffic scenarios [29]. Every condition is evaluated on the same 827 scenes with five stochastic rollouts per scene. We average rollouts within each scene to reduce evaluation variance.

Driving rationales. We use a frozen Lingo-Judge [27] classifier to compare generated reasoning-traces with reference rationales on the Public Golden and OOD benchmarks. The first-sample pass rate is the fraction of examples whose first generated rationale receives a score above 0.5. We also report the fraction for which any of six candidates passes and the mean score across candidates.

Statistical analysis. For open-loop ADE and first-sample rationale pass rates, we compute 95% confidence intervals from 10,000 paired clip-cluster bootstrap resamples. Each resample draws whole clips with replacement and applies the same sampled clips to every condition. These intervals quantify sensitivity to the evaluated clips for fixed checkpoints.

Concurrent Training

We first address Q1 in the simplest setting, where the VLM backbone and a randomly initialized action expert are trained together from the outset. Recall that all four conditions share the same data mixture, initialization, and optimization schedule.

Open-loop driving. Figure 3 shows that the effect of gradient routing depends on action-token supervision. With action-token supervision, insulation slightly lowers ADE on the Public Golden split, while the OOD routing difference is comparatively small. Without action-token supervision, the insulated model has substantially higher ADE in both evaluations. Relative to insulation, coupling reduces ADE by approximately 26% on the Public Golden dataset and 24% OOD.

This roughly one-quarter reduction in mean trajectory error is consistent with action-token supervision and expert gradients providing partially substitutable signals to the VLM. Action-token cross-entropy loss directly supervises the future trajectory even when expert gradients are blocked; without it, the flow loss is the only direct future-action objective that updates the VLM.

Generated reasoning-trace quality. We use the first-sample Lingo-Judge pass rate as a diagnostic of the language generated during concurrent training (Table 2). With action-token supervision, insulated and coupled models obtain nearly identical pass rates on Public Golden (64.7% and 64.9%, respectively). Without action-token supervision, the coupled model has a higher observed pass rate on Public Golden (63.5% versus 56.6%), while the OOD routing difference is small and points in the opposite direction. Insulation shows no consistent advantage in generated driving-rationale quality under the evaluated vision–language co-training recipe.

Schedule	Routing	Action CE	Public Golden: Top-1 [95% CI]	Top-$k$	Avg-$k$	OOD: Top-1 [95% CI]	Top-$k$	Avg-$k$
Concurrent	Insulated	On	0.647 [0.621,0.673]	0.808	0.630	0.414 [0.355,0.474]	0.659	0.466
Concurrent	Coupled	On	0.649 [0.623,0.674]	0.823	0.629	0.425 [0.364,0.487]	0.689	0.471
Concurrent	Insulated	Off	0.566 [0.539,0.593]	0.767	0.573	0.363 [0.305,0.423]	0.652	0.435
Concurrent	Coupled	Off	0.635 [0.610,0.661]	0.811	0.622	0.348 [0.292,0.406]	0.685	0.450
VLM-First	Insulated	On	0.639 [0.614,0.665]	0.807	0.626	0.392 [0.335,0.452]	0.652	0.462
VLM-First	Coupled	On	0.635 [0.609,0.661]	0.814	0.625	0.403 [0.344,0.464]	0.641	0.458
VLM-First	Insulated	Off	0.630 [0.603,0.657]	0.800	0.616	0.447 [0.386,0.508]	0.711	0.465
VLM-First	Coupled	Off	0.613 [0.585,0.639]	0.798	0.611	0.396 [0.339,0.452]	0.667	0.451
Expert-First	Insulated	On	0.644 [0.618,0.670]	0.819	0.626	0.458 [0.399,0.518]	0.700	0.470
Expert-First	Coupled	On	0.649 [0.623,0.675]	0.811	0.628	0.388 [0.331,0.447]	0.659	0.453
Expert-First	Insulated	Off	0.620 [0.594,0.646]	0.804	0.610	0.407 [0.348,0.466]	0.659	0.450
Expert-First	Coupled	Off	0.612 [0.585,0.639]	0.798	0.611	0.392 [0.333,0.451]	0.692	0.459

Closed-loop driving. In concurrent training, coupling raises AlpaSim route progress from 0.600 to 0.703 with action-token supervision and from 0.425 to 0.601 without it (Table 3). The latter is a 17.6 percentage-point increase, or about 41% relative, on the same evaluation scenes. Coupling without tokens also reaches approximately the same route progress as insulated training with tokens. Thus, allowing the expert’s loss to update the VLM yields a substantial gain in interactive driving, with the largest observed gain when action-token supervision is absent.

History	Insulated: Tokens on	Insulated: Tokens off	Coupled: Tokens on	Coupled: Tokens off
Concurrent	0.600	0.425	0.703	0.601
VLM-First	0.648	0.555	<u>0.646</u>	0.587
Expert-First	0.597	0.538	0.625	<u>0.604</u>

Thus, Q1 has a conditional answer: without action-token supervision, coupling substantially improves open- and closed-loop driving; with supervision, it still improves closed-loop progress, while the open-loop difference is small. The most important factor is that the VLM receives a direct action learning signal, supplied by either action-token supervision or expert-gradient coupling.

Dependence on Training History

To address Q2, we next test whether the supervision–routing pattern depends on training history by first adapting either the action expert or the VLM for 100k steps. We align staged and concurrent checkpoints by total optimizer steps; component-level gradient exposure nevertheless differs across histories by design (see Table 6 in the Appendix).

History	Schedule	VLM updates	Expert updates
Concurrent	300k joint steps	300k	$\approx$ 145.5k
Expert-First	100k expert-only + 200k joint	200k	$\approx$ 197k
VLM-First	100k VLM-only + 200k joint	300k	$\approx$ 97k

Open-loop driving. The supervision–routing pattern observed under concurrent training persists when either module is adapted first (Figure 3). Across both evaluation datasets, insulation increases mean ADE without action-token supervision in every history, while its effect with supervision is smaller and depends on training order. Thus, training order changes the size of the routing effect but does not remove its dependence on action-token supervision.

Generated reasoning-trace quality. The staged Lingo-Judge results do not mirror the trajectory-ADE pattern (Table 2). Without action-token supervision, insulated models have higher observed first-sample pass rates on Public Golden under both staged schedules, even though coupled models have lower ADE. Routing differences on OOD vary. Thus, improvements in continuous trajectory prediction do not necessarily coincide with improvements in generated reasoning traces; neither gradient routing choice uniformly favors both outcomes.

Closed-loop driving. We evaluate both staged histories on the same AlpaSim scenes using the concurrent-training protocol from § 4.1. Table 3 shows that coupling increases observed route progress without action-token supervision after both Expert-First and VLM-First training. With supervision, coupling raises progress after Expert-First training but makes little difference after VLM-First training. Coupling without tokens roughly matches insulated training with tokens in the Expert-First history, but falls below it in the VLM-First history. Concurrent coupled training with tokens achieves the highest observed progress overall (0.703). Full staged competence and safety results appear in Appendix Table 12 and Table 13.

Metric	Insulated	Insulated	Coupled	Coupled
	Tokens on	Tokens off	Tokens on	Tokens off
Progress ↑	0.597	0.538	0.625	0.604
Distance traveled (m) ↑	115.25	104.28	119.83	116.13
Duration fraction ↑	0.599	0.461	0.561	0.553
Collision, any ↓	0.230	0.305	0.165	0.232
Collision, at fault ↓	0.061	0.182	0.097	0.139
Off-road ↓	0.159	0.247	0.134	0.182
Off-road or at-fault collision ↓	0.220	0.429	0.231	0.321
Distance to ground truth (m) ↓	2.152	2.569	2.850	2.514
Plan deviation ↓	0.346	0.549	0.353	0.413
Wrong lane ↓	0.190	0.202	0.288	0.222

Metric	Insulated	Insulated	Coupled	Coupled
	Tokens on	Tokens off	Tokens on	Tokens off
Progress ↑	0.648	0.555	0.646	0.587
Distance traveled (m) ↑	128.59	104.96	129.10	113.02
Duration fraction ↑	0.624	0.508	0.588	0.512
Collision, any ↓	0.208	0.264	0.204	0.175
Collision, at fault ↓	0.076	0.141	0.117	0.106
Off-road ↓	0.126	0.215	0.166	0.199
Off-road or at-fault collision ↓	0.203	0.355	0.282	0.305
Distance to ground truth (m) ↓	2.515	2.512	2.675	2.829
Plan deviation ↓	0.360	0.442	0.412	0.418
Wrong lane ↓	0.228	0.244	0.238	0.237

Thus, for Q2, training order changes the size of the routing effect, but coupling improves open-loop accuracy without action-token supervision under every schedule. Importantly, the greatest absolute performance in action-prediction across all training histories is achieved by concurrent training, which stands in contrast to nominal VLM-First recipes adopted by prior works [7, 34].

Transfer to Robot Manipulation

Finally, to address Q3, we repeat the concurrent training recipe from § 3 in robot manipulation with two model families: a $\pi_{0.5}$ re-implementation [34] and MolmoAct2 [7]. The four training conditions vary FAST supervision and expert-to-VLM gradient routing while sharing the data mixture, initialization procedure, and optimization schedule to isolate the effect of action-token prediction and gradient insulation.

Our $\pi_{0.5}$ re-implementation pairs a PaliGemma-3B VLM with a flow-matching action expert [1]. We use OpenPi [2] with the FAST implementation from Zha et al. [41], and train for 400k steps. Batches are divided equally between VQA and robot data. The VQA mixture comprises VQAv2 question answering (35%) [8], COCO captioning (20%) [4] and localization (20%) [24], and Visual Genome question answering (15%) and region tasks (10%) [18]. Robot data comprises 90% DROID [15] and 10% PolaRiS training data [14].

For MolmoAct2 [7], we initialize all four conditions from the official Molmo2-ER checkpoint, with fresh action experts and train concurrently for 300k steps. We reconstruct its published mixture from available public sources, retaining the 90% robot / 10% multimodal split. Each checkpoint then receives 1k steps of adaptation on 90% DROID and 10% PolaRiS training data. Architecture, attention interfaces, data mixtures, optimization, and adaptation settings are detailed in Appendix § 6.§ 3.

We evaluate both model families in the same three PolaRiS environments, which support comparisons from matched initial states [14]. BlockStackKitchen requires placing and stacking blocks on a green tray; FoodBussing requires putting food items into a bowl; and PanClean requires using a sponge on a frying pan in a cluttered kitchen. We run 50 rollouts per condition and environment from matched initial states. Because full task completion is sparse, our primary metric is normalized task progress, averaged equally across the three environments. Normalized task progress is the fraction of rubric criteria achieved during an episode, with credit retained once a criterion is satisfied (Appendix § 8).

For our $\pi_{0.5}$ re-implementation, the observed gain from coupling is larger without FAST: progress rises from 10.3% to 32.9%, compared with 19.5% to 30.7% with FAST (Table 4). Coupled training without FAST achieves the highest mean progress among our four conditions, about 1.7 times that of insulated training with FAST. For context, CoVeR reports 40.0% progress for a published $\pi_{0.5}$ checkpoint under its PolaRiS evaluation protocol [20]. That checkpoint was pretrained on more than 10,000 hours of robot data [34], whereas our robot training draws primarily from DROID, a 3,500-hour dataset [15].

Condition	$\pi_{0.5}$	MolmoAct2
Insulated / FAST on	19.5%	22.0%
Coupled / FAST on	30.7%	24.3%
Insulated / FAST off	10.3%	14.5%
Coupled / FAST off	32.9%	36.5%
Insulated / FAST on	40.0%	32.9%

MolmoAct2 exhibits the same qualitative pattern: coupling increases progress from 22.0% to 24.3% with FAST and from 14.5% to 36.5% without FAST. As before, coupled training without action token prediction achieves the highest mean progress among our four training strategies. For context, the released MolmoAct2 checkpoint reaches 32.9% after our adaptation and evaluation protocol (Table 4). Unlike our concurrent training recipe, the released checkpoint follows a VLM-First procedure that trains discrete action prediction with FAST before introducing the continuous expert [7]. Differences in training history and the VLM training mixture may contribute to the performance gap between our concurrent insulated model with FAST and the released checkpoint. The architecture, attention interfaces, training, and adaptation details appear in Appendix § 6.§ 3. Thus, the answer to Q3 is yes: coupling improves observed closed-loop manipulation progress with and without FAST supervision in both model families, with larger gains when FAST supervision is absent.

Discussion and Conclusion

Insulating the VLM from action-expert gradients is often motivated by preserving its pretrained capabilities. Our findings show that the consequences of this choice depend on the other signals available to the VLM. Action-token supervision directly trains it to predict future actions, which helps explain why blocking expert gradients has little effect on open-loop driving error when that supervision is present. Without action tokens, insulation substantially worsens trajectory prediction across evaluation sets and training schedules. Yet under concurrent training, coupling improves closed-loop route progress even with action-token supervision, and the same pattern appears in manipulation. Routing effects also vary across safety metrics: with action-token supervision, coupled staged training has higher observed at-fault collision rates, despite higher or similar route progress (Appendix Tables 12 and 13).

These results are consistent with action-token supervision and expert gradients providing partially substitutable action-learning signals. Insulation shows no consistent advantage in generated driving-rationale quality under the evaluated vision–language co-training recipe, although this finding does not establish broader VLM capability retention. Moreover, concurrently training the VLM and action expert from the outset is a strong, simple starting point. Our results suggest that gradient routing should be chosen jointly with action-token supervision and the training schedule, with its benefits assessed for the intended evaluation setting. In summary, the value of VLM insulation depends on action supervision, training schedule, and evaluation setting, thereby calling into question its use as a default VLA training strategy.

Limitations. We study only one driving model family and two manipulation model families, making our study somewhat limited in architectural diversity. Moreover, our closed-loop evaluations use only simulated driving scenes and simulated manipulation tasks. We do not test the manipulation policies on physical robots, which is an important next step for assessing whether the observed gains transfer to real-world control. Additionally, while our bootstrap intervals capture variation across evaluation clips for fixed checkpoints, compute constraints limited us to one training run per condition, leaving training-seed variation unmeasured. Finally, our study is empirical: we characterize the interaction between action-token supervision and gradient routing, but do not establish the mechanisms underlying the observed performance differences.

Future Work. Understanding how action-token supervision and expert gradients shape the VLM’s representations and learning dynamics is an important next step toward explaining when insulation helps or hurts. We are also excited to test whether this interaction extends to more embodiments, world-action models, and action-conditioned video world models.

Implementation and Training Details

This section reports the architecture, data mixtures, optimizer settings, and training schedules for the driving and manipulation experiments.

Driving model and action representation

The driving policy uses a Qwen3-VL-2B-Instruct [38] backbone and a non-causal flow-matching action expert. At each layer, the expert attends to the K/V representations of the corresponding VLM layer. Insulation detaches these representations before the expert reads them, leaving their forward values unchanged. Under coupling, the K/V representations remain attached and the flow loss can update the VLM.

The text tower and action expert use 128-dimensional attention heads. The vision tower outputs 2,048-dimensional features from $16 \times16$ patches with spatial merge size two. The action expert uses a two-layer waypoint input projection with width 512 and 20 Fourier features, followed by a linear output projection. The VLM context length is 16,384 tokens.

The expert represents the 6.4 s trajectory as 64 acceleration–curvature control pairs sampled at 10 Hz and converts them to pose waypoints using discretized unicycle dynamics. The VLM receives quantized ego-history tokens and, when action-token supervision is enabled, predicts a 128-token discretization of the same future control sequence.

When future-action CE is disabled, the 128 future value-token targets are masked, but their teacher-forced input tokens remain in the sequence. Past-trajectory values and the future-trajectory boundary markers remain supervised. The CE denominator is the number of supervised positions before masking, so removing future-action CE does not increase the weight of the retained autoregressive targets. The expert-facing K/V cache ends immediately after the <future_start> marker: it contains neither future value-token states nor the <future_end> state. Consequently, the expert cannot read the teacher-forced future values.

More precisely, let $\mathcal{F}$ contain future-trajectory value-token target positions, let $\mathcal{O}$ contain the other supervised positions, and let $\ell_i$ be next-token CE at position $i$. For $p = 1$ when action-token supervision is on and $p = 0$ when it is off, the implemented loss is

The non-action CE contribution therefore has the same scale under both settings. The flow and CE objectives each have unit weight on trajectory-bearing batches.

Driving flow matching and optimization

The driving flow path is $a_t = ta + (1-t)\epsilon$, with target vector field $a-\epsilon$ and $\epsilon\sim\mathcal{N}(0,I)$. We draw $b \sim\operatorname{Beta}(1.5, 1.0)$ and set $t = 0.999(1-b)$. At inference, the expert starts from Gaussian noise and uses 10 Euler steps. Classifier-free guidance is disabled.

Continuous action-prediction batches have sampling probability 48.5%, and vision–language batches have probability $\approx51.5\%$. Training examples with one, two, four, or seven cameras are sampled with relative weights 1:2:3:4; in evaluation, we always use four RGB images. The expert receives a flow update only on action-bearing batches, whereas the VLM receives an autoregressive update on every joint-training step. Each training run used 64 NVIDIA H100 GPUs and consumed approximately 15,000 GPU-hours on average. Table 5 reports the common optimization settings.

Setting	Value
Optimizer	Fused AdamW
$(\beta_1, \beta_2)$	$(0.9, 0.95)$
Adam $\epsilon$	$10^{-8}$
Weight decay	0.1
Vision LR	$10^{-5}$
VLM text/head LR	$10^{-4}$
Expert LR	$4 \times10^{-4}$
LR schedule	Cosine, 400k steps
Minimum LR factor	0.01
Global batch	128
Gradient clipping	1.0
Model precision	Bfloat16
Optimizer master	Float32

The concurrent conditions start from identical VLM and randomly initialized expert weights. Expert-First conditions share a 100k-step expert checkpoint trained on trajectory data with the VLM frozen, then receive 200k joint-training steps. VLM-First conditions train the VLM alone for 100k steps, either with or without future-action CE, before attaching experts with a shared fresh initialization for 200k joint-training steps. The VLM-First action-token-off warm-up uses the vision–language mixture without action-bearing batches. Comparisons within each history have matched training schedules and update opportunities.

Table 6 gives the expected component-level gradient exposure at the 300k-total-step endpoints and the intermediate VLM-First checkpoint. The Expert-First warm-up uses only trajectory data, so every warm-up step updates the expert. In VLM-First training, action-token supervision also changes the first-stage data mixture: the tokens-off backbone is trained on vision–language data alone, while the tokens-on backbone sees trajectory-prediction data. The four conditions share the same joint-training mixture after the expert is introduced.

Manipulation implementation and training

We compare two manipulation model families: a $\pi_{0.5}$ re-implementation and MolmoAct2. Within each family, the four conditions vary FAST supervision and expert-to-VLM gradient flow while sharing the data mixture, initialization procedure, and optimization schedule. The architectures and training settings differ across families, as described below.

$\pi_{0.5}$ implementation. The $\pi_{0.5}$ manipulation policy pairs a PaliGemma VLM with a flow-matching action expert (Table 7). Observations contain an external RGB image and a left-wrist RGB image, each resized to $224 \times224$, together with seven robot joint values and one gripper value. The state vector is padded to 32 dimensions. The expert predicts a 16-step action chunk in a 32-dimensional padded space; the first eight output dimensions encode seven joint values and one gripper value. During training, joint positions are expressed as deltas, while the gripper value remains absolute.

Component	Backbone	Layers	Width	MLP	Heads (Q/KV)
$\pi_{0.5}$
VLM Backbone	Gemma 2B	18	2,048	16,384	8/1
Action expert	Gemma 300M expert (random)	18	1,024	4,096	8/1
MolmoAct2
VLM Backbone	Molmo2-ER	36	2,560	19,456	32/8
Action expert	MolmoAct2 expert (random)	36	768	3,072	8/8

As in driving, insulation stops the expert’s gradient at the backbone K/V interface while preserving the forward representations. In FAST-supervised robot examples, the autoregressive branch predicts a FAST action postfix. Without FAST supervision, the postfix is absent and robot examples contribute no action-token CE; vision–language examples retain their language-answer CE in every condition. The expert is prevented from attending to FAST-token keys. This implementation differs from the driving intervention, where masked future-action value tokens remain in the training sequence. In the $\pi_{0.5}$ implementation, CE loss is normalized by the number of active target positions per example, clipped to at least one, and the flow loss is applied only to robot-action examples.

Each $\pi_{0.5}$ training batch draws a binomially distributed number of vision–language examples with probability one half; the remaining examples come from the robot stream. The streams therefore contribute equally in expectation, although individual batches need not contain exactly half of each type. The robot stream samples 90% DROID data and 10% PolaRiS data. Within the vision–language stream, task weights are 35% VQAv2 question answering, 20% COCO captioning, 20% COCO localization, 15% Visual Genome question answering, and 10% Visual Genome region tasks. A deterministic 2% hash split is held out for validation; robot examples are split by episode, and COCO examples use image identifiers where available to prevent cross-task image overlap. Table 8 records the $\pi_{0.5}$ manipulation training settings. Inference uses 10 Euler steps from Gaussian noise.

Setting	Value
Optimizer	AdamW
$(\beta_1, \beta_2)$	(0.9, 0.95)
Adam $\epsilon$	$10^{-8}$
Weight decay	$10^{-10}$
Peak/terminal LR	$5 \times10^{-5}$
Warm-up	1,000 steps
Global batch	128
Gradient clipping	1.0
EMA decay	0.99

MolmoAct2 implementation. We use the released MolmoAct2 implementation [7], initializing each condition from the official allenai/Molmo2-ER checkpoint with a fresh, randomly initialized action expert. The expert conditions on the VLM’s layerwise K/V representations. Insulation stops the expert’s gradient at this interface while preserving the forward representations; without insulation, the flow-loss gradient reaches the VLM. In FAST-supervised robot batches, training jointly optimizes FAST-token CE and continuous flow matching using allenai/MolmoAct2-FAST-Tokenizer. Without FAST supervision, FAST action targets are absent and robot batches retain only continuous flow matching. Separate multimodal batches retain their language objective and contribute no expert loss. We retain the implementation’s token-loss weighting and global-batch normalization rather than the per-example CE normalization used for $\pi_{0.5}$.

All four conditions train for 300k steps with the action expert present and trainable from the outset. This differs from the released staged recipe, which first trains discrete action prediction and subsequently adds the continuous expert, but keeps relative training and compute comparable. Depth reasoning and depth-token supervision are disabled. The VLM, vision encoder, embedder, and action expert remain trainable in every condition.

MolmoAct2 data mixture. Each MolmoAct2 optimizer step draws a robot or multimodal batch with probabilities 90% and 10%, respectively. Within the robot stream, sampling weights are 30% YAM, 30% SO100/SO101, 30% DROID, and 2.5% each for MolmoAct, BC-Z, RT-1, and Bridge. Our public reconstruction includes all 737 YAM repositories, 1,202 of the 1,220 SO100/SO101 repositories in the released configuration, and all six consolidated repositories covering the remaining robot groups. The 18 unavailable SO repositories account for approximately 0.398% of the original overall loader-selection mass; their weight is redistributed within the SO group.

Within the multimodal stream, sampling weights are 46% Molmo2, 46% Molmo2-ER, and 8% Tulu-3. We use the publicly available sources and redistribute the weights of unavailable sources within each group. All four conditions use the same data mixture. We retain the released embodiment-specific robot processing and quantile normalization. For SO datasets with missing normalization statistics, we estimate them from the converted data.

MolmoAct2 optimization. Training uses global batch 128 and bfloat16 mixed precision on 64 H100 GPUs. We use AdamW with $(\beta_1, \beta_2) = (0.9, 0.95)$, $\epsilon= 10^{-6}$, and the learning rates reported in Table 9. After 200 warm-up steps, learning rates decay with a cosine schedule to 10% of peak at step 200k and remain fixed through step 300k. Gradient clipping uses threshold 1.0.

Component	Peak LR
Language model	$1 \times10^{-5}$
Vision encoder	$5 \times10^{-6}$
Connector	$5 \times10^{-6}$
Action expert	$5 \times10^{-5}$

MolmoAct2 adaptation and inference. Each final 300k-step checkpoint receives 1,000 adaptation steps on 90% DROID and 10% PolaRiS training data. Adaptation uses global batch 128 and the learning rates reported in Table 9, with 200 warm-up steps followed by cosine decay to 10% of peak. Each condition retains its FAST supervision and insulation settings.

Observations contain an exterior RGB image, a left-wrist RGB image, seven robot joint values, one gripper value, and the task instruction. Images use MolmoAct2’s native processing at $378 \times378$, and states use its discrete encoding and saved normalization statistics. The expert predicts 15-step action chunks with eight active dimensions in a 32-dimensional padded space. Joint positions and the gripper value are absolute, and the gripper is not quantile-normalized. Inference uses 10 flow-integration steps.

We also adapt the released allenai/MolmoAct2 checkpoint for 1,000 steps on the same DROID/PolaRiS mixture. These reference models use FAST supervision with insulation off during adaptation and are evaluated separately from the four factorial conditions.

Evaluation and Statistical Details

Open-loop driving

The Public Golden evaluation dataset contains 1,434 examples; the out-of-distribution (OOD) evaluation contains 273 examples. Both evaluations use four cameras, four temporal frames per camera, and a 16-waypoint ego-trajectory history. The policy generates a chain of thought before sampling six continuous future trajectories. Chain-of-thought generation uses temperature 0.6, top-$p$ 0.98, and a maximum of 256 generated tokens, which we found realized the best performance. All driving results in this paper use this inference mode and the same dataset and prompt configuration across training histories.

For each candidate trajectory, ADE is the mean Euclidean distance in the ground plane between predicted and ground-truth waypoints over the 64-step horizon, measured in meters. Minimum ADE selects the lowest-ADE candidate among the six using ground truth. For open-loop bootstrap confidence intervals, we resample base-clip clusters with replacement and retain every example belonging to each sampled clip. We apply the same sampled clips, with the same multiplicities, to all four conditions before recomputing condition means and paired differences. We use 10,000 such resamples and report the 2.5th and 97.5th percentiles of each bootstrap distribution as the 95% interval.

Additional Manipulation Results

We evaluate all four conditions within each model family in three PolaRiS DROID environments [14]: BlockStackKitchen, FoodBussing, and PanClean. The $\pi_{0.5}$ policies are evaluated after 400k training steps, and the MolmoAct2 policies after 300k training steps followed by 1,000 adaptation steps. PolaRiS places reconstructions of real scenes in a physics simulator, allowing policies to be compared from matched initial conditions. Each environment uses one instruction, held fixed across conditions:

• BlockStackKitchen: “Place and stack the blocks on top of the green tray”

• FoodBussing: “Put all the foods in the bowl”

• PanClean: “Use the yellow sponge to scrub the blue handle frying pan”

Evaluation protocol. Each condition is evaluated on the same initial conditions in every environment, which vary per episode, pairing the 50 episodes by scene across conditions. Episodes run for the default 450-step horizon without stopping early on success. The policies receive exterior and wrist RGB images and predict seven joint positions and a gripper command. For $\pi_{0.5}$, images are resized with padding to $224 \times224$; MolmoAct2 uses its native image processing at $378 \times378$. Importantly, we match the training-time inputs in the simulator to prevent distribution shift and give an accurate assessment of policy performance. Actions are executed open-loop in eight-step chunks. We use one training seed per condition.

Scoring. Normalized task progress is the fraction of an environment’s rubric criteria satisfied. Each criterion latches once satisfied; criteria with dependencies count only after the required criteria have been satisfied. Progress therefore cannot decrease, and success requires all criteria. Table 10 specifies the conditions and dependencies. For PanClean, the rubric scores placement of the sponge on the pan; it does not separately score a scrubbing motion. Figure 4 showcases qualitative demonstrations of the rollouts in simulation.

#	Criterion	Condition	Requires
1	Reach sponge	End effector within 0.2 m of sponge	—
2	Lift sponge	Sponge height $> 0.09$ m	1
3	Sponge on pan	At least 80% overlap with pan; gripper open	2
1	Reach ice cream	End effector within 0.2 m	—
2	Reach grapes	End effector within 0.2 m	—
3	Lift ice cream	More than 0.06 m above rest height	1
4	Lift grapes	More than 0.06 m above rest height	2
5	Ice cream in bowl	At least 80% overlap with bowl; gripper open	3
6	Grapes in bowl	At least 80% overlap with bowl; gripper open	4
1	Reach green cube	End effector within 0.2 m	—
2	Reach wood cube	End effector within 0.2 m	—
3	Lift green cube	Cube height $> 0.09$ m	1
4	Lift wood cube	Cube height $> 0.09$ m	2
5	Green cube on tray	At least 80% overlap with tray; gripper open	3
6	Wood cube on tray	At least 80% overlap with tray; gripper open	4
7	Green cube on wood cube	At least 50% overlap with wood cube; gripper open	5, 6

Additional Open-Loop Driving Results

In the main report, we utilized ADE as our metric; here, we report minADE.

Minimum ADE

Minimum ADE uses the ground truth to choose the best of six sampled trajectories. At the 300k endpoints, its routing pattern is consistent with the selected-trajectory ADE results: removing action-token supervision is more costly under insulation than under coupling across the concurrent, VLM-first, and expert-first histories on both evaluation datasets. Concurrent training achieves the lowest observed minADE on Public Golden, whereas VLM-first training achieves the lowest on OOD (Figure 5).

Additional Closed-Loop Driving Results

Tables 11, 12, and 13 report the full AlpaSim results. All conditions use the same 827 scenes, with five stochastic rollouts averaged within each scene. AlpaSim terminates or truncates after safety events and censors trajectories at 4 m from the ground-truth trajectory. Arrows indicate the preferred direction, and the best value in each row is bolded. We moreover provide a qualitative demonstration of policy performance in Figure 6.

Coupling increases route progress without action-token supervision in both staged histories: from 0.538 to 0.604 after expert-first training and from 0.555 to 0.587 after VLM-first training. With action-token supervision, coupling increases progress from 0.597 to 0.625 after expert-first training, while the VLM-first values are nearly identical (0.648 insulated and 0.646 coupled). The highest observed route progress across all three histories is 0.703 under concurrent coupled training with action-token supervision (Table 11).

Metric	Insulated		Coupled
	Tokens on	Tokens off	Tokens on	Tokens off
Progress $\uparrow$	0.600	0.425	0.703	0.601
Distance traveled (m) $\uparrow$	116.68	78.63	139.59	114.78
Duration fraction $\uparrow$	0.590	0.363	0.634	0.541
Collision, any $\downarrow$	0.225	0.311	0.164	0.211
Collision, at fault $\downarrow$	0.087	0.187	0.090	0.127
Off-road $\downarrow$	0.161	0.304	0.136	0.184
Off-road or at-fault collision $\downarrow$	0.248	0.491	0.226	0.311
Distance to ground truth (m) $\downarrow$	2.378	2.471	2.700	2.732
Plan deviation $\downarrow$	0.380	0.508	0.321	0.378
Wrong lane $\downarrow$	0.201	0.170	0.230	0.206