WorksComputer Science
CARE-Drive: Coupled Alternating Refinement for Ego-trajectories
Abstract
Vision-language-action (VLA) models bring multimodal understanding and world knowledge to end-to-end autonomous driving. To further improve the quality of predicted trajectories, existing studies introduce candidate refinement, residual denoising, and token correction. However, these methods typically focus on unified refinement of the entire trajectory, paying limited attention to the distinct physical quantities and their interactions. We further investigate physical specialization and coordination in trajectory refinement, centering on position and heading for simpler and more intuitive correction modeling. To this end, we propose CARE-Drive (Coupled Alternating Refinement for Ego-trajectories), which decomposes trajectory refinement into waypoint residual correction and position-conditioned heading generation. Through imitation learning and a future-token auxiliary task, the backbone provides predictive representations and a base trajectory. The position expert predicts bounded metric waypoint residuals, while the heading expert reconstructs vehicle orientation from adjacent angular increments conditioned on the corrected path. With the backbone frozen, the two experts are optimized alternately through supervised initialization and reinforcement fine-tuning. This design simplifies refinement through clear physical specialization while preserving conditional coupling during prediction and optimization. On NAVSIM v1, CARE-Drive achieves 88.8 PDMS, improving by 2.4 points over the unrefined base trajectories generated by the same backbone. Ablation studies further support the effectiveness of specialized experts and staged training.
Introduction
Autonomous driving models draw on complementary perception and prediction architectures. Convolutional and Transformer-based encoders extract camera or bird’s-eye-view (BEV) features for planning [4], while vision–language models (VLMs) provide pretrained multimodal understanding and a language-generation interface [2, 20]. World models capture temporal structure through video generation [16, 18] or latent prediction. Joint-embedding predictive architecture (JEPA) methods predict target representations without requiring pixel reconstruction [1]. CoWorld-VLA [5] illustrates how predictive representations can complement a VLM backbone.
VLMs are particularly attractive because their language interface supports instruction following and visual question answering. Future-feature supervision can additionally train the same backbone to anticipate scene evolution in latent space [5]. This provides an architectural basis for combining world-future reasoning with language interaction, while exposing shared features for action planning.
Action generation and refinement can use direct trajectory regression, autoregressive physical-action decoding [20], iterative diffusion [11], or candidate evaluation [8]. Reinforcement fine-tuning further adapts policies to planning rewards [20]. Residual correction offers a complementary approach: adjusting an initial trajectory with targeted updates. For physically interpretable refinement, the key question is how to assign these updates to quantities with distinct roles, such as position and heading.
As shown in Figure 1, we propose CARE-Drive, a future-supervised VLA framework with physically meaningful correction experts. A Qwen3-VL-2B backbone learns predictive representations and an initial trajectory. A position expert predicts bounded XY residuals, and a heading expert reconstructs orientation from adjacent angular increments conditioned on corrected positions and shared features. This gives each expert an explicit physical target and makes heading refinement responsive to position changes. The experts are optimized with the backbone frozen, keeping correction learning modular and the backbone’s language-generation interface structurally intact. Deployment produces one trajectory through greedy decoding.

Figure 1. CARE-Drive overview. (a) The predictive backbone produces shared action features and an initial trajectory; a frozen video teacher provides future-feature supervision during training. (b) The position expert predicts metric waypoint residuals, and the heading expert reconstructs orientation from adjacent angular increments conditioned on corrected positions and shared action features. Greedy decoding yields one eight-waypoint trajectory. (c) Dynamically weighted imitation learning trains the backbone, followed by alternating position and heading optimization with the backbone frozen. Each expert update holds its conditioning models fixed. Snowflakes mark frozen modules.
Our contributions are twofold:
An effective world–action framework. We propose CARE-Drive, integrating future-supervised representation learning with staged trajectory refinement for end-to-end driving. Experiments on NAVSIM v1 demonstrate favorable planning performance.
Physically meaningful correction experts. We design complementary position and heading experts that output metric waypoint residuals and adjacent angular increments. Corrected positions condition heading generation, giving spatial and angular refinement explicit physical meaning.
Related Work
End-to-end and generative planning. UniAD [4] studies planning-oriented perception and prediction. Hydra-MDP [10] and DiffusionDrive [11] introduce multimodal planning and generative trajectory modeling. CARE-Drive uses a continuous trajectory anchor followed by categorical refinement. § 4.3.5 compares its end-to-end latency with a parameter-matched single correction expert.
World models for driving. LAW [6] learns latent world representations for planning. Epona [18], PWM [19], WoTE [8], and DriveLaW [16] investigate generative, predictive, or evaluative links between future states and actions. CoWorld-VLA [5] transfers complementary semantic, geometric, dynamic, and trajectory information into expert tokens, and combines them with a hierarchical fusion planner. Its framework is the closest reference for organizing our external comparisons. Our implementation draws on the CoWorld codebase and uses three camera histories, one future-feature teacher, an MLP anchor, and two categorical action experts. Here, world–action modeling couples predictive latent supervision to action learning.
VLA and reinforcement fine-tuning. AutoVLA [20] uses autoregressive physical actions and adaptive language reasoning. ReCogDrive [9], DriveVLA-W0 [7], and Uni-World VLA [12] provide further examples of cognitive or world-informed planning. Our action experts use group-relative policy optimization inspired by GRPO [15] and a clipped token-level surrogate related to PPO [14]. The heading-stage greedy baseline is inspired by self-critical sequence training [13]. We combine these objectives with batch-level RMS calibration of candidate advantages.
Method
Task and Notation
For scene , the observation is , where denotes camera history, the ego-state description, and the high-level route command. The history contains four frames from each of three cameras (front, front-left, and front-right), sampled at , , , seconds at 2 Hz. The task is to predict an ego trajectory from this observation:
Here comprises the backbone and correction experts, and is the ground-truth target. Each trajectory contains positions and headings at 0.5-second intervals over four seconds, expressed in the local ego frame. Position is measured in meters and heading in radians. Subscripts index scenes and future steps; superscript base denotes the backbone prediction, denotes ground truth, and a hat denotes a model prediction. and contain the corresponding XY positions.
During backbone training, CARE-Drive couples trajectory imitation with a future-token auxiliary task. Given , future query tokens learn to predict latent features of future images, with targets supplied by a frozen video teacher. Action query tokens attend to the future-query states through causal attention and provide the features used to generate . Joint optimization of future-feature prediction and trajectory regression trains the shared backbone to encode scene evolution for planning. The trained backbone then supplies the base trajectory and shared action features to the coupled position and heading experts.
Future-Token Auxiliary Task
The backbone uses a frozen visual tower and a trainable language module. Learnable future query tokens and action query tokens follow the observation tokens in the causal input sequence. We use , , and hidden dimension . The future queries use four temporal token identities, each repeated at 24 spatial positions; the eight action queries have distinct token identities. Their embeddings are learned with the language backbone. Writing for the backbone parameters, the corresponding contextual hidden states are
As shown in Figure 1(a), a two-layer projection predicts teacher features , with . The base MLP reads the last action state and produces . Future-query information reaches the action states through the backbone’s causal attention.
A frozen V-JEPA 2.1 ViT-G encoder supplies training targets , following the predictive-feature direction of V-JEPA [1]. It encodes eight future images resized to 384 × 384. The resulting feature grid has four temporal groups, each spatially pooled to , giving 96 vectors of dimension 1,664. The auxiliary future-feature
The prediction loss is
SmoothL1 uses a transition value of one and averages over tokens and features. The teacher and future images provide training supervision; inference uses the observed camera history, ego state, and route command.
Epoch-Wise Difficulty-Weighted Imitation
The base trajectory loss averages the absolute errors of the two position coordinates and heading over all predicted
waypoints:
Position errors are computed in meters and heading errors in radians. Each component enters directly in these physical units with coefficient one.
The first epoch uses uniform weights. Before epochs two through five, the previous frozen epoch endpoint predicts trajectories on the scoreable training population . Let and be official normalized PDMS rewards for the ground-truth trajectory and the model trajectory.
Define
Weights remain fixed within epoch . For training scenes, the objective is
The weights apply to the trajectory term; the future-feature loss retains coefficient one.
Position and Heading Experts
After backbone training, and are fixed. The position expert is an autoregressive Transformer conditioned on both. It generates 16 tokens in interleaved order. Each axis has 401 categories, with index for coordinate . Let and . Training-data residual and quantization analyses support the adopted bounds m and m; the analysis is detailed in Appendix § 8. Residual decoding is
We use fixed symmetric residual bounds with 401 bins per axis, yielding resolutions of 0.025 m in X and 0.015 m in Y.
Supervised labels are formed by clipping and rounding the target residuals. The expert also receives normalized anchor displacement, heading, and time features, as detailed in Appendix § 8.
The heading expert receives and the corrected sequence from the position expert (Figure 1(b)). Analysis of adjacent heading increments in training samples supports the adopted range (Appendix § 8). The expert generates eight increments, each uniformly quantized in steps into 401 values with indices :
The heading sequence is reconstructed by accumulating predicted increments from zero in the local ego frame, conditioned on shared action features and corrected XY positions. Each expert has eight layers, width 768, eight attention heads, and an FFN width of 3,072. The position and heading experts each contain approximately 78.42M parameters, paired with a Qwen3-VL-2B backbone.
During position-policy training, the frozen preliminary heading expert generates a matching heading sequence for each candidate XY trajectory, and the reward is evaluated on the resulting complete trajectory. Subsequent heading adaptation starts from that preliminary expert and holds the final position expert and its greedy XY fixed.
Rewards and Expert Optimization
Training reward. Let denote the normalized NAVSIM v1 planning reward. Position refinement uses
where NC, DA, TTC, and C assess collision, road, time-to-collision, and comfort compliance, and EP measures progress. We use in the first position RL stage and in the second, encouraging progress only among compliant candidates. Evaluation always uses the original reward ; coefficient comparisons are reported in § 4.3.6.
Candidate advantages. For each scene , we sample eight trajectories. Let be the standard PDMS-based advantage, with the numerically stabilized within-scene reward standard deviation. Preliminary heading training uses ; position training and final heading adaptation use
where is the current policy’s greedy reward, treated as a constant. Thus, position candidates are compared with their group mean, while final heading candidates are compared with the policy’s greedy output.
Scale calibration and policy update. These two stages apply a shared batch-wise scale,
where contains the batch’s candidate trajectories. The scaling preserves advantage signs and relative magnitudes while keeping the overall learning-signal scale comparable across reward settings. Experts are updated using a clipped GRPO objective with KL regularization toward the stage’s fixed reference policy [14, 15].
Training Curriculum and Inference
The backbone imitation and alternating expert optimization in Figure 1(c) comprise eight five-epoch stages: backbone imitation; heading SFT followed by two heading RL stages; position SFT followed by two position RL stages; and final heading adaptation.
Both preliminary heading RL stages optimize PDMS on the backbone’s fixed XY. The first starts from the heading SFT endpoint; the second starts from the first RL endpoint and uses it as a refreshed, fixed KL reference. The reward and advantage definition remain unchanged, while the optimizer and scheduler are restarted for the new stage.
Position training keeps this preliminary heading expert frozen. The first position RL stage optimizes PDMS; the second introduces the safety-gated progress bonus. Finally, with the trained position expert and its greedy XY fixed, the heading expert is adapted using the greedy-baseline, RMS-calibrated update described above. Backbone training has 3,625 updates; each expert stage has 3,620 updates, for a total of 28,965 optimizer updates.
Inference computes , , greedily decodes 16 position tokens, then greedily decodes eight heading increments.
Experimental Results
We evaluate on NAVSIM v1 [3], a trajectory-planning benchmark built from real-world driving logs. It provides sensor observations and ego states for predicting future ego trajectories. Evaluation uses non-reactive simulation, in which other traffic participants follow their recorded motion. Planning performance is measured by the PDM Score (PDMS), which combines safety, driving progress, and comfort.
Test-Set Performance
CARE-Drive achieves 88.837 PDMS, placing third among the VLM/VLA methods listed in Table 1. It exceeds the locally evaluated AutoVLA score of 88.026 and approaches CoWorld-VLA’s 90.0, with a gap of 1.163 points. A complete backbone-and-expert training workflow takes approximately 19 hours on 16 Ascend 910 accelerators.
| Family | Method | Sensors | Frames | PDMS ↑ |
| End-to-end | UniAD [4] | C | 83.4 | |
| Hydra-MDP [10] | C+L | 86.5 | ||
| DiffusionDrive [11] | C+L | 88.1 | ||
| World models | LAW [6] | C | 84.6 | |
| Epona [18] | C | 86.2 | ||
| PWM [19] | C | 88.1 | ||
| WoTE [8] | C+L | 88.3 | ||
| ResWorld [17] | C+L | 89.0 | ||
| DriveLaW [16] | C | 89.1 | ||
| VLM / VLA | ReCogDrive (w/o RL) [9] | C | 1 | 86.5 |
| DriveVLA-W0 [7] | C | 87.2 | ||
| AutoVLA [20] | C | 4 | 88.026 | |
| Uni-World VLA [12] | C | 89.4 | ||
| CoWorld-VLA [5] | C | 1 | 90.0 | |
| This work | CARE-Drive | C | 4 | 88.837 |
Table 1.
Effect of Expert Refinement
Table 2 summarizes the planning gains from progressively adding correction experts to the same frozen backbone. The experts are trained on 92,614 scenes, and all four configurations are evaluated on the complete 12,146-scene navtest split. The complete refinement pipeline increases PDMS from 86.462 to 88.837, a gain of 2.375 points over the unrefined backbone trajectory.
| Configuration | Full-test PDMS ↑ | Gain over backbone |
| Backbone only | 86.462 | — |
| Add heading expert | 87.562 | +1.100 |
| Add position expert; freeze heading expert | 88.672 | +2.210 |
| Freeze position expert; adapt heading expert | 88.837 | +2.375 |
Table 2.
The heading expert first preserves the backbone’s XY waypoints and changes only their headings, raising PDMS to 87.562, an increase of 1.100 points. The position expert then undergoes supervised initialization and reward-based training with the preliminary heading expert frozen; each corrected path receives its corresponding conditional headings. This stage raises PDMS to 88.672, a further gain of 1.110 points. Finally, with the corrected XY fixed, the heading expert adapts to its new conditioning inputs and reaches 88.837, adding 0.165 points.
Ablation Studies
Effect of Future Queries and Supervision
We examine future queries and future-feature supervision using 8,933 sampled training scenes and an 885-scene sampled test set, with an L1 trajectory loss. The three variants remove the future branch, retain 96 future queries without future supervision, or retain the same queries with the future-token auxiliary task. The three variants share the sampled data and the remaining training settings, and are evaluated before expert correction.
Table 3 shows a 0.370-point increase from adding future queries alone. With the query structure fixed, future supervision adds a further 2.120 points, bringing the total improvement over the variant without a future branch to 2.490 points. This ablation demonstrates the value of the future-token auxiliary task for trajectory planning.
| Variant | Future queries | Supervision | PDMS ↑ |
| No future branch | 0 | No | 74.310 |
| Queries only | 96 | No | 74.680 |
| Auxiliary task | 96 | Yes | 76.800 |
Table 3.
Visual-Tower Unfreezing
We assess visual-tower unfreezing on 8,933 sampled training scenes and the 885-scene sampled test set. This comparison retains the original trajectory-MSE training setup. The frozen and unfrozen variants start from the same pretrained backbone and prediction-head initialization, with identical 1024 × 512 video inputs, future supervision, and a ten-epoch budget of 700 updates, and the same training seed. The unfrozen variant also trains the visual merger, while the future-feature teacher remains frozen in both variants. We evaluate the final-epoch backbone trajectories before expert correction.
Unfreezing reduces test PDMS by 5.101 points (Table 4), with a paired log-bootstrap 95% interval of [-7.646, −2.548]. This comparison supports keeping the vision tower frozen under the evaluated training configuration.
| Student vision tower | Test PDMS ↑ |
| Frozen | 74.198 |
| Unfrozen (including merger) | 69.097 |
Table 4.
Difficulty-Weighted Imitation Learning
To evaluate difficulty weighting with a larger training set, we train on all 92,772 eligible scenes in our NAVSIM v1 training split and evaluate on the complete navtest split of 12,146 scenes. We compare uniform weighting with dynamic difficulty weighting under the same pretrained initialization, training seed, five-epoch budget, trajectory L1 objective, and future supervision. Uniform weighting assigns unit weights to all scenes. Dynamic weighting starts with uniform weights and then applies the epoch-wise PDMS-gap rule in Equation (5), emphasizing scenes where the previous endpoint’s trajectory scores more than one PDMS point below the ground truth. Evaluation uses the fifth-epoch backbone trajectories before expert correction.
Dynamic weighting increases full-test PDMS from 85.863 to 86.462, a gain of 0.600 points (Table 5). The improvement is measured at the initial-trajectory stage, before expert correction.
| Training | Full-test PDMS |
| Uniform weighting | 85.863 |
| Dynamic weighting | 86.462 |
Table 5.
Table 1: Test-set PDMS on NAVSIM v1. AutoVLA and CARE-Drive are evaluated locally; other results follow CoWorld-VLA [5]. C = camera, L = LiDAR; denotes multiple input frames without an exact count in the source.
Table 2: Effect of progressively adding correction experts on the full test set. Gains are relative to the backbone-only configuration.
Joint Expert Training
Table 3: Effect of the future-token auxiliary task.
The sequential condition is the training procedure used by CARE-Drive: following preliminary heading training, the position expert is optimized with the heading expert frozen, and the heading expert is then adapted with the position expert frozen. The joint conditions update both experts together during SFT and subsequent GRPO.
Table 4: Effect of visual-tower freezing.
Table 6: Staged versus joint expert training.
| Configuration | PDMS ↑ |
| Backbone only | 86.462 |
| Staged expert training (CARE-Drive) | 88.837 |
| Joint SFT | 85.651 |
| Joint SFT + GRPO | 83.671 |
Table 6.
Table 5: Difficulty-weighted imitation learning. Full-test PDMS of the backbone without correction experts.
The proposed staged training reaches 88.837 PDMS, outperforming joint SFT and joint SFT followed by GRPO by 3.186 and 5.166 points, respectively (Table 6). Both joint-training variants are weaker than the sequential training recipe used by CARE-Drive. These results support the staged optimization of the two experts used in our method.
Correction-Module Structure and Capacity
Parameter-matched MLP expansion. To examine whether the gains arise simply from adding parameters, we expand the ordinary MLP trajectory decoder by approximately 156.84M parameters, matching the additional capacity of the two correction experts. The expanded MLP achieves 86.510 PDMS, only 0.048 points above the backbone-only score of 86.462 and 2.327 points below the dual-expert score of 88.837 (Table 7). The physically structured correction experts achieve a larger gain than the parameter-matched MLP expansion in this comparison.
| Configuration | Added parameters (M) | PDMS ↑ |
| Backbone only | 0 | 86.462 |
| MLP decoder expansion | 156.84 | 86.510 |
| Separate experts (CARE-Drive) | 156.84 | 88.837 |
| Single expert, final RL | 156.84 | 84.227 |
| Single expert, after SFT | 156.84 | 85.147 |
Table 7. Correction-module structure and added capacity on the full test set.
Single versus separate experts. We replace the two specialized correction experts with a single expert that handles both position and heading correction. The single expert uses one 16-layer autoregressive decoder to generate position residuals and adjacent heading increments, whereas CARE-Drive uses two independent eight-layer experts. Both configurations use the same frozen backbone, 92,614 training scenes, physical output representations, 156.84M correction parameters, and number of training updates.
Under matched parameter counts and update budgets, CARE-Drive’s separate experts outperform the single expert’s final RL endpoint by 4.610 PDMS points (Table 7). The single expert also has 7.8% higher online inference latency. These results favor the complete recipe with separate position and heading experts in the evaluated setting.
Safety-Gated Progress Reward
We test whether the second position RL stage benefits from reward shaping rather than additional training alone. With a common L1 backbone and heading expert frozen, all variants start from the same first-stage position RL checkpoint and train for five more epochs using the same data, KL reference, optimizer settings, and update budget. Only the coefficient in Eq. (9) changes; continues with the original PDMS reward.
The coefficient gives the highest full-test point estimate of 88.840 PDMS, a gain of 0.101 over the original-reward control (Table 8). At , the test score is slightly below the original-reward control.
| Reward coefficient | Test PDMS | PDMS |
| 0 (original reward) | 88.738 | — |
| 0.5 | 88.749 | +0.010 |
| 1.0 | 88.840 | +0.101 |
| 2.0 | 88.720 | −0.018 |
Table 8. Safety-gated progress reward ablation on the full test set. Differences are relative to .
Performance Analysis
Figure 2 compares backbone and CARE-Drive trajectories in straight-driving, left-turn, right-turn, and lane-change scenes. Position and heading corrections change vehicle footprints relative to road boundaries and nearby objects, improving collision, drivable-area, and TTC compliance in the illustrated cases. Additional examples are provided in Appendix § 11.

Figure 2. Qualitative comparison of backbone and complete-system predictions. Columns show straight driving, a left turn, a right turn, and a lane change. Green denotes the reference trajectory, dashed blue the backbone, and red CARE-Drive. Orange marks related objects; dotted vehicle outlines denote TTC projections, and black regions mark overlaps. Crosses mark off-road corners and circles the corresponding corners of the other prediction. Local enlargements highlight the geometric changes in these selected improvement cases. Scene PDMS is shown on the [0, 1] scale.
Figure 3 summarizes correction magnitudes and safety-component changes on a random sample from the test set. The median and 90th-percentile position and heading changes relative to the backbone increase with the prediction horizon. Improved/degraded scene counts are 50/33 for NC, 293/92 for drivable-area compliance, and 167/84 for TTC, with improvements outnumbering degradations in all three components.

Figure 3. Geometry, magnitude, and safety outcomes of correction. (a) Vehicle-footprint examples near road boundaries, with local enlargements. (b,c) Median and 90th-percentile absolute position and heading changes relative to the backbone, by prediction time. (d) Improved and degraded scene counts for collision compliance (NC), drivable-area compliance (DA), and time-to-collision compliance (TTC). The statistics use a random sample from the test set.
Discussion
Predictive supervision and action interfaces. The system demonstrates how future-feature supervision can coexist with a fully observation-based inference path and separate trajectory controls.
Cross-attention readout. We also tested an additional cross-attention readout adapter, but it did not yield better planning performance. The final model therefore uses no additional cross-attention readout adapter. Detailed experimental metrics are reported in Appendix § 10.§ 1.
Safety objectives and tail behavior. The supplementary reward experiments in Appendix § 10 highlight a practical safety tradeoff. L1 error is continuous and averaged over coordinates, whereas NC and DA multiply the planning reward and can change discontinuously with trajectory geometry. In the heading-penalty experiment, scenes with NC below one decrease from 204 to 201, while scenes with DA below one increase from 497 to 522. The two compliance components therefore move in different directions; the corresponding PDMS and component counts are reported together in Appendix § 10.
Conclusion
We presented CARE-Drive, a future-supervised world–action model combining a deterministic base trajectory with physically meaningful position and heading correction. The complete system achieves 88.837 PDMS on the test set. The ablations support physically structured correction: the proposed staged expert recipe outperforms the tested joint-training, single-expert, and MLP-expansion alternatives. This modular framework provides a basis for extending the approach to more complex, physically meaningful correction experts.
Implementation and Training Details
Backbone. The visual input uses three separate four-frame videos at 1024 × 512 per frame, with the recorded frame indices and FPS. The visual tower, merger, and associated visual projections are frozen; the language backbone, token embeddings, future projection, and trajectory MLP are trained. The forward pass uses BF16 while trainable parameters and AdamW states use FP32. The 96 future query positions use four temporal token identities repeated 24 times each. The eight action queries have distinct token identities. Backbone optimization combines trajectory regression and future-feature supervision.
Backbone AdamW uses , , , weight decay 0.01, and gradient clipping at one. Peak learning rates are for the language module, for the future projection, and for the trajectory head. A continuous cosine schedule follows 5% warmup. The effective batch is 128, with microbatch one, eight accumulation passes, and 16 devices. The five epochs contain 725 updates each, including a final batch of 100 real scenes. Dynamic weights are applied before averaging over the real scenes in the global batch.
Expert memories. The position expert projects eight action features and eight base-trajectory features into separate memory token types. Base features concatenate , , adjacent displacement divided by ten, , , and normalized time . Memory type and temporal embeddings are added before normalization. The autoregressive decoder includes causal self-attention and cross-attention to this memory. Position tokens also use time and axis embeddings.
The heading expert uses the same action features and corrected-XY features: scaled XY, scaled adjacent displacement, and . It has separate memory type and position embeddings. Heading labels are formed by wrapping adjacent ground-truth angle differences, clipping to , and rounding to 0.1-degree units. Decoding sums increments before angle wrapping.
Quantization range analysis. For position residuals, we analyzed the target-minus-backbone XY differences in 92,614 training scenes from the preceding L1-backbone configuration. With m, m, and 401 uniformly spaced values per axis, clipping affected 0.4157% of scenes. Reconstructing the target trajectories from quantized residuals gave an ADE of 0.0088 m. Increasing to 10 m with the same number of values reduced the clipped-scene fraction to 0.1890% and increased the reconstruction ADE to 0.0138 m. This comparison characterizes the coverage–resolution tradeoff underlying the adopted position bounds.
For heading increments, we analyzed 500 uniformly sampled training scenes, containing 4,000 adjacent ground-truth heading differences at 0.5-second intervals. Differences were angle-wrapped before quantization. All 4,000 increments lay within . Quantizing them in steps and accumulating the increments reconstructed the ground-truth heading sequences with an MAE of . This analysis provides training-sample coverage and quantization-error evidence for the adopted heading range.
The codec uses 401 uniformly spaced values including both endpoints, giving 400 intervals. A symmetric bound therefore yields a step of : 0.025 m for X residuals, 0.015 m for Y residuals, and for heading increments.
Reinforcement stages. Expert RL uses FP32, effective batch 128 (microbatch eight on each of 16 devices), and eight stochastic candidates per scene. Each epoch has 724 updates, with 70 real scenes in the final batch. A five-epoch stage therefore has 3,620 updates, 463,070 scene visits, and 3,704,560 stochastic candidate evaluations. The scene visits accumulate over five passes through the training set. Final heading adaptation additionally generates one detached greedy-baseline trajectory per scene visit.
Expert RL uses AdamW with learning rate , , , zero weight decay, constant learning rate, and gradient clipping at one. The optimizer and stage random stream are reset at stage boundaries; each stage retains its fixed policy reference. The backbone seed is 42 and the expert seed is 20261019. All stages use the listed fixed seeds.
Historical Selection and Statistical Interpretation
The historical expert system uses the fourth-epoch L1 backbone, a position endpoint internally named P11, and a heading endpoint named P18. Position-parent selection and the P14–P25 heading exploration consulted the small-test-set results from strategy ablations. The later dynamic and cross-attention branches fix their final-epoch weights before the full-split expansion, but inherit those recipe choices. R0 repeats the endpoint training with the same seed.
The full-split historical-minus-L1 interval and the dynamic-minus-historical interval answer different questions. The former concerns a fixed complete expert chain relative to its associated base policy. The latter concerns a new complete training recipe relative to the historical expert system. The cross-attention branch likewise compares complete systems with their respective training histories.
The reported dynamic-versus-historical interval is , taken from the accepted three-system report. A separate accepted R0 report gives for the same point difference under its own bootstrap run. Both intervals span zero. The intervals describe log-level resampling variability for fixed trained checkpoints.
For R1 and R2, the unadjusted intervals are and . The two prespecified comparisons use Bonferroni-adjusted 97.5% intervals of and , respectively.
Additional Results
This appendix collects historical system comparisons and exploratory safety objectives that supplement the main ablations.
Historical System Comparison
Table 10 and Figure 4 summarize the historical system comparisons. The historical expert system adds 2.821 PDMS over its corresponding L1 backbone on the full split, with a paired log-bootstrap 95% interval of . This comparison measures the performance gain of the complete expert chain.
Figure 4. Test-set performance of complete systems. (a) PDMS of each fixed system. (b) Paired log-bootstrap 95% intervals for differences from the historical expert system. Intervals are computed by resampling logs for fixed trained models.
| System | Test PDMS |
| L1 backbone, epoch 4 | 85.988 |
| Historical experts (H) | 88.810 |
| Cross-attention | 88.726 |
| Dynamic / R0 | 88.837 |
Table 10. Internal system comparison on the test set. H denotes the historical expert system.
The dynamic system reaches 88.837, the highest full-split point estimate among these internal systems. Its difference from the historical expert system is only +0.027, with a 95% interval of . The cross-attention system differs by , with an interval of . Both intervals cross zero. These branches use complete-system retraining with their respective initializations and training histories. The repeated R0 endpoint reproduces the dynamic score.
Controlled Safety-Penalty Interventions
We freeze the dynamic backbone and the relevant preliminary expert parents, and train an actual shared control R0. R1 modifies the final position reward by subtracting
with coefficient one, then applies the original heading-adaptation recipe to its own positions. R2 shares R0’s position policy and fixed XY, and subtracts the same cost from both sampled and greedy-baseline rewards in the heading stage. Other settings, including the official reference advantage RMS, remain fixed. R0 and R1 each train two stages; R2 trains one new heading stage, with the shared R0 position stage counted once. Together, these interventions add five stages and 18,100 updates.
Neither penalty variant improves full-split PDMS or mean risk cost (Table 11). R0, R1, and R2 have average costs 0.0567, 0.0575, and 0.0585, respectively. R2 reduces the number of scenes with NC below one from 204 to 201, but increases the number with DA below one from 497 to 522. Relative to R0, R1 and R2 introduce 75 and 61 new scenes whose NC–DA product becomes zero. Figure 5 reports both adjusted uncertainty and component counts.
| Variant | Val497 | Full PDMS | vs. R0 |
| R0: original recipe | 90.675 | 88.837 | — |
| R1: position penalty | 90.819 | 88.682 | −0.155 |
| R2: heading penalty | 90.949 | 88.655 | −0.182 |
| R1 interval: [−0.383, 0.078] | |||
| R2 interval: [−0.379, 0.014] |
Table 11. Controlled risk experiments. All use the same dynamic backbone. Intervals are Bonferroni-adjusted 97.5% paired log-bootstrap intervals for full-split differences from R0.
Figure 5. Planning scores and compliance under safety reward penalties. (a) Both full-split differences are negative in point estimate, with two-comparison-adjusted intervals crossing zero. (b) A small reduction in collision-compliance failures in R2 accompanies more road-compliance failures. Counts indicate components below one, including partial NC penalties.
The unadjusted 95% interval for R2 excludes zero, but the prespecified two-comparison correction yields an interval crossing zero. Both interventions yield lower observed mean PDMS than R0, with adjusted intervals spanning zero.
| Stage | Updated component | Objective / role | Fixed context | Updates | Val497 PDMS |
| A0 | Backbone | Dynamic weighted L1 + future features | Frozen visual tower | 3,625 | 87.611 |
| B1 | Heading | Token SFT | Backbone; base XY | 3,620 | 87.527 |
| B2 | Heading | First official-reward RL stage | Backbone; base XY | 3,620 | 89.581 |
| B3 | Heading | Second official-reward RL stage | Backbone; base XY | 3,620 | 90.126 |
| C1 | Position | Residual-token SFT | Backbone; B3 heading helper | 3,620 | 90.128 |
| C2 | Position | Official reward; global RMS | Backbone; B3 heading helper | 3,620 | 90.924 |
| C3 | Position | Safety-gated progress; | Backbone; B3 heading helper | 3,620 | 91.043 |
| D1 | Heading | Greedy baseline; RMS matching | Backbone; final C3 XY | 3,620 | 90.675 |
Table 9. Training lineage of the dynamic system. Every row is a five-epoch stage. Each stage retains its final-epoch checkpoint; the table reports the corresponding validation scores. R0 later repeats C3 and D1 from the recorded dynamic parents.
Training-Time Safety Weighting and Conditional Stopping
The original dynamic rule requires extra frozen-endpoint inference. We additionally test IL-C, which records detached training-forward predictions during epochs one through four and uses their NC/DA outcomes to fix the next epoch’s weights. A scene is difficult when predicted NC is lower than GT NC or predicted DA is lower than
GT DA. Difficult and other scoreable scenes receive raw weights two and one, followed by mean-one normalization; the 158 neutral scenes remain at weight one. IL-C reuses training-forward predictions for scoring. These predictions come from the successive model states encountered during optimization.
Before training, continuation to seven new expert stages was conditioned on both a validation PDMS of at least 87.611 and a strictly lower mean safety cost than the dynamic backbone. The new backbone completes five epochs but scores 87.534, a decline of 0.076 points. Both policies have an exact discrete cost sum of 30 over 497 validation scenes. Both average costs equal 30/497.
Under the prespecified continuation criteria, IL-C ended after the five-epoch backbone stage; its evaluation therefore covers this backbone endpoint.
Additional Safety Statistics
| Variant | NC < 1 | DA < 1 | Mean cost |
| R0 | 204 | 497 | 0.0567 |
| R1 | 208 | 502 | 0.0575 |
| R2 | 201 | 522 | 0.0585 |
Table 12. Safety costs and component counts on the full split. NC below one includes partial penalties.
The risk experiments evaluate complete systems, while IL-C evaluates the backbone endpoint described in Section C.3.
The two safety-focused research batches comprise five correction stages plus one IL-C backbone stage, or 21,725 updates.
Additional Qualitative Examples
Figure 6–Figure 8 extend the selected examples in § 5. They retain the same colors, footprint markers, and scene-level PDMS scale. The examples cover low-speed or stopped traffic, lane changes, turns, straight driving, and scenes labeled “Other” in the supplied grouping. The two “both good” examples illustrate unchanged maximum scores.

Figure 6. Additional low-speed and lane-change examples. The upper row includes one improvement case and two cases with unchanged maximum PDMS; the lower row shows lane-change improvements. For examples labeled “All waypoints outside camera view”, the trajectory positions are shown in the bird’s-eye-view panels.

Figure 8. Additional miscellaneous and straight-driving examples. The upper row retains the supplied “Other” category; the lower row shows straight-driving cases.

Figure 7. Additional turning examples. Right turns appear in the upper row and left turns in the lower row. Local enlargements highlight road-boundary and TTC-projection differences associated with the annotated improvements.
References
- [1]M. Assran, A. Bardes, D. Fan, Q. Garrido, R. Howes, M. Muckley, A. Rizvi, C. Roberts, K. Sinha, A. Zholus, et al. V-jepa 2: self-supervised video models enable understanding, prediction and planning. arXiv preprint arXiv:2506.09985, 2025. URL https://arxiv.org/abs/2506.09985.
- [2]S. Bai, Y. Cai, R. Chen, K. Chen, X. Chen, Z. Cheng, L. Deng, W. Ding, C. Gao, C. Ge, et al. Qwen3-vl technical report. arXiv preprint arXiv:2511.21631, 2025. URL https://arxiv.org/abs/2511.21631.
- [3]D. Dauner, M. Hallgarten, T. Li, X. Weng, Z. Huang, Z. Yang, H. Li, I. Gilitschenski, B. Ivanovic, M. Pavone, et al. Navsim: data-driven non-reactive autonomous vehicle simulation and benchmarking. Advances in Neural Information Processing Systems 37, pp. 28706–28719, 2024.arxiv.org/abs/2406.15349
- [4]Y. Hu, J. Yang, L. Chen, K. Li, C. Sima, X. Zhu, S. Chai, S. Du, T. Lin, W. Wang, et al. Planning-oriented autonomous driving. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 17853–17862, 2023.arxiv.org/abs/2212.10156
- [5]Minqing Huang, Yujiao Xiang, Zihan Liang, Jiajie Huang, Jingqi Wang, Yuheng Zhou, Zhi Xu, Feiyang Tan, Hangning Zhou, Mu Yang, and Gong Chen. CoWorld-VLA: Thinking in a Multi-Expert World Model for Autonomous Driving. arXiv preprint arXiv:2605.10426, 2026. URL https://arxiv.org/abs/2605.10426.
- [6]Y. Li, L. Fan, J. He, Y. Wang, Y. Chen, Z. Zhang, and T. Tan. Enhancing end-to-end autonomous driving with latent world model. arXiv preprint arXiv:2406.08481, 2024. URL https://arxiv.org/abs/2406.08481.
- [7]Y. Li, S. Shang, W. Liu, B. Zhan, H. Wang, Y. Wang, Y. Chen, X. Wang, Y. An, C. Tang, et al. DriveVLA-w0: world models amplify data scaling law in autonomous driving. arXiv preprint arXiv:2510.12796, 2025. URL https://arxiv.org/abs/2510.12796.
- [8]Y. Li, Y. Wang, Y. Liu, J. He, L. Fan, and Z. Zhang. End-to-end driving with online trajectory evaluation via bev world model. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 27137–27146, 2025.arxiv.org/abs/2504.01941
- [9]Y. Li, K. Xiong, X. Guo, F. Li, S. Yan, G. Xu, L. Zhou, L. Chen, H. Sun, B. Wang, et al. Recogdrive: a reinforced cognitive framework for end-to-end autonomous driving. arXiv preprint arXiv:2506.08052, 2025. URL https://arxiv.org/abs/2506.08052.
- [10]Z. Li, K. Li, S. Wang, S. Lan, Z. Yu, Y. Ji, Z. Li, Z. Zhu, J. Kautz, Z. Wu, et al. Hydra-mdp: end-to-end multimodal planning with multi-target hydra-distillation. arXiv preprint arXiv:2406.06978, 2024. URL https://arxiv.org/abs/2406.06978.
- [11]B. Liao, S. Chen, H. Yin, B. Jiang, C. Wang, S. Yan, X. Zhang, X. Li, Y. Zhang, Q. Zhang, et al. Diffusiondrive: truncated diffusion model for end-to-end autonomous driving. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 12037–12047, 2025.arxiv.org/abs/2411.15139
- [12]Q. Liu, H. Xu, J. Li, B. Sun, Z. Hao, D. She, X. Zhu, and L. Zhang. Uni-world vla: interleaved world modeling and planning for autonomous driving. arXiv preprint arXiv:2603.27287, 2026. URL https://arxiv.org/abs/2603.27287.
- [13]Steven J. Rennie, Etienne Marcheret, Youssef Mroueh, Jerret Ross, and Vaibhava Goel. Self-Critical Sequence Training for Image Captioning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017. URL https://openaccess.thecvf.com/content_cvpr_2017/html/Rennie_Self-Critical_Sequence_Training_CVPR_2017_paper.html.arxiv.org/abs/1612.00563
- [14]John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal Policy Optimization Algorithms. arXiv preprint arXiv:1707.06347, 2017. URL https://arxiv.org/abs/1707.06347.
- [15]Z. Shao, P. Wang, Q. Zhu, R. Xu, J. Song, X. Bi, H. Zhang, M. Zhang, Y. Li, Y. Wu, et al. Deepseek-math: pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300, 2024. URL https://arxiv.org/abs/2402.03300.
- [16]T. Xia, Y. Li, L. Zhou, J. Yao, K. Xiong, H. Sun, B. Wang, K. Ma, G. Chen, H. Ye, et al. Drivelaw: unifying planning and video generation in a latent driving world. arXiv preprint arXiv:2512.23421, 2025. URL https://arxiv.org/abs/2512.23421.
- [17]J. Zhang, Z. Fu, Z. Xu, W. Dai, Q. Liu, and Y. Wang. ResWorld: temporal residual world model for end-to-end autonomous driving. arXiv preprint arXiv:2602.10884, 2026. URL https://arxiv.org/abs/2602.10884.
- [18]K. Zhang, Z. Tang, X. Hu, X. Pan, X. Guo, Y. Liu, J. Huang, L. Yuan, Q. Zhang, X. Long, et al. Epona: autoregressive diffusion world model for autonomous driving. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 27220–27230, 2025.arxiv.org/abs/2506.24113
- [19]Z. Zhao, T. Fu, Y. Wang, L. Wang, and H. Lu. From forecasting to planning: policy world model for collaborative state-action prediction. arXiv preprint arXiv:2510.19654, 2025. URL https://arxiv.org/abs/2510.19654.
- [20]Zewei Zhou, Tianhui Cai, Seth Z. Zhao, Yun Zhang, Zhiyu Huang, Bolei Zhou, and Jiaqi Ma. AutoVLA: A Vision-Language-Action Model for End-to-End Autonomous Driving with Adaptive Reasoning and Reinforcement Fine-Tuning. Advances in Neural Information Processing Systems, 2025. URL https://arxiv.org/abs/2506.13757.

