betaFeedback

WorksComputer Science

Recovered or Memorized? Investigating Dynamical Consistency and Faithfulness in World Model Latents

Abstract

World models develop internal simulators that can learn the structure and dynamics of an environment. It has been demonstrated that their internal representations are both meaningful and causal, but are these representations faithful to the governing dynamics of the environment they are learning? We introduce Anchored Recovery Certification (ARC), a procedure that utilizes a discrete Fréchet distance calculation with ground truth to evaluate dynamical consistency on candidate world models up to an affine gauge. We use the procedure on TD-MPC2 (decoder-free world model) that has learned the pendulum swing on an observation lift under varying hyperparameters at k = 5, 10, 25 steps. The model is then steered within a PCA-derived intervention subspace and compared against a ground-truth simulator using relative error to investigate dynamical faithfulness. ARC sufficiency held for TD-MPC2 most consistently at k = 10, with six of seven constraints statistically significant after Holm-Bonferroni correction, while the relative error between the simulator and the causal latent remained substantial. Further research could be conducted with decoder world models such as DreamerV3 (reconstruction loss) to investigate whether the results differ.

Code: https://github.com/jfran03/dynamical-investigation

Introduction

Recent work suggests that world models are capable of developing linearly decodable representations of their training environments and are causal. Zhang [9] was able to demonstrate this using linear and nonlinear probing, causal interventions, and attention analysis on IRIS (discrete token transformer) and DIAMOND (continuous diffusion UNet), trained on Atari Breakout and Pong. Although causal representations are essential for understanding how the world model behaves, how do we know if these internal representations are not just memorizing trajectories, but rather learning the governing dynamics of its environment to produce its expected behavior?

World models support prediction, planning, and control in learned environments [10, 3]. We evaluate whether their representations support trajectory-consistent predictions and intervention responses in a controlled pendulum swing-up environment.

This paper proposes Anchored Recovery Certification (ARC), a procedure that uses a discrete Fréchet distance calculation [2] with a ground truth to determine if a world model exhibits evidence of dynamical consistency that cannot be explained by a memorizer. The procedure can be intuitively described as follows: A ruler is developed by creating the ceiling and the floor scored by the same Fréchet calculation to measure a candidate world model. The ceiling is defined to be a model that is an exact replication of the ground truth, whereas the floor is described to be a model that behaves poorly based on the ground truth. To give the ruler markings, perturbations are placed along the ruler, providing a reference for candidate models. Using this ruler, we can measure the dynamical performance of different candidate models, namely a trained world model and a memorizer. If a trained world model were to outperform the memorizer under the same conditions, we can further investigate the trained world models by steering their causal latent to check if it contains the same governing dynamics the true environment has.

We used this procedure using TD-MPC2 [5], trained in a pendulum swing environment with an observation lift. Once the model has finished training, an affine gauge is applied during the Fréchet calculation. DMControl [8] is used as the simulator to define the pendulum swing’s ground truth.

Method

Observation Lift & Affine Gauge

The observation lift is employed to ensure that the procedure can properly assess the dynamical consistency of a trained world model. It is a randomly initialized, frozen MLP with architecture 3→64→323 \to64 \to32 and tanh activations, sampled once and used for all training, calibration, and evaluation data. This gives a memorizer enough structure to be used in the procedure without affecting the score of other models. [6]

We use an affine gauge g(z)=Wz+bg(z) = Wz + b to map latent state zz into the lifted observation space. For each seed, (W,b)(W, b) are fitted by least squares on a held out calibration set A\mathcal{A}, disjoint from training data and evaluation rollouts.

Constructing Anchored Recovery Certification (ARC)

ARC is constructed using three different types of measurements which need to undergo the same calculations as the candidate world model: The ceiling, floor, and perturbation ladder.

The discrete Fréchet distance calculation can be intuitively defined as a man walking a dog on a leash, both are traveling two different curves at two different speeds, we calculate the shortest length of the leash such that both the man and dog are still traversing through their paths without backtracking. The similarity between two curves is calculated and will not punish trained models that may have dynamical consistency to its ground truth, but potentially at a different speed and time. [2]

We convert discrete Fréchet distance into a Fréchet Skill Score (FSS) [7]. For each scoring window w∈Ww \in\mathcal{W}, let τ^cand(w)\hat{\tau}_{\mathrm{cand}}^{(w)} denote the candidate trajectory and let τtrue(w)\tau_{\mathrm{true}}^{(w)} denote the corresponding ground-truth trajectory. We define FSS as

FSScand=1−∑w∈WdF(τ^cand(w),τtrue(w))∑w∈Wmax⁡ϵ∈{0,…,k}∥τtrue,t(w)−τtrue,0(w)∥2,(1)\mathrm{FSS}_{\mathrm{cand}} = 1 - \frac{\sum_{w \in\mathcal{W}} d_F\left(\hat{\tau}_{\mathrm{cand}}^{(w)}, \tau_{\mathrm{true}}^{(w)}\right)}{\sum_{w \in\mathcal{W}} \max_{\epsilon\in\{0,\ldots,k\}} \left\lVert\tau_{\mathrm{true},t}^{(w)} - \tau_{\mathrm{true},0}^{(w)} \right\rVert_2}, \tag*{(1)}

where dFd_F is the discrete Fréchet distance. An FSS of 1 indicates zero Fréchet distance from the ground-truth trajectory. An FSS of 0 indicates that the aggregate trajectory error equals the aggregate ground-truth displacement over the scoring windows, and negative values indicate larger error. We report FSS at horizons k∈{5,10,25}k \in\{5, 10, 25\}. Because the denominator in Eq. (1) is itself the discrete Fréchet distance between the true trajectory and a trajectory held fixed at the initial state, a null that predicts no motion (Persistence) yields FSS = 0 identically, by construction rather than by measurement.

The ceiling is computed through DMControl [8], which is rolled forward kk steps using the true dynamics. The simulator will contain the observation lift and its own affine gauge to ensure a controlled measurement is taken. Because the true operators are being scored, this system will provide the highest possible FSS.

All nulls use the trained model’s initial states, action sequences, rollout horizons, and scoring windows. The replay buffer is partitioned before evaluation into 25% alignment data and 75% probe data. The Memorizer retrieves the Euclidean nearest neighbor in the lifted observation space from the alignment split and iterates its retrieved output. The Inverse Memorizer retrieves the Euclidean nearest neighbor in autoencoder latent space and iterates its decoded output. The Untrained World Model uses the trained-model architecture with random, untrained weights. Persistence repeats the initial observation. Null selection is performed on the alignment split, selecting the null with the highest FSS score; final null and model scores are computed only on the probe split. The floor for a given constraint and horizon is therefore defined as the maximum FSS attained among these four nulls; because the FSS of Persistence is fixed at 0 by the normalization in Eq. (1) rather than estimated from data, a floor equal to 0 indicates that the three estimated nulls (Memorizer, Inverse Memorizer, Untrained World Model) all fell below this fixed value, not that Persistence was the strongest competitor.

The perturbation ladder is developed by perturbing each term on the angular acceleration formula (Eq. (2)):

θ¨=−ω2sin⁡θ−γθ˙+κu(2)\ddot{\theta} = -\omega^2 \sin\theta- \gamma\dot{\theta} + \kappa u \tag*{(2)}

For the perturbation ladder, we evaluate 87 perturbation draws: one unperturbed reference, 54 coefficient perturbations (3ω×2γ×9κ3\omega\times2\gamma\times9\kappa), and 32 random direction ladder draws (4ϵ4\epsilon levels ×8\times8 directions). Reported scores are averaged across these draws. We use η=(2,1,1)\eta= (2, 1, 1) so that perturbations to gravity have an effect comparable in scale to those of damping and control, which are weighted equally.

cipert=citrue⋅exp⁡(ηi⋅ϵ⋅ui),η=(2,1,1), ∥u∥2=1(3)c_i^{\mathrm{pert}} = c_i^{\mathrm{true}} \cdot\exp(\eta_i \cdot\epsilon\cdot u_i), \qquad\eta= (2, 1, 1), \ \lVert u\rVert_2 = 1 \tag*{(3)}

World Model

TD-MPC2 [5] is trained on the pendulum swing environment using the lifted observation state. The model is swept on three constraining hyperparameters to determine whether dynamical consistency would emerge only in certain conditions.

Constraint(low / mid / high)
Horizon (rollout length)1 / 3 / 5
Latent Dim (bottleneck)128 / 512 / 1024
Data Augmentation (σ\sigma)0 / 0.05 / 0.15

Table 1. Constraint Hyperparameters of TD-MPC2

We train 35 models across seven constraint conditions and five random seeds per condition. The baseline uses median constraint values, while the remaining conditions vary one constraint at a time. To align latent states with observations, we fit an affine map g(z)=Wz+bg(z) = Wz + b by least squares on a subset A\mathcal{A} of off-support states [6]. We construct A\mathcal{A} in the simulator by pushing angular velocity beyond the first 99th-percentile band reachable under random actions. A state is retained when its Euclidean distance to its nearest replay-buffer state exceeds the fixed threshold of 1.0; all states in A\mathcal{A} are excluded from training and reported evaluation rollouts.

W∗,b∗=arg⁡min⁡W,b∑i∈A∥Wzi+b−yi∥22.(4)W^*, b^* = \underset{W,b}{\arg\min} \sum_{i \in\mathcal{A}} \lVert Wz_i + b - y_i\rVert_2^2. \tag*{(4)}

The fitted map is used only to project latent rollouts into the lifted observation space for evaluation.

Causal Intervention & Faithfulness Protocol

Due to TD-MPC2’s SimNorm latent architecture [5], we perform principal component analysis (PCA) on encoder state-Jacobians [1] evaluated on states in A\mathcal{A}. For each run, subspace S\mathcal{S} is the minimum set of principal components explaining at least 99% of the variance; its dimension and explained variance are recorded per run. For each off-support probe, we patch the component of the encoder latent state projected onto S\mathcal{S} before the model rollout and evaluate the resulting trajectory at k∈{5,10,25}k \in\{5, 10, 25\}.

We test faithfulness using randomly sampled off-support probes projected into S\mathcal{S}; intervention magnitudes are therefore determined by the probes rather than fixed globally. We compare the gauge-projected model response with the simulator response over k∈{5,10,25}k \in\{5, 10, 25\} steps:

RelErr⁡k=∥Δy^k−Δyk∥2∥Δyk∥2+δ(5)\operatorname{RelErr}_{k} = \frac{\lVert\Delta\hat{y}_{k} - \Delta y_{k} \rVert_{2}}{\lVert\Delta y_{k} \rVert_{2} + \delta} \tag*{(5)}

where Δy^k\Delta\hat{y}_{k} and Δyk\Delta y_{k} are the model and simulator responses, respectively, and δ>0\delta> 0 is fixed.

Results

ARC Sufficiency

By calculating the Fréchet Skill Score (FSS) of the ceiling and the floor of a given constraint, we are able to highlight the maximum and minimum possible scores a model can attain. The FSS for the ceiling will always be 1, and Table 3 indicates the floor FSS for a given constraint. A trained world model under the respective constraint must attain an FSS score between the ceiling and the floor to be considered ARC-sufficient.

Table 3: FSS on TD-MPC2 Constraint Floors, based on a given constraint and kk step horizon, with five seeds per constraint. FSS values at k={5,10}k = \{5, 10\} are memorizer floors, whereas the FSS values at k=25k = 25 are persistence floors. Values are means across five seeds; uncertainties denote one standard error of the mean.

Constraintk=5k = 5k=10k = 10k=25k = 25
Baseline Null0.465±0.02880.465 \pm0.02880.277±0.03350.277 \pm0.03350.0000±0.0000∗0.0000 \pm0.0000^{*}
Short Horizon Null0.510±0.01770.510 \pm0.01770.297±0.03650.297 \pm0.03650.0000±0.0000∗0.0000 \pm0.0000^{*}
Long Horizon Null0.446±0.02850.446 \pm0.02850.253±0.03880.253 \pm0.03880.0000±0.0000∗0.0000 \pm0.0000^{*}
Narrow Latent Dim Null0.477±0.02760.477 \pm0.02760.269±0.04240.269 \pm0.04240.0000±0.0000∗0.0000 \pm0.0000^{*}
Wide Latent Dim Null0.520±0.02390.520 \pm0.02390.325±0.03480.325 \pm0.03480.0000±0.0000∗0.0000 \pm0.0000^{*}
No Data Augmentation Null0.495±0.02810.495 \pm0.02810.269±0.05840.269 \pm0.05840.0000±0.0000∗0.0000 \pm0.0000^{*}
High Data Augmentation Null0.325±0.02840.325 \pm0.02840.114±0.04530.114 \pm0.04530.0000±0.0000∗0.0000 \pm0.0000^{*}
Table 3.
  • The three estimated nulls scored below 0, this is Persistence and not a measured floor

Table 2 compares mean trained-model FSS with the corresponding null floors in Table 3. At k=5k = 5, six of seven constraints exceed their selected floor on average, though a paired significance test (Table 4) confirms this for only two of seven (Baseline, Narrow Latent Dim) after Holm–Bonferroni correction. All seven constraints exceed their selected floor on average at k=10k = 10, and six of seven remain significant after correction. At k=25k = 25, only Wide Latent Dim and Long Horizon exceed their listed floors on average, but neither difference is statistically significant (Table 4); at this horizon the Memorizer, Inverse Memorizer, and Untrained World Model all score below the fixed value of Persistence, whose FSS is 0 by construction rather than by measurement (Eq. (1)). The listed floor at k=25k = 25 is therefore not an empirical measurement, and ARC sufficiency cannot be meaningfully certified against it at this horizon. Thus, ARC sufficiency is strongest and most reliably supported at k=10k = 10 (six of seven constraints significant), partially supported at k=5k = 5 (two of seven), and statistically unsupported at k=25k = 25 (zero of seven) despite two constraints appearing to exceed the floor on raw means. These results indicate that the trained models can produce trajectories that are more consistent with the ground-truth simulator than the selected null models under the conditions in which they significantly exceed the corresponding floor. Because discrete Fréchet distance permits monotone temporal reparameterization, this result should be interpreted as evidence of trajectory consistency rather than direct recovery of the pendulum’s governing operators.

Constraintk=5k=10k=25
Baseline0.6 ±0.04200.577 ±0.0414-0.0132 ±0.110
Short Horizon0.615 ±0.03870.54 ±0.0435-0.421 ±0.0846
Long Horizon0.539 ±0.05340.523 ±0.05680.0682 ±0.0848
Narrow Latent Dim0.744 ±0.05900.642 ±0.0237-0.105 ±0.0489
Wide Latent Dim0.571 ±0.02370.581 ±0.02240.106 ±0.104
No Data Augmentation0.43 ±0.1100.403 ±0.141-0.171 ±0.219
High Data Augmentation0.445 ±0.07500.379 ±0.0704-0.564 ±0.130

Table 2. FSS on TD-MPC2 Constraints, based on a given constraint and kk step horizon, with five seeds per constraint. However, the trained world models at k=25k = 25 steps seems to fail in learning the pendulum swing task overall. Values are means across five seeds; uncertainties denote one standard error of the mean.

Table 4: Paired one-sided tt-test of the mean FSS difference between each TD-MPC2 constraint and its floor, based on a given constraint and kk step horizon, with five paired seeds per constraint. Significance reflects a Holm-Bonferroni correction applied across all 21 tests (7 constraints ×\times 3 horizons). At k={5,10}k = \{5, 10\}, the floor is the maximum FSS attained among the estimated nulls (Table 3); at k=25k = 25, the floor is the persistence floor, which is zero by construction rather than measured. Values are mean paired differences (Δ\Delta) with 95% confidence intervals.

Constraintk=5k = 5k=10k = 10k=25†k = 25^{\dagger}
Baseline0.135 [0.074,0.195]∗0.135\ [0.074, 0.195]^{*}0.300 [0.219,0.381]∗0.300\ [0.219, 0.381]^{*}−0.013 [−0.320,0.293]-0.013\ [-0.320, 0.293]
Short Horizon0.105 [0.035,0.174]0.105\ [0.035, 0.174]0.243 [0.140,0.345]∗0.243\ [0.140, 0.345]^{*}−0.421 [−0.656,−0.186]-0.421\ [-0.656, -0.186]
Long Horizon0.093 [0.022,0.164]0.093\ [0.022, 0.164]0.270 [0.220,0.321]∗0.270\ [0.220, 0.321]^{*}0.068 [−0.167,0.304]0.068\ [-0.167, 0.304]
Narrow Latent Dim0.267 [0.129,0.404]∗0.267\ [0.129, 0.404]^{*}0.372 [0.268,0.477]∗0.372\ [0.268, 0.477]^{*}−0.105 [−0.241,0.031]-0.105\ [-0.241, 0.031]
Wide Latent Dim0.051 [−0.031,0.134]0.051\ [-0.031, 0.134]0.256 [0.186,0.327]∗0.256\ [0.186, 0.327]^{*}0.105 [−0.185,0.394]0.105\ [-0.185, 0.394]
No Data Augmentation−0.064 [−0.374,0.246]-0.064\ [-0.374, 0.246]0.134 [−0.271,0.539]0.134\ [-0.271, 0.539]−0.017 [−0.624,0.590]-0.017\ [-0.624, 0.590]
High Data Augmentation0.124 [−0.055,0.303]0.124\ [-0.055, 0.303]0.264 [0.144,0.384]∗0.264\ [0.144, 0.384]^{*}−0.564 [−0.927,−0.201]-0.564\ [-0.927, -0.201]
Table 4.
  • significant after Holm–Bonferroni correction across all 21 tests (α=0.05\alpha= 0.05, one-sided). † k=25k = 25 tests the model against the fixed value 0 (Persistence floor), not an estimated floor.

Table 5: FSS for the TD-MPC2 perturbation ladder at k=10k = 10. The ladder evaluates ground-truth simulators with controlled perturbations to the pendulum-dynamics coefficients and provides a diagnostic reference for interpreting the magnitude of trained-model FSS values. Similar FSS values do not imply recovery of a corresponding fraction of the governing dynamics. Values are means across five seeds; uncertainties denote one standard error of the mean.

Constraintϵ=0.05\epsilon= 0.05ϵ=0.1\epsilon= 0.1ϵ=0.2\epsilon= 0.2ϵ=0.5\epsilon= 0.5
Baseline0.945±0.004000.945 \pm0.004000.896±0.006480.896 \pm0.006480.810±0.01200.810 \pm0.01200.589±0.05510.589 \pm0.0551
Short Horizon0.945±0.003880.945 \pm0.003880.897±0.006380.897 \pm0.006380.812±0.01210.812 \pm0.01210.595±0.05410.595 \pm0.0541
Long Horizon0.944±0.004680.944 \pm0.004680.895±0.007450.895 \pm0.007450.808±0.01340.808 \pm0.01340.586±0.05510.586 \pm0.0551
Narrow Latent Dim0.945±0.003520.945 \pm0.003520.896±0.005740.896 \pm0.005740.810±0.01220.810 \pm0.01220.591±0.05720.591 \pm0.0572
Wide Latent Dim0.944±0.003920.944 \pm0.003920.894±0.006370.894 \pm0.006370.808±0.01220.808 \pm0.01220.590±0.05410.590 \pm0.0541
No Data Augmentation0.944±0.004470.944 \pm0.004470.895±0.007310.895 \pm0.007310.809±0.01350.809 \pm0.01350.588±0.05560.588 \pm0.0556
High Data Augmentation0.942±0.005040.942 \pm0.005040.891±0.008160.891 \pm0.008160.802±0.01450.802 \pm0.01450.581±0.05500.581 \pm0.0550

Table 5.

Steering Subspace Faithfulness

Table 6 shows the relative error based on the response of a state nudge between the trained world model (predicted value) and a pendulum with the true governing dynamics (expected value). For reference, model where a nudge would have no effect on its outcome (worse case) would evaluate a relative error of 1.

Constraintk=5k = 5k=10k = 10k=25k = 25
Baseline1.001.612.09
Short Horizon0.941.502.27
Long Horizon1.051.571.95
Narrow Latent Dim0.921.381.93
Wide Latent Dim0.891.421.89
No Data Augmentation0.841.291.73
High Data Augmentation1.031.452.03

Table 6. Average relative error between the gauge-predicted model response and the ground-truth simulator response after interventions within subspace SS, evaluated on off-support states. Values are means across five seeds.

Results show that more often than not, at all kk steps, the average relative error for all constraints of a trained world model is greater than 1, meaning that a predicted response to a state nudge is worse than predicting no response at all. At k=10k = 10 and k=25k = 25, the mean relative error exceeds 1 for all evaluated constraints. This indicates that the tested interventions do not reliably reproduce the simulator response at these horizons. The findings are conditional on the affine gauge, PCA-derived subspace, and activation patching protocol. They do not exclude the possibility that relevant dynamics are encoded nonlinearly or outside the selected subspace.

Discussion and Conclusion

Our results show that TD-MPC2 exhibits trajectory consistency with the pendulum simulator under some constraints and horizons, most consistently at k=10k = 10. However, the intervention results do not establish that its internal representation faithfully recovers the pendulum’s governing operators.

The decoder-free architecture of TD-MPC2 [5] may be the cause of the results. The model would predict its next latent against its encoder’s embedding of the true next observation as opposed to reconstruction loss.

Limitations Our paper has not investigated whether a similar result would occur with world models containing decoders such as DreamerV3, [4] which are trained using reconstruction loss. A comparison of two different world models would strengthen the investigation.

Another limitation comes from the affine gauge used. If TD-MPC2 learned the governing dynamics of the pendulum swing nonlinearly, we may have observed that TD-MPC2 contains lower FSS scores than intended. This was an accepted limitation, mitigated as much as possible, to properly convert TD-MPC2’s latent into a predicted observation.

Societal Impacts This approach can advance the scientific understanding of the internal representations in world models, enabling transparency, alignment, and bias detection with respect to the dynamical consistency and faithfulness of a world model. Insight into a model’s latent, however, can also enable malicious manipulation, misinformation, or falsified confidence in tools relying on world models. Advisory from ethicists, social scientists, and policy experts are recommended suggestions to take.

Detailed Per-Seed Results and Null Baselines

ConstraintSeed 1Seed 2Seed 3Seed 4Seed 5Uncertainty
Baseline0.4990.7270.5610.5460.665±0.0418\pm0.0418
Short Horizon0.6320.5320.7150.5190.675±0.0387\pm0.0387
Long Horizon0.410.4470.5140.6810.645±0.0534\pm0.0534
Narrow Latent Dim0.6710.7380.9620.6150.733±0.0590\pm0.0590
Wide Latent Dim0.5810.6260.6150.5280.505±0.0237\pm0.0237
No Data Augmentation0.0270.5910.460.4210.657±0.1097\pm0.1097
High Data Augmentation0.4560.5870.3390.2320.634±0.0750\pm0.0750

Table 7. Fréchet Skill Score on TD-MPC2 Constraints k=5k = 5. Values are means across five seeds; uncertainties denote one standard error of the mean.

ConstraintSeed 1Seed 2Seed 3Seed 4Seed 5Uncertainty
Baseline0.4920.6840.5220.5170.671±0.0414\pm0.0414
Short Horizon0.5410.5210.640.3890.609±0.0435\pm0.0435
Long Horizon0.3860.4270.4950.6810.627±0.0568\pm0.0568
Narrow Latent Dim0.6080.670.6580.5680.702±0.0237\pm0.0237
Wide Latent Dim0.5730.5920.6580.5210.562±0.0224\pm0.0224
No Data Augmentation-0.1050.5770.4750.3490.717±0.141\pm0.141
High Data Augmentation0.3360.5090.2540.2180.576±0.0704\pm0.0704

Table 8. Fréchet Skill Score on TD-MPC2 Constraints k=10k=10. Values are means across five seeds; uncertainties denote one standard error of the mean.

ConstraintSeed 1Seed 2Seed 3Seed 4Seed 5Uncertainty
Baseline0.145-0.362-0.003-0.1240.278±0.1104\pm0.1104
Short Horizon-0.581-0.32-0.388-0.638-0.178±0.0846\pm0.0846
Long Horizon-0.0746-0.1620.1790.3080.0907±0.0848\pm0.0848
Narrow Latent Dim-0.0566-0.0701-0.00652-0.291-0.102±0.0489\pm0.0489
Wide Latent Dim0.1450.09990.195-0.2740.357±0.1041\pm0.1041
No Data Augmentation-0.6710.287-0.0225-0.2710.592±0.2186\pm0.2186
High Data Augmentation-0.763-0.304-0.755-0.806-0.191±0.1306\pm0.1306

Table 9. Fréchet Skill Score on TD-MPC2 Constraints k=25k = 25. Values are means across five seeds; uncertainties denote one standard error of the mean.

ConstraintSeed 1Seed 2Seed 3Seed 4Seed 5Uncertainty
Baseline Null0.4120.5310.3930.4590.53±0.0288\pm0.0288
Short Horizon Null0.4910.4640.5630.4950.537±0.0177\pm0.0177
Long Horizon Null0.3890.3940.4180.5240.505±0.0285\pm0.0285
Narrow Latent Dim Null0.3820.4690.5270.4710.537±0.0276\pm0.0276
Wide Latent Dim Null0.4420.5230.5930.5150.525±0.0239\pm0.0239
No Data Augmentation Null0.5170.4920.3950.5050.567±0.0281\pm0.0281
High Data Augmentation Null0.2450.3520.2890.3280.413±0.0284\pm0.0284

Table 10. Fréchet Skill Score on TD-MPC2 Constraint Floors k=5k = 5. Values are means across five seeds; uncertainties denote one standard error of the mean.

ConstraintSeed 1Seed 2Seed 3Seed 4Seed 5Uncertainty
Baseline0.2040.2890.2060.3030.383±0.0335\pm0.0335
Short Horizon0.2780.1810.3840.2770.367±0.0365\pm0.0365
Long Horizon0.1640.1850.2290.3620.324±0.0388\pm0.0388
Narrow Latent Dim0.1060.2770.3230.2930.345±0.0424\pm0.0424
Wide Latent Dim0.2260.3220.4310.2830.363±0.0348\pm0.0348
No Data Augmentation0.2880.2920.05070.310.403±0.0584\pm0.0584
High Data Augmentation-2.22E-16*0.1530.08610.06580.267±0.0453\pm0.0453

Table 11. Fréchet Skill Score on TD-MPC2 Constraint Floors k=10k = 10. Values are means across five seeds; uncertainties denote one standard error of the mean.

  • Seed 1 for High Data Augmentation is a persistence floor

ConstraintSeed 1Seed 2Seed 3Seed 4Seed 5Uncertainty
Baseline Null00000±0.0000\pm0.0000
Short Horizon Null00000±0.0000\pm0.0000
Long Horizon Null00000±0.0000\pm0.0000
Narrow Latent Dim Null00000±0.0000\pm0.0000
Wide Latent Dim Null00000±0.0000\pm0.0000
No Data Augmentation Null00000±0.0000\pm0.0000
High Data Augmentation Null00000±0.0000\pm0.0000

Table 12. Fréchet Skill Score on TD-MPC2 Constraint Floors k=25k = 25. Values are means across five seeds; uncertainties denote one standard error of the mean.

ConstraintSeed 1Seed 2Seed 3Seed 4Seed 5Uncertainty
Baseline0.2990.3070.2480.2770.391±0.0240\pm0.0240
Short Horizon0.3170.2630.3730.2730.326±0.0199\pm0.0199
Long Horizon0.2640.2830.2670.3540.389±0.0254\pm0.0254
Narrow Latent Dim0.2610.3080.3150.290.364±0.0169\pm0.0169
Wide Latent Dim0.3230.3390.340.2180.391±0.0285\pm0.0285
No Data Augmentation0.3840.3610.230.3420.422±0.0323\pm0.0323
High Data Augmentation0.04760.180.1060.1910.235±0.0334\pm0.0334

Table 13. Inverse Memorizer null on TD-MPC2 constraints, k=5k = 5. Values are per-seed; uncertainties denote one standard error of the mean across five seeds.

ConstraintSeed 1Seed 2Seed 3Seed 4Seed 5Uncertainty
Baseline0.0727-0.0344-0.03810.1040.173±0.0408\pm0.0408
Short Horizon0.0708-0.06850.1470.01880.124±0.0387\pm0.0387
Long Horizon-0.01590.0143-0.01770.1850.226±0.0526\pm0.0526
Narrow Latent Dim-0.07260.0390.06240.08490.152±0.0367\pm0.0367
Wide Latent Dim0.07830.09930.0998-0.03840.232±0.0429\pm0.0429
No Data Augmentation0.1130.129-0.1960.1340.207±0.0701\pm0.0701
High Data Augmentation-0.391-0.0907-0.226-0.1540.00329±0.0665\pm0.0665

Table 14. Inverse Memorizer null on TD-MPC2 constraints, k=10k = 10. Values are per-seed; uncertainties denote one standard error of the mean across five seeds.

ConstraintSeed 1Seed 2Seed 3Seed 4Seed 5Uncertainty
Baseline-0.765-0.987-0.83-0.669-0.628±0.0636\pm0.0636
Short Horizon-0.817-0.945-0.817-0.886-0.578±0.0623\pm0.0623
Long Horizon-0.992-0.875-0.838-0.622-0.579±0.0783\pm0.0783
Narrow Latent Dim-1.191-0.848-0.829-0.719-0.554±0.1046\pm0.1046
Wide Latent Dim-0.888-0.768-0.783-0.845-0.513±0.0654\pm0.0654
No Data Augmentation-1.085-0.862-1.210-0.846-0.587±0.1076\pm0.1076
High Data Augmentation-1.269-0.853-0.992-1.236-0.682±0.1119\pm0.1119

Table 15. Inverse Memorizer null on TD-MPC2 constraints, k=25k = 25. Values are per-seed; uncertainties denote one standard error of the mean across five seeds.

ConstraintSeed 1Seed 2Seed 3Seed 4Seed 5Uncertainty
Baseline0.4120.5310.3940.460.531±0.0288\pm0.0288
Short Horizon0.4910.4640.5630.4950.537±0.0177\pm0.0177
Long Horizon0.3910.3940.420.5240.505±0.0282\pm0.0282
Narrow Latent Dim0.3820.4690.5270.4710.537±0.0276\pm0.0276
Wide Latent Dim0.4420.5230.5930.5160.525±0.0240\pm0.0240
No Data Augmentation0.5170.4920.3960.5060.567±0.0280\pm0.0280
High Data Augmentation0.2470.3530.290.3290.413±0.0282\pm0.0282

Table 16. Memorizer Max null on TD-MPC2 constraints, k=5k = 5. Values are per-seed; uncertainties denote one standard error of the mean across five seeds.

ConstraintSeed 1Seed 2Seed 3Seed 4Seed 5Uncertainty
Baseline0.2040.290.2070.3060.383±0.0334\pm0.0334
Short Horizon0.2790.1810.3840.2770.367±0.0365\pm0.0365
Long Horizon0.1680.1860.2320.3620.326±0.0383\pm0.0383
Narrow Latent Dim0.1080.2780.3230.2950.347±0.0422\pm0.0422
Wide Latent Dim0.2270.3230.4320.2840.363±0.0349\pm0.0349
No Data Augmentation0.2890.2930.05240.3120.403±0.0582\pm0.0582
High Data Augmentation-0.02550.1550.08620.0660.268±0.0489\pm0.0489

Table 17. Memorizer Max null on TD-MPC2 constraints, k=10k = 10. Values are per-seed; uncertainties denote one standard error of the mean across five seeds.

ConstraintSeed 1Seed 2Seed 3Seed 4Seed 5Uncertainty
Baseline-0.625-0.843-0.617-0.544-0.408±0.0705\pm0.0705
Short Horizon-0.763-0.868-0.629-0.821-0.468±0.0726\pm0.0726
Long Horizon-0.859-0.794-0.593-0.558-0.46±0.0750\pm0.0750
Narrow Latent Dim-1.063-0.59-0.714-0.651-0.399±0.1085\pm0.1085
Wide Latent Dim-0.759-0.605-0.574-0.872-0.381±0.0838\pm0.0838
No Data Augmentation-0.882-0.67-1.029-0.594-0.351±0.1173\pm0.1173
High Data Augmentation-1.067-0.733-0.792-1.076-0.513±0.1067\pm0.1067

Table 18. Memorizer Max null on TD-MPC2 constraints, k=25k = 25. Values are per-seed; uncertainties denote one standard error of the mean across five seeds.

ConstraintSeed 1Seed 2Seed 3Seed 4Seed 5Uncertainty
Baseline-2.22E-16-8.88E-16-4.44E-16-2.22E-16-4.44E-16±0.0000\pm0.0000
Short Horizon2.22E-16-4.44E-16-4.44E-16-6.66E-160±0.0000\pm0.0000
Long Horizon-6.66E-16-2.22E-16-6.66E-16-4.44E-16-2.22E-16±0.0000\pm0.0000
Narrow Latent Dim-2.22E-16-4.44E-16-2.22E-16-2.22E-16-4.44E-16±0.0000\pm0.0000
Wide Latent Dim0-2.22E-1602.22E-16-2.22E-16±0.0000\pm0.0000
No Data Augmentation-2.22E-16-2.22E-16-4.44E-16-4.44E-16-2.22E-16±0.0000\pm0.0000
High Data Augmentation-2.22E-16-2.22E-16-6.66E-16-6.66E-16-1.33E-15±0.0000\pm0.0000

Table 19. Persistence null on TD-MPC2 constraints, k=5k = 5. Values are per-seed; uncertainties denote one standard error of the mean across five seeds.

ConstraintSeed 1Seed 2Seed 3Seed 4Seed 5Uncertainty
Baseline0-8.88E-16-2.22E-16-4.44E-160±0.0000\pm0.0000
Short Horizon2.22E-16-6.66E-16-4.44E-16-4.44E-16-2.22E-16±0.0000\pm0.0000
Long Horizon-6.66E-16-4.44E-16-4.44E-16-4.44E-16-2.22E-16±0.0000\pm0.0000
Narrow Latent Dim-4.44E-16-4.44E-162.22E-16-4.44E-16-2.22E-16±0.0000\pm0.0000
Wide Latent Dim0003.33E-16-2.22E-16±0.0000\pm0.0000
No Data Augmentation-4.44E-16-4.44E-16-4.44E-16-4.44E-163.33E-16±0.0000\pm0.0000
High Data Augmentation-2.22E-16-2.22E-16-4.44E-16-4.44E-16-6.66E-16±0.0000\pm0.0000

Table 20. Persistence null on TD-MPC2 constraints, k=10k = 10. Values are per-seed; uncertainties denote one standard error of the mean across five seeds.

ConstraintSeed 1Seed 2Seed 3Seed 4Seed 5Uncertainty
Baseline2.22E-16-2.22E-160-2.22E-162.22E-16±0.0000\pm0.0000
Short Horizon2.22E-16-4.44E-16-2.22E-16-4.44E-16-2.22E-16±0.0000\pm0.0000
Long Horizon-6.66E-16-2.22E-160-4.44E-160±0.0000\pm0.0000
Narrow Latent Dim1.11E-16-2.22E-162.22E-16-2.22E-162.22E-16±0.0000\pm0.0000
Wide Latent Dim0-2.22E-161.11E-165.55E-16-4.44E-16±0.0000\pm0.0000
No Data Augmentation-2.22E-1600-2.22E-164.44E-16±0.0000\pm0.0000
High Data Augmentation-2.22E-16008.88E-164.44E-16±0.0000\pm0.0000

Table 21. Persistence null on TD-MPC2 constraints, k=25k = 25. Values are per-seed; uncertainties denote one standard error of the mean across five seeds.

ConstraintSeed 1Seed 2Seed 3Seed 4Seed 5Uncertainty
Baseline-5.399-6.135-5.823-5.719-5.458±0.1330\pm0.1330
Short Horizon-6.159-5.206-5.822-6.239-4.675±0.2982\pm0.2982
Long Horizon-5.169-5.265-5.734-5.463-4.878±0.1433\pm0.1433
Narrow Latent Dim-4.779-4.580-4.502-4.487-3.563±0.2112\pm0.2112
Wide Latent Dim-2.712-2.518-3.002-2.490-3.000±0.1115\pm0.1115
No Data Augmentation-5.523-5.334-6.451-5.902-4.870±0.2667\pm0.2667
High Data Augmentation-5.282-5.582-5.954-6.230-4.770±0.2555\pm0.2555

Table 22. Untrained Max null on TD-MPC2 constraints, k=5k = 5. Values are per-seed; uncertainties denote one standard error of the mean across five seeds.

ConstraintSeed 1Seed 2Seed 3Seed 4Seed 5Uncertainty
Baseline-3.449-3.985-3.738-3.829-3.343±0.1193\pm0.1193
Short Horizon-4.039-3.365-3.791-4.087-2.805±0.2401\pm0.2401
Long Horizon-3.438-3.341-3.632-3.503-2.932±0.1192\pm0.1192
Narrow Latent Dim-3.076-2.924-2.899-2.825-2.090±0.1731\pm0.1731
Wide Latent Dim-1.587-1.409-1.742-1.398-1.650±0.0674\pm0.0674
No Data Augmentation-3.541-3.375-4.187-3.814-2.868±0.2202\pm0.2202
High Data Augmentation-3.405-3.442-3.721-4.038-2.993±0.1741\pm0.1741

Table 23. Untrained Max null on TD-MPC2 constraints, k=10k = 10. Values are per-seed; uncertainties denote one standard error of the mean across five seeds.

ConstraintSeed 1Seed 2Seed 3Seed 4Seed 5Uncertainty
Baseline-2.886-3.285-3.069-3.268-2.552±0.1362\pm0.1362
Short Horizon-3.473-2.853-3.176-3.413-2.202±0.2325\pm0.2325
Long Horizon-3.064-2.857-2.907-2.930-2.210±0.1499\pm0.1499
Narrow Latent Dim-2.636-2.382-2.449-2.369-1.483±0.2010\pm0.2010
Wide Latent Dim-1.260-1.020-1.308-1.072-1.098±0.0560\pm0.0560
No Data Augmentation-2.982-2.830-3.339-3.220-2.077±0.2218\pm0.2218
High Data Augmentation-2.871-2.712-2.881-3.390-2.215±0.1881\pm0.1881

Table 24. Untrained Max null on TD-MPC2 constraints, k=25k = 25. Values are per-seed; uncertainties denote one standard error of the mean across five seeds.

References

  1. [1]Chunfeng Cui, Kaiqi Zhang, Talqat Daulbaev, Julia Gusak, Ivan V. Oseledets, and Zheng Zhang. Active subspace of neural networks: Structural analysis and universal attacks. CoRR, abs/1910.13025, 2019.arxiv.org/abs/1910.13025
  2. [2]Thomas Eiter and Heikki Mannila. Computing discrete frechet distance. 05 1994.
  3. [3]David Ha and Jürgen Schmidhuber. World models. 2018.DOI
  4. [4]Danijar Hafner, Jurgis Pasukonis, Jimmy Ba, and Timothy Lillicrap. Mastering diverse domains through world models, 2024.arxiv.org/abs/2301.04104
  5. [5]Nicklas Hansen, Hao Su, and Xiaolong Wang. TD-MPC2: Scalable, robust world models for continuous control. In The Twelfth International Conference on Learning Representations, 2024.arxiv.org/abs/2310.16828
  6. [6]Paolo Muratore and Mackenzie Weygandt Mathis. Extracting governing equations from latent dynamics via multi-view contrastive learning, 2026.arxiv.org/abs/2606.13260
  7. [7]Allan H. Murphy. Skill scores based on the mean square error and their relationships to the correlation coefficient. Monthly Weather Review, 116(12):2417 – 2424, 1988.doi.org/10.1175/1520-0493(1988)116<2417:ssbotm>2.0.co;2
  8. [8]Yuval Tassa, Yotam Doron, Alistair Muldal, Tom Erez, Yazhe Li, Diego de Las Casas, David Budden, Abbas Abdolmaleki, Josh Merel, Andrew Lefrancq, Timothy Lillicrap, and Martin Riedmiller. Deepmind control suite, 2018.arxiv.org/abs/1801.00690
  9. [9]Xinyu Zhang. What do world models learn in rl? probing latent representations in learned environment simulators, 2026.arxiv.org/abs/2603.21546
  10. [10]Arif Hassan Zidan, Yi Pan, Hanqi Jiang, Ruiyu Yan, Wei Ruan, Zihao Wu, Lifeng Chen, Weihang You, Xinliang Li, Bowen Chen, Huawen Hu, Peilong Wang, Sizhuang Liu, Jing Zhang, Siyuan Li, Zhengliang Liu, Yu Bao, Lin Zhao, Lichao Sun, Dajiang Zhu, Xiang Li, Jinglei Lv, Quanzheng Li, Wei Liu, Tianming Liu, and Wei Zhang. World models: A comprehensive survey of architectures, methodologies, reasoning paradigms, and applications, 2026.

Paper details

vlnc:W_99xyhvff
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 6

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents