WorksComputer Science
Operator-Mismatch Contrastive Learning for World-Relative Trajectory Coherence
Abstract
World models are increasingly judged by how well they rank plausible futures, yet plausibility alone is the wrong target. The same scene can evolve under many latent rules, and a future that is smooth and physically valid under one rule can be incoherent under the rule actually in force. We formalize the resulting failure, operator-blindness, and show that it is systematically hidden by standard evaluation: a scorer can attain near-perfect generic ranking while separating current-world from other-world futures no better than chance. We introduce Operator-Mismatch Contrastive (OMC) learning, whose semantic hard negatives are valid-but-wrong-operator trajectories, and a matching protocol that measures world-relative coherence directly. On a controlled synthetic benchmark with in- and out-of-distribution operator parameters, across two trajectory-scoring backbones, and on feature-level external validation with CoPhy ballsCF, OMC raises operator-sensitive scoring far above a chance baseline that nonetheless achieves 0.99 generic top-1. We prove that the wrong-operator negative is necessary both for the protocol to detect the failure and for the objective to fix it, and position OMC as an auxiliary coherence signal (a trajectory-coherence scorer, not a generative model, planner, or control agent) that can be attached to existing world-model representations.
Introduction
A world model should anticipate futures, and we usually grade it by whether it ranks a true continuation above corrupted or random alternatives. This target—generic plausibility—is necessary but quietly insufficient. The same multi-object scene can evolve under many latent rules (gravity, damping, attraction, restitution), and a future that is perfectly plausible under one rule can be incoherent under the rule actually governing the present context. A world model must therefore answer not only “is this future plausible?” but “under which world is it coherent?”
The distinction mirrors an asymmetry in human cognition. People reason about what is factually plausible, but they also imagine freely—recombining familiar elements into scenes that need not match the present world, yet remain internally consistent under different rules [1, 2, 24, 25]. A capable world model arguably needs both faculties: the generativity to entertain plausible-elsewhere futures and the discernment to recognize which of them is coherent here. This paper isolates the second, measurable faculty—world-relative coherence—and treats free generativity as motivation rather than claim.
Where should coherence come from? Contemporary world models acquire physical regularities in two broad ways. One line learns them implicitly from data: joint-embedding predictive architectures acquire intuitive physics from natural video without hardwired core knowledge [5, 6, 23], and latent-dynamics models learn dynamics from interaction [3, 4]. A complementary line imposes physical structure explicitly via physics-informed losses or conservation-law architectures [7, 8, 9]. Our contribution is orthogonal to this axis: whether physics is learned implicitly or imposed explicitly, the resulting scorer can still be operator-blind, judging a future plausible without registering that it is coherent only under a different world than the present one.
We make the failure precise (§ 3). A context induces a latent operator , and a candidate future is scored by an energy , lower meaning more coherent. We distinguish coherent-here (, valid under the current operator), plausible-elsewhere (, valid under a different operator), and incoherent (). Operator-blindness is the inability to order below even when noise is rejected—and, as we show formally, it is invisible to any generic metric whose candidate set omits .
Our remedy is a change in what counts as a negative. OMC inserts valid futures from a different operator as semantic hard negatives in a contrastive ranking objective (§ 4). Such a negative is a legitimate physical rollout—wrong only here—so contrasting against it forces the scorer to track the inferred operator rather than surface plausibility.
Contributions.
A failure mode, formalized. We define operator-blindness and prove that generic top-1 ranking cannot detect it whenever the candidate set excludes wrong-operator futures (§ 3, Prop. 1).
An objective and a protocol. OMC uses wrong-operator hard negatives; we pair it with operator-sensitive ranking, coherence stratification, and an operatortrajectory energy matrix, and prove that the wrong-operator negative is what drives the trained ordering (§ 4, Prop. 2).
Evidence. On synthetic IID/OOD operator parameters, two backbones, and feature-level CoPhy ballsCF, OMC lifts operator-sensitive metrics well above a baseline that retains generic top-1 (§ 6–§ 8).
OMC is a coherence signal for trajectory scoring, not a generative architecture or planner; we substantiate exactly the relational claim above and delimit the rest in § 9. In one line: generic top-1 answers “is this future plausible?”, whereas OMC asks “is it coherent here?”—and our central finding is that the first question can be answered almost perfectly while the second is answered at chance.
Related Work
World models. Latent-dynamics and Dreamer-style models learn dynamics and imagine rollouts for control [3, 26, 4, 27]; joint-embedding predictive architectures learn predictive representations and acquire intuitive physics from video [5, 6, 23, 28]. These are representational/generative approaches; OMC is a complementary objective/evaluation signal that can sit atop such representations, and the latent-operator inference we use is standard rather than a contribution.
Explicit physical structure. A parallel tradition encodes physics through physics-informed losses [7] or conservation-law architectures [8, 9]. OMC neither imposes nor presupposes such structure; it asks the scorer to learn a relative coherence signal, and our results show that neither implicit nor explicit physics by itself yields operator-sensitive scoring.

Figure 1. Operator-blindness. Under operator , both and (valid under a different operator ) are physically plausible; only is coherent here. A scorer that merely rejects may still fail to separate the dashed pair—and, by Prop. 1, generic ranking cannot reveal it.
Contrastive and energy-based models. Structured contrastive models such as C-SWM [10] and the InfoNCE family [31, 11, 12, 17, 18, 19, 20, 21, 22] learn representations from contrastive negatives. OMC differs not in using a contrastive energy—that is standard—but in the semantics of the hard negative: a valid trajectory from a mismatched operator. Hard-negative mining [19] sharpens such objectives, but its negatives are defined by embedding similarity rather than operator identity. Ordinary structured negatives (random, state-corrupted, temporal) do not, by themselves, induce operator-sensitive scoring (§ 7). CoPhy [13] (see also CLEVRER [32]) supplies counterfactual mechanics with hidden confounders, a natural external testbed used here for validation.
System identification and context inference. A large literature infers environment parameters or latent task context for prediction and control—system identification, adaptive control, and meta-reinforcement-learning methods that encode task/dynamics context for fast adaptation [30, 14, 15, 16]. These aim to use the inferred context to act; OMC instead asks whether a scorer’s compatibility function is sensitive to that context at all, and supplies a negative-construction and protocol to enforce and measure it. Interactive world models and benchmarks [29] emphasize action-conditioned controllability and long-horizon rollout; OMC is complementary, targeting operator-relative coherence of the scorer rather than control.
Problem Formulation
Generative process. A latent operator parameterizes the dynamics; a context and its true future are draws from . A context encoder infers a posterior over the operator,
and a scorer assigns an energy to a candidate future , with lower energy denoting greater coherence. Coherence is identified with operator-conditional likelihood: is coherent under to the extent that is high, the ideal scorer tracking .
| Method family | Learns dynamics | Contrastive negatives | Wrong-operator future | World-relative coherence |
| Latent-dynamics / Dreamer | ✓ | × | × | × |
| JEPA / I-JEPA | ∼ | ∼ | × | × |
| C-SWM / structured contrastive | ✓ | ✓ | × | × |
| CPC / InfoNCE family | ∼ | ✓ | × | × |
| Physics-informed / HNN / LNN | ✓ | × | × | × |
| Meta-RL context inference | ∼ | × | × | × |
| OMC (ours) | × | ✓ | ✓ | ✓ |
Table 1. Positioning of OMC against related method families (citations in the text of this section). “Wrong-operator future” marks whether a method ever contrasts against a valid future from a different operator; “World-relative coherence” marks whether evaluation tests coherence under the in-force operator (not merely generic plausibility). ✓ yes, × no, ∼ partial/implicit.
What the operator can be. We keep abstract on purpose. Although our experiments instantiate it as simulator parameters, the same formulation covers any latent that selects which dynamics are in force: a latent dynamics code, a hidden confounder (as in CoPhy), an environmental law or physical regime, a task/parameter context in meta-RL, or an intervention setting in a causal model. Operator-blindness is therefore not a quirk of one simulator but a generic failure of any scorer whose compatibility function is insensitive to the latent that governs the present context.
Candidate classes. From the generative process we define
A world-relatively coherent scorer should satisfy
Definition 1 (Operator-blindness). A scorer is operator-blind at if
i.e. it rejects noise but orders same- and wrong-operator futures at chance.
The next statement is why operator-blindness goes unnoticed: it is invisible to any generic ranking metric whose candidate pool excludes .
Proposition 1 (Generic ranking cannot detect operator-blindness). Let the generic candidate set be with containing only corrupted/temporal/random negatives, and suppose a.s. for every . Then top1-acc regardless of the ordering of and . In particular an operator-blind scorer and a perfectly coherent scorer are indistinguishable under generic top-1.
Proof. Top-1 accuracy is determined solely by whether attains the minimum energy within ; since is absent from this set, the same/wrong ordering does not enter the event. The hypothesis makes the event certain, so top1-acc for both scorers.
Proposition 1 motivates the protocol of Section 5: to detect operator-blindness one must explicitly insert into the comparison, which is exactly what LAW_PAIR and SAME_LT_WRONG do.
Operator-Mismatch Contrastive Learning
Scorer and operator encoder. The context encoder is a permutation-invariant (DeepSets [33]) network over the context transitions, producing , with latent dimension . The energy scorer is object-centric, in the spirit of interaction networks [34] and slot-based models [21]: with the state of object at step and its action, a per-object self term scores each transition under the operator code, and an optional pairwise term scores object pairs,
with masking over inactive slots and a sum reduction. The pairwise term is an architectural option, disabled in the main objective and studied as a component ablation (§ 6). A direct-context variant keeps the same objective but drops the explicit operator bottleneck, isolating the objective from the backbone.
Candidate set. For each context we form of size : the positive plus one negative of each type, , where perturbs future positions/velocities, reverses or permutes the future, is impossible, and —the OMC negative—is a valid future from a different operator (distinct identity or parameter distance ). Batches contain distinct operators, so each is a genuine wrong-operator rollout.
Objective. We minimize an InfoNCE ranking loss with logits equal to negative energies,
regularized by a KL term on the operator posterior [35] and a stability term tying two halves of a context to the same operator, and . The total objective is
where is a symmetric InfoNCE over an operatortrajectory energy matrix (Appendix § 13). In all reported runs : the matrix term is computed for diagnosis but does not enter the gradient, so the effect we report comes entirely from the wrong-operator negative in Eq. (4); the energy matrix is an evaluation instrument, not a trained objective. The only further axis we vary is the architectural pairwise head of Eq. (3).
The wrong-operator negative is not merely helpful but necessary for the objective to induce the target ordering:
Proposition 2 (The wrong-operator negative induces coherence). For a sufficiently expressive scorer and in the population limit, the minimizer of Eq. (4) satisfies for a candidate-independent ; hence whenever . Conversely, if the objective places no constraint on the relative energies of same- and wrong-operator futures, admitting operator-blind minimizers.
Algorithm 1 OMC training step (one minibatch)
sample a minibatch with distinct operators across samples
for each sample do
context; valid future under the current operator
negatives // : valid future from a different operator
end for
; ;
AdamW update on

Figure 2. OMC pipeline. A context encoder infers ; an object-centric energy scorer ranks the candidate set with InfoNCE. The defining choice is the wrong-operator hard negative . The operatortrajectory energy matrix / OCCL term is a diagnostic only ().
Proof sketch. Softmax cross-entropy over candidates is minimized when the induced selection distribution matches the true posterior that the positive is the in-context continuation; the noise-contrastive optimality result [12, 11] gives optimal scores up to an additive constant, and yields the ordering. If never appears in , the loss is invariant to the value of , so minimizers need not satisfy Eq. (2)—the operator-blind regime. Together with Prop. 1, the wrong-operator trajectory is necessary both to train and to detect world-relative coherence.
Training-time requirement and inference-time usage. A natural objection is that constructing presupposes access to alternative-operator trajectories. We stress that this is a training-and evaluation-time requirement only. Wrong-operator rollouts are needed to build the contrastive candidate set during learning and to probe operator sensitivity during evaluation; they are not required at inference. Once trained, the scorer consumes a context and a single candidate and returns an energy with no alternative-operator data in the loop. OMC is thus an attachable coherence signal: the cost it imposes is the availability of operator-labelled rollouts at train time, exactly the supervision our synthetic generator and CoPhy’s hidden confounder already provide.
Experimental Setup
Synthetic benchmark. LOWM-Synth is a 2D multi-object simulator (horizon ) whose hidden operator is drawn from four families (gravity/damping, attraction/repulsion, collision elasticity, boundary behavior) with continuous parameters. test_iid samples operators from the training ranges; test_ood_param uses held-out parameter ranges of the same families, testing whether coherence generalizes beyond memorized parameters. Operators are metadata used to build negatives and to evaluate; they are never given to the model.
| Metric | Candidate set | Reads as |
| top1_acc | easy negs | generic plausibility (diagnostic) |
| law_pair | full set incl. | ; n/a if absent |
| same_lt_wrong | , fixed 3-way set | |
| full_stratification | ||
| matrix_mrr | operatortraj. | operatortrajectory retrieval |
Table 2. Metrichypothesis map. Each operator-sensitive metric uses a different candidate set; law_pair is defined only when a wrong-operator negative is present.
External validation and backbones. We additionally validate on CoPhy ballsCF [13] at the feature level (segmentation-derived object features, 64px, 20 frames; 35k/7.5k/7.5k split). Two backbones, LOWM (latent operator) and direct-context (no operator bottleneck), separate the objective from the architecture.
Metrics. We separate primary operator-sensitive metrics from a diagnostic one. Primary: law_pair (), same_lt_wrong and full_stratification (the orderings of Eq. (2)), and energy-matrix mrr/top1 (operatortrajectory retrieval). Diagnostic: top1_acc over the generic candidate set. By Prop. 1 the diagnostic alone cannot reveal operator-blindness, so we report it only to expose the trade-off. Formal definitions are in Appendix B. All energy-matrix metrics use matrix_size = 16, max_batches = 8 and are never compared across sizes. We report meanstd over 5 seeds (synthetic/CoPhy) or 3 (baselines); per-seed values are in Appendix D.
Reading law_pair. It is computed only over samples whose candidate set contains a wrong-operator negative. The no-law baseline has none, so its law_pair is reported as 0/n.a. (vacuous)—an instance of Prop. 1, not a behavior; its genuine chance-level operator sensitivity is read from same_lt_wrong 0.5 (Table 7).
| Negative set | top1 | law_pair | same<wr | mat_mrr |
| law-mismatch only | 0.419±0.003 | 0.707±0.002 | 0.711±0.007 | 0.289±0.002 |
| law-mismatch + random | 0.561±0.001 | 0.708±0.003 | 0.713±0.003 | 0.292±0.007 |
| all-4 (main OMC) | 0.384±0.125 | 0.694±0.007 | 0.693±0.003 | 0.269±0.004 |
| no-law ( removed) | 0.996±0.002 | 0.000 | 0.506±0.014 | 0.211±0.003 |
Table 7. Sensitivity to negative-set composition (LOWM, , no-pairwise; 3 seeds, test_iid).
Synthetic Diagnostics and Ablations
Reading guide. We organize the empirical sections as answers to five questions a reviewer would ask. R1: Does operator-blindness exist?—Table 3 and Figure 4. R2: Does OMC fix it?—Table 3, on both backbones and both IID/OOD splits. R3: Is the wrong-operator negative the active ingredient, rather than contrastive learning in general? —the negative-type and composition ablations (Table 4, Table 7) and the general world-model baselines (Table 8). R4: Does it generalize beyond a memorized simulator?—the OOD-parameter rows throughout and the external CoPhy validation (Table 9). R5: How should OMC be used?—as an attachable verifier signal, discussed in § 9. Each table below is tagged with the question it answers.
| Backbone | Obj. | Split | top1 | law_pair | law_only | same<wr | mat_mrr |
| LOWM | OMC | IID | 0.375±0.139 | 0.694±0.005 | 0.411±0.003 | 0.697±0.006 | 0.272±0.012 |
| LOWM | OMC | OOD | 0.363±0.116 | 0.650±0.006 | 0.359±0.011 | 0.665±0.009 | 0.271±0.009 |
| LOWM | no-law | IID | 0.996±0.001 | 0.000 | 0.196±0.013 | 0.507±0.010 | 0.213±0.003 |
| LOWM | no-law | OOD | 0.991±0.001 | 0.000 | 0.205±0.004 | 0.514±0.007 | 0.209±0.004 |
| direct | OMC | IID | 0.574±0.014 | 0.709±0.008 | 0.419±0.008 | 0.723±0.002 | 0.262±0.001 |
| direct | OMC | OOD | 0.562±0.023 | 0.700±0.012 | 0.409±0.018 | 0.705±0.011 | 0.259±0.005 |
| direct | no-law | IID | 0.997±0.001 | 0.000 | 0.207±0.007 | 0.508±0.002 | 0.211±0.000 |
| direct | no-law | OOD | 0.996±0.000 | 0.000 | 0.200±0.007 | 0.502±0.009 | 0.211±0.001 |
Table 3. Main synthetic results (5 seeds, mean±std; matrix_size = 16). LAW_PAIR, SAME_LT_WRONG, MATRIX_MRR are primary; TOP1 is diagnostic.
| Objective | Split | top1 | law_pair | law_gap | same<wr | full str. | mat_mrr |
| all-neg (main OMC) | IID | 0.375±0.139 | 0.694±0.005 | 1.30±0.60 | 0.697±0.006 | 0.697±0.006 | 0.272±0.012 |
| all-neg (main OMC) | OOD | 0.363±0.116 | 0.650±0.006 | 1.21±0.61 | 0.665±0.009 | 0.665±0.009 | 0.271±0.009 |
| no-law-mismatch | IID | 0.996±0.001 | 0.000 | 0.00 | 0.507±0.010 | 0.507±0.010 | 0.213±0.003 |
| no-law-mismatch | OOD | 0.991±0.001 | 0.000 | 0.00 | 0.514±0.007 | 0.514±0.007 | 0.209±0.004 |
| law-mismatch only | IID | 0.416±0.007 | 0.700±0.004 | 2.17±0.08 | 0.712±0.004 | 0.676±0.011 | 0.284±0.012 |
| law-mismatch only | OOD | 0.403±0.003 | 0.695±0.003 | 2.11±0.11 | 0.692±0.006 | 0.660±0.013 | 0.286±0.013 |
Table 4. Negative-type ablation (LOWM, 5 seeds, matrix_size = 16).
Figure 4. Coherence stratification (seed 0, IID). OMC separates , , ; the baseline collapses same vs. wrong while placing noise far above.
| Model | Variant | top1 | law_pair | same<wr | mat_mrr |
| C-SWM-style | OMC-enhanced | 0.358±0.042 | 0.512±0.006 | 0.491±0.012 | 0.212±0.000 |
| C-SWM-style | random-neg | 0.214±0.012 | 0.483±0.003 | 0.493±0.006 | 0.211±0.000 |
| C-SWM-style | state-corrupt | 0.237±0.011 | 0.482±0.007 | 0.492±0.004 | 0.211±0.000 |
| C-SWM-style | temporal-neg | 0.115±0.017 | 0.513±0.024 | 0.517±0.026 | 0.228±0.015 |
| RSSM-lite | (no-law obj.) | 0.590±0.005 | 0.596±0.005 | 0.609±0.007 | 0.270±0.011 |
Table 8. General world-model baselines (3 seeds, IID; OOD in Appendix D). Feature-level scorers.
| Backbone | Obj. | generic top1 | same<wr | full str. | mat_mrr | diag-gap |
| LOWM | OMC | 0.733±0.006 | 0.735±0.006 | 0.714±0.014 | 0.344±0.004 | 1.368±0.031 |
| LOWM | no-law | 0.496±0.002 | 0.496±0.002 | 0.449±0.041 | 0.211±0.001 | 0.332±0.215 |
| direct | OMC | 0.736±0.014 | 0.736±0.014 | 0.736±0.014 | 0.309±0.007 | 1.044±0.014 |
| direct | no-law | 0.504±0.007 | 0.504±0.007 | 0.504±0.007 | 0.210±0.002 | 0.012±0.033 |
Table 9. CoPhy ballsCF external validation (test, 5 seeds).
(R1) Operator-blindness is real and hidden behind top-1. Table 3 states the central result. The no-law-mismatch baseline reaches generic top-1 yet law_pair = 0 and same_lt_wrong 0.5: it rejects noise almost perfectly while ordering current- vs. other-world futures at chance—an empirical instance of Definition 1 that Prop. 1 predicts the diagnostic will miss. OMC inverts the profile, trading generic top-1 (expected for a harder, operator-targeted objective) for large gains on every operator-sensitive metric. The pattern holds for both backbones, so it is driven by the objective rather than the architecture, and it persists under the OOD operator split, ruling out parameter memorization.
(R3) The wrong-operator negative is the active ingredient. Table 4 ablates the negative set, corroborating Prop. 2. Removing the wrong-operator negative (NO-LAW) collapses operator-sensitive scores to chance while preserving generic ranking; using it exclusively (LAW-ONLY) maximizes the targeted metrics. The balanced all-negative objective trades a little of that peak for broader coverage. This isolates the wrong-operator negative—not contrastive learning per se—as the source of the effect.
The pairwise head trades off, it does not dominate. Table 5 compares the pairwise-on (main_all) and pairwise-off settings under the same all-negative objective. The pairwise head raises generic top-1 and lowers variance and slightly improves SAME_LT_WRONG, while pairwise-off attains slightly higher MATRIX_MRR. We therefore do not attribute the OMC effect to the pairwise component; it shifts the trade-off between generic ranking and matrix retrieval.
| Objective | Split | top1 | law_pair | same<wr | mat_mrr | mat_top1 |
| main_all (pairwise on) | IID | 0.509±0.010 | 0.692±0.003 | 0.703±0.004 | 0.255±0.007 | 0.092±0.008 |
| main_all (pairwise on) | OOD | 0.482±0.011 | 0.656±0.004 | 0.670±0.003 | 0.257±0.003 | 0.096±0.005 |
| no_pairwise (main OMC) | IID | 0.375±0.139 | 0.694±0.005 | 0.697±0.006 | 0.272±0.012 | 0.105±0.017 |
| no_pairwise (main OMC) | OOD | 0.363±0.116 | 0.650±0.006 | 0.665±0.009 | 0.271±0.009 | 0.103±0.011 |
Table 5. Component ablation (LOWM, 5 seeds, all-negative objective, matrix_size=16).
Visualizing the mechanism. Figure 3 shows a representative operatortrajectory energy matrix: under OMC the matched (diagonal) operator gives the lowest energy within each column, whereas the baseline shows no such structure. Figure 4 shows the corresponding energy distributions: OMC realizes , while the baseline collapses and yet still separates —operator-blindness made visible.
Figure 3. Energy matrix (seed 0, IID, 16 × 16), per-operator normalized (0 = most coherent in each column). OMC concentrates low energy on the diagonal; the baseline does not.
The effect is relational, not operator decodability. OMC does not produce a more decodable operator code: linear/MLP probes on recover operator identity and binned parameters about equally for OMC and the baseline (seed-0 IID: linear operator-id 0.60 vs. 0.61; binned-parameter 0.89 vs. 0.90; parameter 0.35 vs. 0.39). The gain is therefore relational—OMC shapes the compatibility , consistent with Prop. 2, rather than the operator embedding. Full probe numbers are in Appendix D.
Sensitivity analysis
We vary the main design choices around the paper’s operating point (no-pairwise, , all-4 negatives), changing one knob at a time (3 seeds; the / all-4 row reproduces Table 3). We report test_iid; the OOD split shows the same pattern.
Training the matrix term (). Weighting the operator-coherence matrix loss (Table 6) gives a small additional gain in operator-sensitive metrics (LAW_PAIR 0.694→0.710, MATRIX_MRR 0.269→0.289) at a further cost to generic top-1 (0.38→0.24). Since the wrong-operator negative alone () already captures most of the effect, we keep as the simplest sufficient setting; training the matrix term is a viable, marginally stronger alternative when generic top-1 is not a concern.
| top1 | law_pair | same<wr | mat_mrr | |
| 0.0 | 0.384±0.125 | 0.694±0.007 | 0.693±0.003 | 0.269±0.004 |
| 0.1 | 0.219±0.016 | 0.703±0.006 | 0.700±0.007 | 0.290±0.006 |
| 0.5 | 0.230±0.033 | 0.706±0.008 | 0.703±0.004 | 0.292±0.010 |
| 1.0 | 0.239±0.043 | 0.710±0.009 | 0.710±0.006 | 0.289±0.005 |
Table 6. Sensitivity to (LOWM, no-pairwise, all-4 negatives; 3 seeds, test_iid). =0 is the paper setting.
Negative-set composition. Table 7 varies the candidate negatives. Any set containing the wrong-operator negative reaches LAW_PAIR/SAME_LT_WRONG 0.69–0.71; removing it (NO-LAW) collapses operator sensitivity to chance (SAME_LT_WRONG 0.506, MATRIX_MRR 0.211) while generic top-1 stays near-perfect (0.996). The non-operator negatives mainly trade operator sensitivity for generic top-1 (adding random raises top-1 to 0.56 at no cost to operator metrics), making the OMC top-1 reduction a tunable design choice rather than an inherent penalty. This is the cleanest confirmation of Prop. 2: the wrong-operator negative is necessary and largely sufficient.
Robustness to and temperature. Operator-sensitive metrics are essentially flat across the operator-distance threshold and the InfoNCE temperature (Figure 5): LAW_PAIR stays at 0.69 throughout, so OMC is insensitive to these hyperparameters within the tested ranges.

Figure 5. Robustness of OMC to wrong-operator distance (left) and InfoNCE temperature (right); operator-sensitive metrics stay flat (test_iid, 3 seeds).
General World-Model Baselines
Table 8 places OMC against feature-level C-SWM-style [10] and RSSM-lite scorers. With ordinary structured negatives (random, state-corrupted, temporal), the C-SWM-style scorer stays at chance on operator-sensitive metrics—contrastive structure alone does not induce operator sensitivity, exactly as Prop. 2 predicts when . RSSM-lite is stronger but remains below OMC-trained scorers. These are feature-level scorers, not full agents.
Clarification (Table 8). “OMC-enhanced” adds the wrong-operator negative to the C-SWM-style set. That operator sensitivity still stays at chance indicates an objective–architecture interaction: the C-SWM-style encoder does not expose the operator information the negative would exploit.
CoPhy External Validation
The effect transfers to an external counterfactual-physics dataset (Table 9, Figure 6). On CoPhy ballsCF, OMC raises wrong-confounder rejection (same_lt_wrong ≈ 0.74 vs. 0.50) and operator×trajectory retrieval (matrix_mrr ≈ 0.34 vs. 0.21) for both backbones, so operator-blindness and its OMC remedy are not artifacts of the synthetic generator. CoPhy is used as feature-level external validation.
Figure 6. Summary across settings. OMC vs. no-law on operator-coherent ranking (left) and energy-matrix retrieval (right) for synthetic IID/OOD and CoPhy. The baseline stays near chance everywhere; OMC is consistently above it, including on external CoPhy.
Limitations and Future Directions
By design OMC is a minimal and attachable coherence signal rather than a standalone world model: it demonstrates improved discrimination of world-relative coherence—not generation, planning, or control—and its effect is relational rather than a more decodable operator code. This minimality is a feature: the entire gain follows from one drop-in change to what counts as a negative, which makes the signal portable. The pairwise head is not universally beneficial, LAW_GAP is comparable only within a model family (energy scales differ across backbones), and the CoPhy results are feature-level rather than raw-video. Constructing the wrong-operator negative requires access to alternative-operator trajectories at training time, a property of the signal, not of inference.
The coherence signal suggests a generator–critic decomposition: a proposal model free to entertain plausible-elsewhere futures, paired with OMC as a critic that reranks them for coherence here. Whether such generativity helps downstream world models, and whether OMC can serve as an auxiliary verifier inside latent-dynamics or JEPA-style models, is left to future work.
Three analyses would sharpen the present claims and are immediate next steps: (i) a finer operator-distance study—beyond the threshold sweep of § 6.1—quantifying how same-vs-wrong separation degrades as the wrong operator approaches the true one; (ii) a calibration/verifier analysis (reliability or AUROC) testing OMC as a decision rule rather than only a ranking metric; and (iii) a data/operator-budget curve establishing how many alternative-operator trajectories the signal needs. We regard these as confirmatory rather than load-bearing: the necessity of the wrong-operator negative is already established theoretically (Prop. 2) and by the ablations of § 6.
Conclusion
Plausibility is not coherence. A trajectory scorer can rank generic futures almost perfectly while remaining blind to which world is in force—a failure we formalized as operator-blindness and proved invisible to generic ranking. Operator-Mismatch Contrastive learning makes valid-but-wrong-operator trajectories the hard negatives, and we showed both theoretically and empirically—across synthetic IID/OOD, two backbones, and external CoPhy—that this single change to what counts as a negative exposes and repairs the failure.
Notation and Formal Definitions
Context ; operator prior and conditional ; posterior ; operator code ; energy (lower = more coherent); candidate classes as in § 3; target ordering Eq. (2); operator-blindness Definition 1.
Metric Definitions
For a sample with candidate energies and positive index :
top1_acc= over the set (diagnostic).law_pair= .law_gap= (within model family only).law_only_top1=top1_accon .same_lt_wrong= on .full_stratification= .Energy matrix over distinct-operator samples; with the diagonal ranks,
matrix_mrr= ,matrix_top1= ,diagonal_vs_offdiag_gap= .
Definitions match the evaluation code; energy-matrix metrics use matrix_size=16, max_batches=8.
Objective Details and Proofs
is a symmetric InfoNCE over (matching trajectories to operators in both directions, temperature 1); in all reported runs , so it never enters the gradient. Proofs of Propositions 1 and 2 are given with their statements in § 3 and § 4; both rely only on (i) the candidate set membership and (ii) the standard noise-contrastive optimality result [12, 11].
Robustness of Proposition 2 in practice. The argument is asymptotic (population limit, expressive scorer, temperature 1). Empirically the induced ordering is stable away from that idealization: it is essentially unchanged across InfoNCE temperature and operator-distance threshold (§ 6.1, Figure 5), and holds even though the candidate set mixes in noise negatives drawn from outside the data distribution—these are separated trivially and do not interfere with the same-vs-wrong ordering (Figure 4).
Per-Seed Result Tables
All experiments, scenarios, and seeds are reported below (matrix_size=16, max_batches=8). Splits: IID (test_iid), OOD (test_ood_param), and test/val for CoPhy.
Main synthetic (LOWM and direct-context, OMC vs. no-law-mismatch)
| seed | split | top1 | law_pair | law_gap | law_only | same<wr | full | str. | mat_mrr | mat_t1 | diag-gap |
| direct_context_energy_direct_context_energy | |||||||||||
| 0 | IID | 0.586 | 0.699 | 3.87 | 0.413 | 0.707 | 0.707 | 0.260 | 0.098 | 2.51 | |
| 1 | IID | 0.402 | 0.490 | -0.01 | 0.199 | 0.499 | 0.499 | 0.211 | 0.062 | 0.00 | |
| 2 | IID | 0.590 | 0.705 | 4.10 | 0.409 | 0.703 | 0.703 | 0.255 | 0.090 | 2.66 | |
| 3 | IID | 0.611 | 0.717 | 2.57 | 0.401 | 0.716 | 0.716 | 0.257 | 0.090 | 1.76 | |
| 4 | IID | 0.583 | 0.704 | 3.94 | 0.414 | 0.716 | 0.716 | 0.257 | 0.098 | 2.56 | |
| 0 | OOD | 0.567 | 0.690 | 3.67 | 0.397 | 0.713 | 0.713 | 0.262 | 0.102 | 2.47 | |
| 1 | OOD | 0.391 | 0.489 | -0.04 | 0.208 | 0.490 | 0.490 | 0.211 | 0.062 | 0.00 | |
| 2 | OOD | 0.563 | 0.684 | 3.98 | 0.391 | 0.706 | 0.706 | 0.255 | 0.094 | 2.64 | |
| 3 | OOD | 0.583 | 0.687 | 2.58 | 0.385 | 0.712 | 0.712 | 0.255 | 0.090 | 1.75 | |
| 4 | OOD | 0.563 | 0.689 | 3.69 | 0.376 | 0.704 | 0.704 | 0.260 | 0.102 | 2.48 | |
| 0 | val | 0.608 | 0.744 | 4.58 | 0.461 | – | – | – | – | – | |
| 1 | val | 0.435 | 0.513 | -0.02 | 0.210 | – | – | – | – | – | |
| 2 | val | 0.621 | 0.749 | 4.87 | 0.461 | – | – | – | – | – | |
| 3 | val | 0.644 | 0.750 | 3.09 | 0.450 | – | – | – | – | – | |
| 4 | val | 0.610 | 0.750 | 4.70 | 0.457 | – | – | – | – | – | |
| lowm_lowm_omcr_no_pairwise | |||||||||||
| 0 | IID | 0.331 | 0.695 | 0.99 | 0.409 | 0.696 | 0.696 | 0.273 | 0.113 | 0.66 | |
| 1 | IID | 0.526 | 0.687 | 2.07 | 0.409 | 0.692 | 0.692 | 0.268 | 0.102 | 1.43 | |
| 2 | IID | 0.294 | 0.700 | 0.83 | 0.409 | 0.690 | 0.690 | 0.265 | 0.094 | 0.55 | |
| 3 | IID | 0.514 | 0.698 | 1.98 | 0.413 | 0.705 | 0.705 | 0.261 | 0.086 | 1.35 | |
| 4 | IID | 0.211 | 0.691 | 0.65 | 0.416 | 0.702 | 0.702 | 0.292 | 0.129 | 0.43 | |
| 0 | OOD | 0.320 | 0.647 | 0.87 | 0.340 | 0.657 | 0.657 | 0.262 | 0.094 | 0.59 | |
| 1 | OOD | 0.497 | 0.647 | 2.00 | 0.367 | 0.659 | 0.659 | 0.268 | 0.098 | 1.46 | |
| 2 | OOD | 0.294 | 0.660 | 0.73 | 0.362 | 0.677 | 0.677 | 0.286 | 0.117 | 0.48 | |
| 3 | OOD | 0.474 | 0.650 | 1.90 | 0.366 | 0.659 | 0.659 | 0.265 | 0.094 | 1.38 | |
| 4 | OOD | 0.232 | 0.647 | 0.57 | 0.358 | 0.671 | 0.671 | 0.273 | 0.113 | 0.38 | |
| 0 | val | 0.369 | 0.724 | 1.06 | 0.399 | – | – | – | – | – | |
| 1 | val | 0.563 | 0.727 | 2.41 | 0.430 | – | – | – | – | – | |
| 2 | val | 0.334 | 0.734 | 0.90 | 0.436 | – | – | – | – | – | |
| 3 | val | 0.537 | 0.724 | 2.27 | 0.429 | – | – | – | – | – | |
| 4 | val | 0.270 | 0.730 | 0.70 | 0.431 | – | – | – | – | – | |
| lowm_no_law_mismatch | |||||||||||
| 0 | IID | 0.996 | 0.000 | 0.00 | 0.185 | 0.511 | 0.511 | 0.209 | 0.055 | 1.62 | |
| 1 | IID | 0.997 | 0.000 | 0.00 | 0.207 | 0.516 | 0.516 | 0.214 | 0.062 | 3.41 | |
| 2 | IID | 0.994 | 0.000 | 0.00 | 0.192 | 0.490 | 0.490 | 0.211 | 0.059 | 1.53 | |
| 3 | IID | 0.995 | 0.000 | 0.00 | 0.184 | 0.511 | 0.511 | 0.215 | 0.062 | 0.57 | |
| 4 | IID | 0.996 | 0.000 | 0.00 | 0.212 | 0.506 | 0.506 | 0.215 | 0.066 | 1.45 | |
| 0 | OOD | 0.992 | 0.000 | 0.00 | 0.207 | 0.505 | 0.505 | 0.205 | 0.055 | 0.24 | |
| 1 | OOD | 0.992 | 0.000 | 0.00 | 0.204 | 0.524 | 0.524 | 0.207 | 0.062 | 0.29 | |
| 2 | OOD | 0.992 | 0.000 | 0.00 | 0.211 | 0.517 | 0.517 | 0.212 | 0.062 | 0.36 | |
| 3 | OOD | 0.989 | 0.000 | 0.00 | 0.202 | 0.509 | 0.509 | 0.207 | 0.059 | -0.30 | |
| 4 | OOD | 0.992 | 0.000 | 0.00 | 0.200 | 0.515 | 0.515 | 0.215 | 0.070 | 0.21 | |
| 0 | val | 0.997 | 0.000 | 0.00 | 0.207 | – | – | – | – | – | |
| 1 | val | 0.995 | 0.000 | 0.00 | 0.241 | – | – | – | – | – | |
| 2 | val | 0.997 | 0.000 | 0.00 | 0.208 | – | – | – | – | – | |
| 3 | val | 0.997 | 0.000 | 0.00 | 0.216 | – | – | – | – | – | |
| 4 | val | 0.995 | 0.000 | 0.00 | 0.203 | – | – | – | – | – |
Table 10.
Negative-type ablation: law-mismatch-only
| seed | split | top1 | law_pair | law_gap | law_only | same<wr | full str. | mat_mrr | mat_t1 | diag-gap |
| 0 | IID | 0.406 | 0.700 | 2.26 | – | 0.716 | 0.688 | 0.299 | 0.133 | 1.60 |
| 1 | IID | 0.419 | 0.695 | 2.18 | – | 0.716 | 0.675 | 0.277 | 0.105 | 1.52 |
| 2 | IID | 0.417 | 0.699 | 2.11 | – | 0.707 | 0.667 | 0.266 | 0.094 | 1.47 |
| 3 | IID | 0.426 | 0.707 | 2.26 | – | 0.708 | 0.661 | 0.294 | 0.125 | 1.57 |
| 4 | IID | 0.412 | 0.697 | 2.04 | – | 0.713 | 0.688 | 0.282 | 0.113 | 1.43 |
| 0 | OOD | 0.399 | 0.699 | 2.20 | – | 0.699 | 0.675 | 0.309 | 0.129 | 1.48 |
| 1 | OOD | 0.401 | 0.690 | 2.11 | – | 0.698 | 0.666 | 0.269 | 0.090 | 1.41 |
| 2 | OOD | 0.405 | 0.692 | 2.01 | – | 0.690 | 0.654 | 0.280 | 0.102 | 1.35 |
| 3 | OOD | 0.403 | 0.696 | 2.25 | – | 0.685 | 0.639 | 0.284 | 0.102 | 1.55 |
| 4 | OOD | 0.406 | 0.697 | 1.95 | – | 0.688 | 0.667 | 0.289 | 0.109 | 1.31 |
| 0 | val | 0.445 | 0.741 | 2.84 | 0.445 | – | – | – | – | – |
| 1 | val | 0.441 | 0.737 | 2.73 | 0.441 | – | – | – | – | – |
| 2 | val | 0.452 | 0.738 | 2.63 | 0.452 | – | – | – | – | – |
| 3 | val | 0.430 | 0.733 | 2.82 | 0.430 | – | – | – | – | – |
| 4 | val | 0.439 | 0.737 | 2.53 | 0.439 | – | – | – | – | – |
Table 11.
Component ablation: main_all (pairwise energy on)
| seed | split | top1 | law_pair | law_gap | law_only | same<wr | full str. | mat_mrr | mat_t1 | diag-gap |
| 0 | IID | 0.508 | 0.691 | 1.61 | – | 0.701 | 0.701 | 0.250 | 0.090 | 1.13 |
| 1 | IID | 0.520 | 0.696 | 1.71 | – | 0.696 | 0.696 | 0.264 | 0.102 | 1.20 |
| 2 | IID | 0.508 | 0.688 | 1.63 | – | 0.705 | 0.705 | 0.252 | 0.086 | 1.15 |
| 3 | IID | 0.492 | 0.691 | 1.72 | – | 0.703 | 0.703 | 0.246 | 0.082 | 1.20 |
| 4 | IID | 0.518 | 0.692 | 1.64 | – | 0.708 | 0.708 | 0.261 | 0.102 | 1.17 |
| 0 | OOD | 0.477 | 0.651 | 1.54 | – | 0.669 | 0.669 | 0.258 | 0.094 | 1.18 |
| 1 | OOD | 0.495 | 0.659 | 1.64 | – | 0.671 | 0.671 | 0.261 | 0.102 | 1.22 |
| 2 | OOD | 0.489 | 0.661 | 1.57 | – | 0.675 | 0.675 | 0.253 | 0.090 | 1.18 |
| 3 | OOD | 0.463 | 0.653 | 1.63 | – | 0.667 | 0.667 | 0.253 | 0.094 | 1.21 |
| 4 | OOD | 0.487 | 0.656 | 1.60 | – | 0.670 | 0.670 | 0.260 | 0.102 | 1.18 |
| 0 | val | 0.524 | 0.726 | 2.09 | 0.411 | – | – | – | – | – |
| 1 | val | 0.527 | 0.719 | 2.24 | 0.406 | – | – | – | – | – |
| 2 | val | 0.532 | 0.722 | 2.14 | 0.413 | – | – | – | – | – |
| 3 | val | 0.500 | 0.721 | 2.24 | 0.395 | – | – | – | – | – |
| 4 | val | 0.532 | 0.730 | 2.14 | 0.417 | – | – | – | – | – |
Table 12.
Component ablation: no_pairwise (pairwise energy off)
| seed | split | top1 | law_pair | law_gap | law_only | same<wr | full str. | mat_mrr | mat_t1 | diag-gap |
| 0 | IID | 0.331 | 0.695 | 0.99 | – | 0.696 | 0.696 | 0.273 | 0.113 | 0.66 |
| 1 | IID | 0.526 | 0.687 | 2.07 | – | 0.692 | 0.692 | 0.268 | 0.102 | 1.43 |
| 2 | IID | 0.294 | 0.700 | 0.83 | – | 0.690 | 0.690 | 0.265 | 0.094 | 0.55 |
| 3 | IID | 0.514 | 0.698 | 1.98 | – | 0.705 | 0.705 | 0.261 | 0.086 | 1.35 |
| 4 | IID | 0.210 | 0.691 | 0.65 | – | 0.702 | 0.702 | 0.292 | 0.129 | 0.43 |
| 0 | OOD | 0.320 | 0.647 | 0.87 | – | 0.657 | 0.657 | 0.262 | 0.094 | 0.59 |
| 1 | OOD | 0.497 | 0.647 | 2.00 | – | 0.659 | 0.659 | 0.268 | 0.098 | 1.46 |
| 2 | OOD | 0.294 | 0.660 | 0.73 | – | 0.677 | 0.677 | 0.286 | 0.117 | 0.48 |
| 3 | OOD | 0.474 | 0.650 | 1.90 | – | 0.659 | 0.659 | 0.265 | 0.094 | 1.38 |
| 4 | OOD | 0.232 | 0.647 | 0.57 | – | 0.671 | 0.671 | 0.273 | 0.113 | 0.38 |
| 0 | val | 0.369 | 0.724 | 1.06 | 0.399 | – | – | – | – | – |
| 1 | val | 0.563 | 0.727 | 2.41 | 0.430 | – | – | – | – | – |
| 2 | val | 0.334 | 0.734 | 0.90 | 0.436 | – | – | – | – | – |
| 3 | val | 0.537 | 0.724 | 2.27 | 0.429 | – | – | – | – | – |
| 4 | val | 0.270 | 0.730 | 0.70 | 0.431 | – | – | – | – | – |
Table 13.
C-SWM-style baselines (feature-level)
| seed | split | top1 | law_pair | law_gap | law_only | same<wr | full str. | mat_mrr | mat_t1 | diag-gap |
| 0 | IID | 0.348 | 0.505 | 0.01 | 0.194 | 0.485 | 0.485 | 0.212 | 0.062 | 0.00 |
| 1 | IID | 0.404 | 0.517 | 0.02 | 0.196 | 0.505 | 0.505 | 0.211 | 0.062 | -0.00 |
| 2 | IID | 0.322 | 0.514 | 0.01 | 0.219 | 0.484 | 0.484 | 0.211 | 0.062 | 0.00 |
| 0 | OOD | 0.337 | 0.502 | -0.03 | 0.199 | 0.485 | 0.485 | 0.212 | 0.062 | 0.00 |
| 1 | OOD | 0.364 | 0.516 | 0.01 | 0.205 | 0.500 | 0.500 | 0.211 | 0.062 | -0.00 |
| 2 | OOD | 0.299 | 0.496 | 0.01 | 0.225 | 0.492 | 0.492 | 0.212 | 0.062 | 0.00 |
| cswm_style_energy_cswm_random_neg | ||||||||||
| 0 | IID | 0.204 | 0.481 | -2.59 | 0.188 | 0.488 | 0.078 | 0.211 | 0.062 | 0.16 |
| 1 | IID | 0.227 | 0.486 | -3.80 | 0.184 | 0.492 | 0.058 | 0.211 | 0.062 | 0.08 |
| 2 | IID | 0.212 | 0.483 | -2.26 | 0.193 | 0.499 | 0.109 | 0.211 | 0.062 | -0.03 |
| 0 | OOD | 0.213 | 0.467 | -6.66 | 0.197 | 0.512 | 0.088 | 0.211 | 0.062 | 0.29 |
| 1 | OOD | 0.210 | 0.458 | -9.25 | 0.199 | 0.510 | 0.060 | 0.211 | 0.062 | 0.63 |
| 2 | OOD | 0.225 | 0.475 | -3.96 | 0.204 | 0.501 | 0.087 | 0.211 | 0.062 | -0.08 |
| cswm_style_energy_cswm_state_corrupted_neg | ||||||||||
| 0 | IID | 0.244 | 0.490 | -36.80 | 0.195 | 0.489 | 0.375 | 0.211 | 0.062 | 2.51 |
| 1 | IID | 0.242 | 0.480 | -99.12 | 0.182 | 0.496 | 0.157 | 0.211 | 0.062 | 4.47 |
| 2 | IID | 0.224 | 0.477 | -74.54 | 0.189 | 0.492 | 0.216 | 0.211 | 0.062 | 4.33 |
| 0 | OOD | 0.222 | 0.476 | -100.08 | 0.195 | 0.511 | 0.391 | 0.212 | 0.062 | 3.46 |
| 1 | OOD | 0.221 | 0.462 | -292.88 | 0.202 | 0.504 | 0.160 | 0.211 | 0.062 | 13.84 |
| 2 | OOD | 0.204 | 0.463 | -205.58 | 0.199 | 0.513 | 0.225 | 0.211 | 0.062 | 7.40 |
| cswm_style_energy_cswm_temporal_neg | ||||||||||
| 0 | IID | 0.103 | 0.514 | 1297.24 | 0.238 | 0.499 | 0.002 | 0.234 | 0.086 | 610.12 |
| 1 | IID | 0.108 | 0.489 | 3862.39 | 0.206 | 0.504 | 0.000 | 0.211 | 0.066 | 1398.84 |
| 2 | IID | 0.134 | 0.537 | 6644.52 | 0.245 | 0.547 | 0.001 | 0.240 | 0.082 | 6941.41 |
| 0 | OOD | 0.129 | 0.495 | -119.40 | 0.215 | 0.514 | 0.002 | 0.206 | 0.055 | 169.49 |
| 1 | OOD | 0.114 | 0.493 | 3586.63 | 0.200 | 0.501 | 0.000 | 0.212 | 0.066 | 2645.30 |
| 2 | OOD | 0.171 | 0.547 | 8494.51 | 0.238 | 0.558 | 0.000 | 0.238 | 0.086 | 2536.95 |
Table 14.
RSSM-lite baseline (feature-level)
| seed | split | top1 | law_pair | law_gap | law_only | same<wr | full str. | mat_mrr | mat_t1 | diag-gap |
| rssm_lite_scorer_rssm_lite_mse | ||||||||||
| 0 | IID | 0.596 | 0.602 | 0.00 | 0.282 | 0.608 | 0.608 | 0.269 | 0.113 | 0.00 |
| 1 | IID | 0.588 | 0.594 | 0.00 | 0.284 | 0.616 | 0.616 | 0.281 | 0.133 | 0.00 |
| 2 | IID | 0.587 | 0.592 | 0.00 | 0.281 | 0.602 | 0.602 | 0.259 | 0.102 | 0.00 |
| 0 | OOD | 0.548 | 0.557 | 0.00 | 0.296 | 0.571 | 0.571 | 0.260 | 0.086 | 0.00 |
| 1 | OOD | 0.548 | 0.557 | 0.00 | 0.293 | 0.567 | 0.567 | 0.262 | 0.090 | 0.00 |
| 2 | OOD | 0.549 | 0.557 | 0.00 | 0.284 | 0.574 | 0.574 | 0.248 | 0.070 | 0.00 |
Table 15.
CoPhy ballsCF external validation (feature-level)
| seed | split | top1 | law_pair | law_gap | law_only | same<wr | full str. | mat_mrr | mat_t1 | diag-gap |
| direct_context_energy_direct_context_cophy_OMC | ||||||||||
| 0 | test | 0.730 | 0.730 | 1.07 | 0.406 | – | – | 0.314 | 0.137 | 1.23 |
| 1 | test | 0.740 | 0.740 | 1.08 | 0.424 | – | – | 0.310 | 0.125 | 1.23 |
| 2 | test | 0.728 | 0.730 | 1.05 | 0.402 | – | – | 0.310 | 0.125 | 1.23 |
| 3 | test | 0.756 | 0.758 | 1.08 | 0.460 | – | – | 0.334 | 0.145 | 1.28 |
| 4 | test | 0.732 | 0.732 | 1.07 | 0.386 | – | – | 0.329 | 0.152 | 1.26 |
| 0 | val | 0.700 | 0.706 | 1.00 | 0.424 | – | – | 0.315 | 0.145 | 1.05 |
| 1 | val | 0.704 | 0.712 | 1.02 | 0.410 | – | – | 0.308 | 0.125 | 1.07 |
| 2 | val | 0.704 | 0.712 | 1.00 | 0.412 | – | – | 0.294 | 0.109 | 1.08 |
| 3 | val | 0.724 | 0.732 | 1.06 | 0.444 | – | – | 0.323 | 0.141 | 1.12 |
| 4 | val | 0.720 | 0.728 | 1.03 | 0.428 | – | – | 0.304 | 0.129 | 1.08 |
Table 16.
| seed | split | top1 | law_pair | law_gap | law_only | same<wr | full str. | mat_mrr | mat_t1 | diag-gap |
| direct_context_energy_direct_context_cophy_no_law_mismatch | ||||||||||
| 0 | test | 0.996 | 0.000 | 0.00 | 0.188 | – | – | 0.212 | 0.062 | -0.11 |
| 1 | test | 0.996 | 0.000 | 0.00 | 0.198 | – | – | 0.212 | 0.062 | -0.14 |
| 2 | test | 0.996 | 0.000 | 0.00 | 0.186 | – | – | 0.210 | 0.059 | -0.11 |
| 3 | test | 0.996 | 0.000 | 0.00 | 0.188 | – | – | 0.209 | 0.059 | -0.17 |
| 4 | test | 0.996 | 0.000 | 0.00 | 0.192 | – | – | 0.208 | 0.059 | -0.13 |
| 0 | val | 0.978 | 0.000 | 0.00 | 0.172 | – | – | 0.213 | 0.062 | 0.14 |
| 1 | val | 0.978 | 0.000 | 0.00 | 0.180 | – | – | 0.212 | 0.062 | 0.10 |
| 2 | val | 0.980 | 0.000 | 0.00 | 0.180 | – | – | 0.213 | 0.062 | 0.11 |
| 3 | val | 0.980 | 0.000 | 0.00 | 0.188 | – | – | 0.209 | 0.059 | 0.14 |
| 4 | val | 0.980 | 0.000 | 0.00 | 0.184 | – | – | 0.213 | 0.062 | 0.10 |
| lowm_lowm_cophy_OMC | ||||||||||
| 0 | test | 0.738 | 0.752 | 1.44 | 0.426 | – | – | 0.365 | 0.160 | 1.74 |
| 1 | test | 0.724 | 0.740 | 1.44 | 0.420 | – | – | 0.366 | 0.164 | 1.76 |
| 2 | test | 0.720 | 0.738 | 1.54 | 0.418 | – | – | 0.364 | 0.156 | 1.84 |
| 3 | test | 0.730 | 0.748 | 1.45 | 0.430 | – | – | 0.372 | 0.168 | 1.75 |
| 4 | test | 0.748 | 0.756 | 1.46 | 0.438 | – | – | 0.366 | 0.160 | 1.74 |
| 0 | val | 0.720 | 0.742 | 1.41 | 0.430 | – | – | 0.372 | 0.184 | 1.53 |
| 1 | val | 0.716 | 0.744 | 1.45 | 0.426 | – | – | 0.363 | 0.168 | 1.55 |
| 2 | val | 0.736 | 0.762 | 1.53 | 0.436 | – | – | 0.379 | 0.191 | 1.63 |
| 3 | val | 0.720 | 0.742 | 1.43 | 0.430 | – | – | 0.374 | 0.184 | 1.55 |
| 4 | val | 0.722 | 0.740 | 1.45 | 0.446 | – | – | 0.363 | 0.172 | 1.55 |
| lowm_lowm_cophy_no_law_mismatch | ||||||||||
| 0 | test | 0.978 | 0.000 | 0.00 | 0.202 | – | – | 0.212 | 0.062 | 1.09 |
| 1 | test | 0.982 | 0.000 | 0.00 | 0.198 | – | – | 0.211 | 0.062 | 0.66 |
| 2 | test | 0.978 | 0.000 | 0.00 | 0.190 | – | – | 0.210 | 0.059 | 0.23 |
| 3 | test | 0.982 | 0.000 | 0.00 | 0.194 | – | – | 0.213 | 0.066 | 0.56 |
| 4 | test | 0.988 | 0.000 | 0.00 | 0.190 | – | – | 0.212 | 0.062 | 0.41 |
| 0 | val | 0.970 | 0.000 | 0.00 | 0.204 | – | – | 0.214 | 0.066 | 1.33 |
| 1 | val | 0.972 | 0.000 | 0.00 | 0.208 | – | – | 0.213 | 0.066 | 0.81 |
| 2 | val | 0.968 | 0.000 | 0.00 | 0.216 | – | – | 0.214 | 0.066 | 0.37 |
| 3 | val | 0.970 | 0.000 | 0.00 | 0.210 | – | – | 0.213 | 0.066 | 0.74 |
| 4 | val | 0.970 | 0.000 | 0.00 | 0.212 | – | – | 0.213 | 0.066 | 0.59 |
Table 19.
Reproducibility
Optimizer AdamW [36] (lr , weight decay ); , , , OCCL temperature 1; checkpoint by best LAW_PAIR. Candidate set (one negative per type), operator distance threshold , distinct operators per batch; , hidden/context dim 128, 2 layers, sum energy reduction. Synthetic: 30 epochs, batch 64, 5000/1000 train/val per epoch, seeds 0–4. CoPhy: 30 epochs, batch 2048, 20000/3000, seeds 0–4. Baselines (C-SWM-style, RSSM-lite): 30 epochs, batch 1024, 20000/3000, seeds 0–2. Energy-matrix eval: matrix_size=16, max_batches=8. Hardware: Colab GPU (T4/L4-class).
CoPhy Preprocessing
CoPhy ballsCF (224px) is reduced to segmentation-derived object features at 64px over 20 frames; the hidden confounder serves as the operator metadata for negatives and evaluation. Episode split: 35,000/7,500/7,500 (train/val/test), total 50,000.
References
- [1]B. Lake, T. Ullman, J. Tenenbaum, S. Gershman. Building machines that learn and think like people. Behavioral and Brain Sciences, 2017.arxiv.org/abs/1604.00289
- [2]P. Battaglia, J. Hamrick, J. Tenenbaum. Simulation as an engine of physical scene understanding. PNAS, 2013.DOI
- [3]D. Ha, J. Schmidhuber. World Models. NeurIPS, 2018.DOI
- [4]D. Hafner, T. Lillicrap, J. Ba, M. Norouzi. Dream to Control: Learning Behaviors by Latent Imagina- tion. ICLR, 2020.
- [5]Y. LeCun. A Path Towards Autonomous Machine Intelligence. 2022.
- [6]Q. Garrido, N. Ballas, M. Assran, A. Bardes, L. Najman, M. Rabbat, E. Dupoux, Y. LeCun. Intuitive physics understanding emerges from self-supervised pretraining on natural videos. arXiv:2502.11831, 2025.arxiv.org/abs/2502.11831
- [7]M. Raissi, P. Perdikaris, G. Karniadakis. Physics-informed neural networks. J. Comput. Phys., 2019.
- [8]S. Greydanus, M. Dzamba, J. Yosinski. Hamiltonian Neural Networks. NeurIPS, 2019.
- [9]M. Cranmer, S. Greydanus, S. Hoyer, P. Battaglia, D. Spergel, S. Ho. Lagrangian Neural Networks. ICLR DeepDiffEq Workshop, 2020.arxiv.org/abs/2003.04630
- [10]T. Kipf, E. van der Pol, M. Welling. Contrastive Learning of Structured World Models. ICLR, 2020.
- [11]A. van den Oord, Y. Li, O. Vinyals. Representation Learning with Contrastive Predictive Coding. 2018.
- [12]M. Gutmann, A. Hyvärinen. Noise-contrastive estimation. AISTATS, 2010.
- [13]F. Baradel, N. Neverova, J. Mille, G. Mori, C. Wolf. CoPhy: Counterfactual Learning of Physical Dynamics. ICLR, 2020.arxiv.org/abs/1909.12000
- [14]C. Finn, P. Abbeel, S. Levine. Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks. ICML, 2017.
- [15]K. Rakelly, A. Zhou, D. Quillen, C. Finn, S. Levine. Efficient Off-Policy Meta-RL via Probabilistic Context Variables (PEARL). ICML, 2019.DOI
- [16]L. Zintgraf, K. Shiarli, M. Igl, S. Schulze, Y. Gal, K. Hofmann, S. Whiteson. VariBAD: Bayes-Adaptive Deep RL via Meta-Learning. ICLR, 2020.
- [17]T. Chen, S. Kornblith, M. Norouzi, G. Hinton. A Simple Framework for Contrastive Learning of Visual Representations (SimCLR). ICML, 2020.arxiv.org/abs/2002.05709
- [18]K. He, H. Fan, Y. Wu, S. Xie, R. Girshick. Momentum Contrast for Unsupervised Visual Representation Learning (MoCo). CVPR, 2020.arxiv.org/abs/1911.05722
- [19]J. Robinson, C.-Y. Chuang, S. Sra, S. Jegelka. Contrastive Learning with Hard Negative Samples. ICLR, 2021.arxiv.org/abs/2010.04592
- [20]C.-Y. Chuang, J. Robinson, L. Lin, A. Torralba, S. Jegelka. Debiased Contrastive Learning. NeurIPS, 2020.arxiv.org/abs/2007.00224
- [21]F. Locatello, D. Weissenborn, T. Unterthiner, et al. Object-Centric Learning with Slot Attention. NeurIPS, 2020.
- [22]Y. LeCun, S. Chopra, R. Hadsell, M. Ranzato, F. Huang. A Tutorial on Energy-Based Learning. 2006.
- [23]M. Assran, Q. Duval, I. Misra, et al. Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture (I-JEPA). CVPR, 2023.arxiv.org/abs/2301.08243
- [24]R. Riochet, M. Castro, M. Bernard, et al. IntPhys: A Benchmark for Visual Intuitive Physics Reasoning. IEEE TPAMI, 2022.
- [25]B. Schölkopf, F. Locatello, S. Bauer, et al. Toward Causal Representation Learning. Proceedings of the IEEE, 2021.DOI
- [26]D. Hafner, T. Lillicrap, I. Fischer, R. Villegas, D. Ha, H. Lee, J. Davidson. Learning Latent Dynamics for Planning from Pixels. ICML, 2019.arxiv.org/abs/1811.04551
- [27]D. Hafner, J. Pasukonis, J. Ba, T. Lillicrap. Mastering Diverse Domains through World Models. arXiv:2301.04104, 2023.arxiv.org/abs/2301.04104
- [28]A. Bardes, Q. Garrido, J. Ponce, X. Chen, M. Rabbat, Y. LeCun, M. Assran, N. Ballas. Revisiting Feature Prediction for Learning Visual Representations from Video. arXiv:2404.08471, 2024.arxiv.org/abs/2404.08471
- [29]J. Bruce, M. Dennis, A. Edwards, et al. Genie: Generative Interactive Environments. ICML, 2024.arxiv.org/abs/2402.15391
- [30]L. Ljung. System Identification: Theory for the User. Prentice Hall, 2nd edition, 1999.
- [31]R. Hadsell, S. Chopra, Y. LeCun. Dimensionality Reduction by Learning an Invariant Mapping. CVPR, 2006.DOI
- [32]K. Yi, C. Gan, Y. Li, P. Kohli, J. Wu, A. Torralba, J. Tenenbaum. CLEVRER: Collision Events for Video Representation and Reasoning. ICLR, 2020.arxiv.org/abs/1910.01442
- [33]M. Zaheer, S. Kottur, S. Ravindraksh, B. Poczos, R. Salakhutdinov, A. Smola. Deep Sets. NeurIPS, 2017.
- [34]P. Battaglia, R. Pascanu, M. Lai, D. Rezende, K. Kavukcuoglu. Interaction Networks for Learning about Objects, Relations and Physics. NeurIPS, 2016.arxiv.org/abs/1612.00222
- [35]D. Kingma, M. Welling. Auto-Encoding Variational Bayes. ICLR, 2014.arxiv.org/abs/1312.6114
- [36]I. Loshchilov, F. Hutter. Decoupled Weight Decay Regularization. ICLR, 2019.


