WorksComputer Science
WorldFoundry: Towards Unified Infrastructure for World Intelligence
Abstract
Visual generative models increasingly blur the line between synthesizing observations and simulating environments. Yet visual plausibility alone does not constitute successful world modeling: compelling outputs may lack persistent state, physical dynamics, or causally consistent responses to interactions. We define a generative world model as a predictive dynamical system that maintains a persistent, history-conditioned representation of an environment, updates that representation in response to actions or interventions, and generates temporally coherent trajectories that reflect outcomes under observed and counterfactual interventions. From this definition, we derive an intelligence-oriented framework spanning perceptual grounding, persistent state modeling, controllable transformation, dynamics-aware prediction, and closed-loop interaction. We present WorldFoundry, open-source infrastructure for reproducible inference and evaluation across video, 3D and 4D, embodied action, and interactive world systems. WorldFoundry unifies model and benchmark discovery, runtime and asset preparation, execution orchestration, artifact management, and interactive visualization, while modality-specific adapters preserve native inputs, execution semantics, output representations, and evaluation protocols. Explicit manifests, durable artifact contracts, and evidence-aware scorecards separate execution readiness from evaluative claims, making each claim traceable to its supporting artifacts. The catalog contains 270 model entries, 74 benchmarks, and 62 registered metric entries, with readiness and execution evidence recorded separately. WorldFoundry enables reproducible comparison of persistent, controllable dynamics across heterogeneous systems, advancing the field from perceptual synthesis toward measurable world intelligence.
See Appendix A for author contributions.
Date: November 7, 2026
Email: [email protected]
Project Page: https://openenvision.org/WorldFoundry/
Code: https://github.com/OpenEnvision/WorldFoundry
Introduction
Recent advances in generative modeling are moving beyond synthesizing individual objects or scenes toward modeling worlds that evolve over time and respond to interventions. This shift calls for a broader notion of world intelligence, in which systems must perceive observations, infer structured states, predict their evolution, express resulting consequences through task-specific outputs, and exploit these predictions for interaction [1, 2, 3, 4]. Video generators, neural scene representations, and embodied policies address complementary parts of this problem. However, incompatible representations, runtimes, interfaces, and evaluation protocols make capabilities difficult to compose, results difficult to reproduce, and progress toward genuine world models difficult to measure. The bottleneck is therefore not only algorithmic, but also infrastructural. Recent efforts such as OpenWorldLib [5] address complementary aspects of this integration problem. Progress therefore requires shared infrastructure grounded in a common functional abstraction that makes heterogeneous models composable, reproducible, and comparable across modalities while preserving their native semantics.
To establish common ground without constraining how different systems represent the world, we formulate the abstraction at the level of function. Building on insights from earlier definitions in reinforcement learning [1, 2, 3] and recent efforts to delineate their scope [5], we identify three tightly coupled components. State inference estimates a latent environmental state from the observation history; forward simulation predicts its evolution under endogenous dynamics and external interventions; and interactive response produces distinct, actionconditioned futures as visual, geometric, or task-specific outputs. This decomposition is functional rather than architectural and defines a shared interface, not a requirement that every system implement all three operations. Static 3D methods emphasize persistent-state inference and geometry decoding, video models primarily learn implicit transitions and observation generation, 4D methods explicitly represent time-varying structure, and embodied models connect state estimation and action prediction to decision-making. Characterizing systems by the operations they implement and the capability profiles they support provides a common vocabulary for comparing them.
These operations alone, however, describe what a system does without determining whether it qualifies as a world model. Qualification is decided not by any single operation but by the state that persists across all of them: we define a generative world model as a learned, history- and control-conditioned dynamical system that infers and maintains a persistent environmental state, evolves that state under endogenous dynamics and external interventions, and assimilates observational feedback into coherent future trajectories. Persistence in this sense is operational rather than nominal: an intervention must leave a trace in the state that constrains later predictions, and each incoming observation must in turn revise that state. Only when both hold is the world-modeling loop closed. Visual plausibility, by contrast, reflects only the observation decoder; it is neither necessary nor sufficient evidence that the underlying state persists, a gap that recent physics- and world-oriented benchmarks repeatedly expose [6, 7, 8, 9]. It is closure, not plausibility, that draws a principled boundary between foundational generative models, which synthesize or transform observations without closing the loop, and generative world models, which couple generation to an evolving environmental state through closed-loop updates. Measuring progress from foundational generation toward world intelligence therefore calls for evidence of closure, which in turn depends on shared infrastructure for reproducible execution and comprehensive, fair evaluation.
We realize this infrastructure in WorldFoundry, an open-source system for integrating, executing, and evaluating video, 3D, 4D, interactive world, and embodied-action models. WorldFoundry spans the research lifecycle, from model and benchmark discovery and runtime and asset preparation to execution orchestration, artifact management, evaluation, and interactive visualization. Modality-specific adapters preserve native execution and evaluation semantics; declarative manifests specify dependencies, assets, and interfaces; durable artifact contracts decouple execution from evaluation; and evidence-aware scorecards separate declared support, operational readiness, and validated performance. The current release integrates 270 models and over 100 benchmark and metric implementations, with every model runner validated through end-to-end execution of a documented reference workflow. By combining our theory of world modeling with the proposed infrastructure, WorldFoundry transforms isolated generative demonstrations into cumulative, auditable evidence and advances the field from perceptual synthesis toward measurable world intelligence.
Our key contributions can be summarized as follows:
World Model Definition. We formally define world models through a minimal abstraction comprising state inference, forward simulation, and interactive response. We then characterize the core capabilities of perception, modeling, manipulation, dynamics, and interaction within a structured functional framework, thereby organizing and differentiating existing generative and world-modeling systems.
Unified Inference and Evaluation Infrastructure. We introduce WorldFoundry, an open-source infrastructure that standardizes execution and evaluation across video, 3D, 4D, embodied-action, and interactive world systems, making inference reproducible and configurable without overriding native model semantics, and placing heterogeneous systems under a common operational definition of world modeling.
Validation at Scale. We integrate 270 generative models and over 100 benchmark and metric implementations. We validate every model runner through a documented end-to-end reference workflow, yielding an auditable record of operational readiness under the tested configurations.
Related Work
| System | Video | 3D/4D | Inter. WM | Embodied | Model runner | Hosted API | Closed loop | Training | Protocols | Evidence | WM def. | Interfaces |
| WORLD-MODEL INFRASTRUCTURE | ||||||||||||
| OpenWorldLib [5, 10] | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | × | ✓ | ◐ | ✓ | Py · CLI |
| Cosmos [11, 12, 13] | ✓ | × | ✓ | ✓ | ◐ | ✓ | ✓ | ✓ | ✓ | ◐ | ◐ | Py · CLI · Web |
| Genie Envisioner [14, 15] | ◐ | × | ✓ | ✓ | ◐ | × | ✓ | ✓ | ◐ | ◐ | ◐ | Py · CLI<br>Web (monitor) |
| minWM [16] | ◐ | × | ✓ | × | ◐ | × | × | ✓ | × | ◐ | ◐ | Py · CLI |
| MODALITY-SPECIFIC TOOLKITS | ||||||||||||
| Diffusers [17] | ✓ | ◐ | ◐ | ◐ | ✓ | × | × | ✓ | ◐ | ◐ | × | Py |
| LeRobot [18] | ◐ | × | ✓ | ✓ | × | × | ✓ | ✓ | ◐ | ◐ | × | Py · CLI |
| StarVLA [19] | × | × | × | ✓ | ◐ | × | ✓ | ✓ | ◐ | ◐ | × | Py · CLI |
| EVALUATION SUITES | ||||||||||||
| VBench [6, 20, 21] | ✓ | × | × | × | × | × | × | × | ◐ | ◐ | × | Py · CLI · LB |
| WorldScore [22] | ✓ | ✓ | × | × | ◐ | ◐ | × | × | ◐ | ◐ | ◐ | Py · CLI · LB |
| WorldArena [9, 23] | ◐ | × | ◐ | ✓ | ◐ | × | ✓ | ◐ | ◐ | ◐ | ◐ | Py · CLI · LB |
| World-in-World [24] | ◐ | × | ◐ | ✓ | ◐ | ◐ | ✓ | ◐ | ◐ | ◐ | ◐ | Py · CLI |
| WorldFoundry | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | × | ✓ | ✓ | ✓ | TUI · CLI · Web<br>Py · MCP |
Table 1. Comparison with world-model infrastructures, modality-specific toolkits, and evaluation suites. ✓: supported; ◐: specialized support; ×: no documented workflow. Column definitions follow the table.
Reading the table. Coverage: ◐ restricted task or domain; Inter. WM requires action-conditioned dynamics, not camera-path rendering alone. Model runner: ✓ independently released families behind shared interfaces; ◐ first-party models or benchmark-specific adapters. Hosted API: provider-hosted generation or policy services. Closed loop: controls depend on prior outputs or environment feedback. Training: ◐ protocol-specific recipe only. Protocols: ✓ independent benchmarks across task families; ◐ a single suite or domain. Evidence: ✓ links entry readiness to result provenance; ◐ stores configurations or scores only. WM def.: ✓ framework-level criterion and taxonomy; ◐ domain-specific formulation. VBench, WorldArena, and Genie Envisioner rows include VBench++/2.0, WorldArena 2.0, and GE-Sim 2.0. LB: leaderboard; MCP: Model Context Protocol. Marks describe documented workflows; entry-level readiness and validation scope are reported in the appendices. Validation applies to the documented route and configuration.
World Models and Research Infrastructure
Learned world models link environmental prediction to planning and control. World Models [1] combined compact visual representations with recurrent dynamics and policy learning; PlaNet [2] planned through latent dynamics, while DreamerV3 [3] learned policies from imagined trajectories. Research on video generation, spatial reconstruction, and embodied control introduces diverse conditioning schemes, state representations, and execution requirements. Specialized toolkits address these requirements within individual domains: Diffusers [17] organizes generative pipelines, Nerfstudio [25] supports neural reconstruction and rendering, LeRobot [18] and StarVLA [19] support policy development and deployment, and Isaac Lab [26] provides simulation and learning workflows. World-model platforms cover broader development lifecycles. Cosmos [11, 12] combines data preparation, foundation models, and post-training, complemented by scoring orchestration and durable result storage in Cosmos Evaluator [13]. Genie Envisioner and GE-Sim 2.0 [14, 15] connect action-conditioned video simulation with robot-policy evaluation and learning, while minWM [16] supports fine-tuning and distillation for controllable autoregressive generation. OpenWorldLib [5, 10] is the most closely related infrastructure for heterogeneous integration: its pipeline, operator, representation, reasoning, and memory modules unify video, spatial, and embodied systems through common interfaces, supporting multi-turn execution and evaluation from saved results. Its formulation emphasizes perception, interaction, and long-term memory. WorldFoundry couples an operational criterion grounded in persistent state, intervention effects, and observational feedback (§ 4–§ 4.1) with explicit execution contracts: manifests specify runtime and asset requirements, serializable requests preserve native controls, and typed artifact references link outputs to execution provenance. These contracts connect model invocation, artifact management, and evaluation within a shared experimental lifecycle; general training remains future work.
Evaluation Protocols and Experimental Evidence
Evaluation suites operationalize complementary notions of generative quality, environmental consistency, and interactive utility. VBench and its extensions [6, 20, 21] provide a multidimensional assessment of perceptual quality, semantic alignment, and intrinsic faithfulness, while VideoPhy [27] and WorldModelBench [8] examine whether generated events and interactions satisfy physical constraints and commonsense expectations. WorldScore [22] evaluates video and spatial generation under camera-conditioned protocols, connecting assessment to specified controls over the generated scene. iWorld-Bench [28] focuses on interactive control and memory, where model responses must be interpreted in relation to preceding inputs and interactions. WorldArena [9, 23] and World-in-World [24] extend assessment to embodied utility, policy performance, and closed-loop planning, linking world-model capabilities to their effectiveness within agent workflows. Evaluation infrastructure supports these assessments through complementary mechanisms: AIGVE-Tool [29] provides modular scoring components, and HarnessEval-W [30] organizes judgments around structured supporting evidence. The evidence required for evaluation consequently varies across protocols. Some assessments can operate on fixed generated artifacts, whereas interactive and embodied tasks may require control histories, environment feedback, and executed rollouts. Differences in these requirements and in score aggregation make execution conditions and protocol compliance central to meaningful comparison. WorldFoundry records resolved configurations, artifact references, sample coverage, failures, and validation scope for each evaluation (§ 5.5). Its scorecards distinguish diagnostic results, benchmark-comparable results, and complete reproductions under the corresponding protocol. Compatible evaluators can rescore saved outputs; protocols requiring new interventions or policy rollouts retain their execution requirements. These records make evaluative claims traceable to their supporting artifacts, execution conditions, and validation evidence.
Unified Definition of World Modeling
Figure 2. World Modeling. State inference maintains a persistent environment representation from observations; forward simulation predicts coherent state transitions; interactive response produces action-conditioned futures. Observation feedback updates the state for subsequent predictions.
A fundamental obstacle to progress in generative world modeling research is the absence of a precise and shared conceptual foundation. Without an operational definition of what constitutes a world model and a principled criterion for distinguishing it from related systems, the field risks conflating fundamentally different modeling objectives. This section aims to address this gap. We first establish an operational definition of world modeling; we then situate the model landscape along an intelligence-oriented capability progression that traces increasingly demanding functional behaviors; finally, we formulate a principled boundary between foundational generative models that synthesize observations and generative world models that simulate world.
What is World Model: An Operational Definition
Motivated by previous work on learned world models in reinforcement learning [1, 2, 3], we characterize world modeling by three tightly coupled operations over task-relevant latent state rather than observations alone: (i) state inference, which estimates latent environmental state from observations; (ii) forward simulation, which predicts state evolution under autonomous dynamics, physical constraints, and external actions; and (iii) interactive response, which adapts predicted futures coherently under alternative actions or interventions. Let denote an observation, an action or intervention, and the interaction history. A minimal, modality-agnostic abstraction is
Here denotes a task-relevant environmental state, while , , and denote state inference, transition, and observation models. Repeated application of yields predictive rollouts under alternative control sequences; once is observed, the state is re-inferred as before subsequent predictions, closing the loop between prediction and feedback, following the latent-dynamics perspective used in model-based planning [2]. This formulation is functional rather than architectural: may be recurrent, object-centric, geometric, token-based, or otherwise structured. Coupling these generic world-modeling operations into a single persistent process likewise yields our foundational definition of generative world models:
Definition: Generative World Model. A Generative World Model is a learned, history- and control-conditioned latent dynamical system that infers and maintains a persistent environmental state, evolves that state under endogenous dynamics and external interventions, and assimilates feedback to generate coherent future trajectories.
With this definition, a generative world model synthesizes possible future trajectories of observations grounded in a persistent, history-conditioned environmental state. Generation under alternative interventions branches from a shared state, with the consequences of each intervention constraining subsequent outputs. Newly acquired observations update the maintained state and inform further generation. The defining property is therefore the coupling of observation synthesis with persistent state, intervention-conditioned dynamics, and observational feedback. These requirements motivate the capability progression below, which characterizes how constituent capabilities jointly support coherent world generation and interaction.
What World Models Can Do: Capability Progression
Figure 3. Capability progression from generative modeling toward world modeling. Perceptual grounding provides the foundation; persistent state modeling, dynamics-aware prediction, and closed-loop interaction form the world-model spine; controllable transformation provides an orthogonal capability for manipulation.
To situate existing generative systems with respect to this definition, we organize them along a capability progression in which generation becomes increasingly grounded in an evolving environmental state. Perceptual Grounding constitutes the foundation: the capability to interpret, reconstruct, or synthesize observations without maintaining a persistent environmental state. The World-Model Spine comprises three successive stages, each building on its predecessor: Persistent State Modeling infers and maintains a structured environmental state across observations; Dynamics-Aware Prediction predicts physically and causally coherent transitions of that state under endogenous dynamics and external interventions; and Closed-Loop Interaction sustains repeated cycles of action, observation, and state update, in which each observation revises the state that conditions subsequent predictions. Controllable Transformation, the capability to modify represented entities, attributes, or configurations as specified while preserving content outside the scope of the edit, is orthogonal to the spine and can complement any of its stages; by itself, however, it entails none of the spine capabilities. Rather than assigning each system to a single discrete level, the progression characterizes it by a capability profile: which capabilities it exhibits and the degree to which they are integrated.
Foundational Capability: Perceptual Grounding
Perceptual Grounding (Perception) is the capability to interpret, reconstruct, or synthesize observable content. It captures structure at the observation level, supplying the perceptual basis for state inference without itself maintaining a persistent representation of the underlying environment.
World-Model Spine
Persistent State → Dynamics Prediction → Interactive Feedback
Spine I: Persistent State Modeling
Persistent State Modeling (Modeling) is the capability to infer, maintain, and recursively update a structured representation of environmental state from sequential observations. This representation tracks task-relevant entities and their identities, geometry, spatial relations, and latent attributes across viewpoint changes and occlusions under partial observability.
Spine II: Dynamics-Aware Prediction
Dynamics-Aware Prediction (Dynamics) is the capability to predict environmental state transitions in accordance with physical and causal regularities. It propagates the effects of endogenous processes, entity interactions, and external interventions through the evolving state, producing temporally coherent trajectories in which earlier events constrain later predictions through their persisting consequences.
Spine III: Closed-Loop Interaction
Closed-Loop Interaction (Interaction) is the capability to sustain recurrent cycles of action, environmental transition, observation, and state update within a persistent environment. Feedback from observed action outcomes revises the maintained state, which in turn conditions subsequent predictions and actions, preserving causal continuity across successive interactions.
Orthogonal Capability: Controllable Transformation
Controllable Transformation (Manipulation) is the capability to modify represented content or state according to specified controls while preserving properties and structural relations outside the intended scope of the edit. As an orthogonal capability, it can complement any stage of the progression, but on its own establishes neither persistent state modeling nor coherent dynamics or closed-loop interaction.
Taken together, these capabilities characterize world intelligence through their integration rather than their individual strength: observations update persistent state, state constrains predictions, and action outcomes inform subsequent inference and interaction. Current systems occupy this progression unevenly. Video generators achieve strong perceptual grounding with largely implicit state; 3D and 4D methods maintain explicit spatial structure but rarely predict its evolution under actions; interactive world models close the loop over short horizons and have begun to address longer-range memory; and embodied policies act within persistent environments under narrow task and embodiment coverage. Evaluating where a system sits therefore requires evidence about the cycle itself, not only about output quality. Existing benchmarks test complementary segments of this progression: visual and physical fidelity [6, 8], camera-controlled world generation [22], interactive memory [28], and downstream embodied utility [9, 23].
Taxonomy: Foundational Generative Models and Generative World Models
Building on the definition and capability progression above, we draw a single boundary through contemporary generative systems: whether a system closes the world-modeling loop. Models that primarily generate or transform observations count as foundational generative models, even with partial, local, or implicit dynamics, unless they maintain persistent state, support intervention-conditioned dynamics, and assimilate observational feedback. Generative world models, by contrast, meet all three conditions: they maintain latent world states that evolve under intervention and are revised by incoming observations.
Foundational Generative Models learn visual, geometric, or spatiotemporal priors for synthesis, reconstruction, and controllable manipulation, but do not integrate state, dynamics, and feedback into a complete world-modeling loop. Their defining limitation is the inability to maintain an evolving environmental state that preserves intervention effects and incorporates observational feedback into subsequent predictions. Strong individual capabilities therefore do not suffice to sustain a coherent, revisable model of the world.
Generative World Models, by contrast, integrate persistent state, intervention-conditioned dynamics, and observational feedback into a complete world-modeling loop. Their defining property is functional coupling: intervention effects persist across state transitions, while each new observation revises the state governing subsequent predictions. Generation is thereby made accountable to an evolving environmental model, supplying a functional basis for world intelligence, the capacity to anticipate the consequences of intervention and to revise those expectations through interaction.
WorldFoundry Design
Assessing the capabilities defined above requires reproducible experiments that tie observed behavior to explicitly specified tasks and evaluation conditions. Existing implementations, however, rely on incompatible environments, input conventions, execution procedures, artifact formats, and evaluation protocols, making it difficult to tell whether observed differences arise from model capabilities or from the surrounding experimental pipelines. This fragmentation also blurs the distinctions among catalog inclusion, successful inference, partial validation, and benchmark reproduction, which are often conflated under a single notion of “model support.” Related infrastructures address complementary aspects of this problem, including OpenWorldLib for world-model inference [5], Cosmos for physical-AI model development [11], and StarVLA for modular embodied-policy research [19]. Common to all of these settings is a central design challenge: preserving model-specific execution semantics while making explicit what has been executed, under what conditions, and which conclusions the resulting evidence supports.
We introduce WorldFoundry as open-source infrastructure organized around a common lifecycle of discovery, runtime preparation, execution, inspection, and evaluation. It serves three functions. (I) It integrates declarative model and benchmark catalogs with explicit runtime, dependency, and asset management, allowing researchers to discover, configure, and execute foundational generative models for video, 3D, and 4D alongside generative world models, embodied simulators, and vision-language-action, vision-action, and world-action models, while preserving native execution semantics. (II) It decouples execution from evaluation through durable artifacts and serializable request and result contracts, so that generated videos, spatial representations, actions, trajectories, and execution traces can be inspected, reused, compared, and scored without re-instantiating the originating model or runtime; run manifests, sample results, metric summaries, reports, and scorecards record the provenance and evidential scope of each experiment. (III) It provides interactive interfaces for configuring runs, issuing control inputs, visualizing multimodal outputs, comparing results across model families, and diagnosing failures throughout generation, interaction, and evaluation.
System Overview
WorldFoundry is organized around a user-oriented, capability-aware research lifecycle that brings heterogeneous visual generation and world-modeling systems into a coherent operational framework. Rather than requiring users to navigate fragmented, model-specific inference scripts and disconnected evaluation pipelines, it establishes shared conventions for model discovery, runtime preparation, execution, artifact inspection, and evidence-based evaluation while preserving model-specific semantics. The framework comprises five components (Figure 1), each described in a subsequent subsection. (1) _Surface Layer_ provides consistent user-facing access through a terminal interface, command-line tools, a browser workspace, a Python API, and programmatic interfaces. (2) _Unified Inference and Interaction_ resolves model configurations, runtime profiles, and required assets, and supports both stateless generation and stateful interaction through shared request and execution pathways without altering model-native inputs, parameters, or execution semantics. (3) _Artifact and Representation Contracts_ specify how requests, results, generated outputs, state traces, and execution records are materialized as durable, inspectable artifacts with explicit identity, semantics, metadata, and provenance. (4) Evaluation and Evidence applies compatible metrics and benchmark protocols to these artifacts and records per-sample results, aggregate scores, coverage, validation status, unresolved blockers, and comparison eligibility. (5) Model and Benchmark Zoo maintains manifest-driven registries of models, benchmarks, tasks, runtime profiles, assets, evaluators, readiness states, and known blockers.
Figure 1. Overview of WorldFoundry. Open-source infrastructure for generative world models. (1) one run is reachable from every interactive and programmatic surface; (2) preflight requests execute through a unified runner, one-shot or session-resident; (3) outputs are persisted behind durable artifact contracts with typed references; (4) benchmarks score persist in artifacts reported in an evidence scorecard; and (5) declarative manifests bind discovery, runtime and evaluation, keeping claims traceable to evidence.
Across these components, persistent artifacts form the explicit boundary between execution and assessment. This separation allows computationally expensive outputs to be inspected, compared, rescored, and audited without re-executing the originating model, and keeps catalog inclusion, runnable integration, bounded validation, and complete benchmark reproduction as distinct claims rather than a single notion of support.
Surface Layer
The Surface Layer offers multiple user-facing interfaces that share a single orchestration path, ensuring consistent execution and evaluation across access modes. The terminal user interface (TUI) guides discovery by querying model and benchmark catalogs, checking requirements and readiness, and resolving an executable command before resources are allocated. The command-line interface (CLI) covers discovery, preparation, inference, evaluation, validation, and reporting, and is designed for automated experimentation and reproducible benchmarking. The browser-based Workspace supports multi-model job configuration, model-specific input specification, runtime selection, execution monitoring, artifact inspection, and cross-run comparison, while the single-model Studio offers tailored controls for interactive use and detailed inspection. Python and MCP interfaces extend this functionality to external programs and agents. Every surface translates user intent into normalized requests that are resolved against shared catalogs, runtime profiles, Pipelines, Operators, and result contracts; this shared resolution keeps results comparable regardless of how a run is launched. Figure 4–Figure 6 illustrate the complementary roles of the three primary human-facing surfaces.

Figure 4. Command-line interface (CLI). WorldFoundry CLI is a unified command-line interface for configuring and executing model inference and evaluation workflows. Screenshot from the CLI reference.

Figure 6. Terminal user interface (TUI). WorldFoundry TUI is an interactive terminal interface for configuring and executing model inference and evaluation workflows through guided menus. Screenshot from the TUI guide.

Figure 5. Workspace and Studio. WorldFoundry Workspace provides a unified browser interface for managing model inference and evaluation workflows, while Studio offers interactive environments tailored to different modalities. Screenshots from the Studio guide.
Studio additionally provides modality-aware visualization and interaction. It displays generated videos, temporal previews, galleries, and artifact references; renders point clouds and meshes in dedicated Viser views; and presents Gaussian splats in the Spark viewer. For interactive world models, a resident model session allows navigation, camera, and text controls to update the environment state without reinitializing the Pipeline for each input. Execution status, runtime logs, generation progress, and performance telemetry help users diagnose model-loading failures, interaction latency, generation instability, and invalid artifacts. Embodied systems that require native simulation connect through an external simulator bridge, which isolates simulator-specific dependencies from the browser-based Workspace. Together, these features unify discovery, configuration, execution control, qualitative inspection, and runtime diagnosis, while preserving generated outputs as durable artifacts for subsequent evaluation.
Unified Inference and Interaction
WorldFoundry resolves inference and interaction requests through a manifest-driven execution sequence. Each model identifier or alias is mapped to a canonical catalog entry whose manifest specifies provenance, supported tasks and variants, input and output modalities, required assets, runtime configuration, execution binding, and readiness constraints. Given the requested task and variant, the resolver verifies environment and asset compatibility and availability, reports blockers, and selects an execution backend. Once resolution succeeds, the input is encoded as a GenerationRequest, which defines common fields for sample identity, task context, conditioning inputs, control signals, generation parameters, and output specifications while retaining model-specific payloads. This contract standardizes orchestration without imposing uniform input semantics: video systems retain temporal and media-conditioning semantics; 3D and 4D systems retain camera, coordinate-frame, geometry, and rendering semantics; embodied systems retain observation–action schemas and simulator contracts; and interactive world models retain session state and sequential controls. The corresponding GenerationResult records execution status, timing, errors, the resolved configuration, and artifact references, establishing a common basis for traceability and downstream evaluation.
Given a resolved GenerationRequest, the WorldModelRunner coordinates execution through two complementary abstractions: an Operator and a Pipeline. The Operator handles task-specific adaptation, including input validation, preprocessing, translation into model-native arguments, postprocessing, and conversion of native outputs into framework-compatible artifacts. The Pipeline manages the model lifecycle and inference runtime, including model construction, checkpoint loading, device placement, runtime configuration, distributed execution, and invocation of native inference procedures. This division separates model adaptation from runtime management while accommodating domain-specific dependencies and simulator integration. Interactive systems may retain a resident Pipeline and session state, allowing successive actions or conditions to update the environment without repeated checkpoint loading or runtime initialization. Artifact references and execution metadata populate the GenerationResult; benchmark runners subsequently consume the materialized outputs and provenance records, decoupling evaluation from model loading and inference.
Artifact and Representation Contracts
WorldFoundry defines a two-level contract system that separates sample-level execution from run-level organization. A GenerationRequest represents a single execution unit: inputs contains text, source media, or task-specific observations; controls specifies action sequences, camera trajectories, or other control signals; generation_kwargs retains model-native parameters such as random seeds and frame rates; and output_schema declares expected artifact types and structural requirements. The corresponding GenerationResult records stable sample and model identifiers, execution status, errors, timing, resolved metadata, and output references. Each materialized output is described by an ArtifactRef, which identifies externally stored content by its semantic role and URI, with optional MIME type, media properties, file size, and content hash. Being JSON-serializable, these contracts persist, transfer, and can be inspected across process and environment boundaries independently of the originating model implementation.
The contracts standardize artifact identification, provenance, references, and run organization while preserving native encodings and modality-specific semantics across media, geometry, dynamic 4D representations, actions, and state traces. At the run level, execution_plan.json specifies intended stages and sample identifiers; requests.jsonl and results.jsonl retain corresponding requests and outcomes; sample_ledger.jsonl tracks per-stage status and failures; artifacts.jsonl indexes materialized outputs; and run_manifest.json records the resolved run identity, configuration, runtime context, and output locations. Failed requests remain explicit in result and ledger records, preserving traceability and coverage accounting. This persistent record structure decouples artifact inspection and evaluation from model execution, enabling outputs to be transferred, rescored, compared, and audited without reloading the originating model. It also distinguishes structural conformance, artifact availability, qualitative validity, and benchmark eligibility as separate claims requiring appropriate evidence.
Evaluation and Evidence
WorldFoundry treats evaluation as an artifact-centered process, allowing scoring to proceed independently of the originating model runtime once the required outputs are materialized. The evaluation pipeline resolves the model, benchmark, metrics, data source, runtime profile, and output directory, and maps task or dataset entries to GenerationRequest records with stable sample identifiers, split assignments, and input keys. It supports scoring existing GenerationResult records, generating outputs for registered metrics, executing a complete model–benchmark evaluation, and applying benchmark-specific evaluators to existing artifact directories. Metrics and benchmarks carry distinct responsibilities: metrics compute per-sample measurements and aggregate statistics from compatible artifacts, spanning semantic alignment, quality, distributional similarity, perceptual comparison, and editing or layout assessment, whereas benchmarks additionally prescribe data, sample coverage, preprocessing, artifact layout, evaluator assets, judge configuration, and aggregation rules, with benchmark runners implementing protocol-specific logic and normalizing external evaluator outputs into a shared result structure. Dependencies and evaluator checkpoints are declared explicitly, and unavailable resources yield explicit skipped or failed states rather than undeclared substitutions.
The evidence layer records evaluation outcomes and the conditions under which they support scientific comparisons. Each evaluation preserves its execution plan, configuration, environment and code revision, request and result records, sample coverage, stage-level errors, metric outputs, artifact references, and unresolved blockers. Per-sample measurements are stored in metrics/per_sample.jsonl, aggregate values and failed or skipped sample identifiers in metrics/summary.json, and run metadata and a reproducibility fingerprint in run_manifest.json. These records inform scorecard.json, the authoritative machine-readable record of score validity, provenance, protocol fidelity, coverage, and comparison eligibility. Derived summary.json and report.md files provide concise machine-readable and human-readable views consistent with the scorecard. Evidence is classified as diagnostic, benchmark-comparable, or an exact benchmark reproduction according to fidelity to the prescribed generation settings, data, evaluator, runtime, and reference configuration. Importing official results validates result normalization, while a bounded evaluator run demonstrates integration operability; neither alone establishes complete benchmark reproduction. Leaderboard eligibility further requires prescribed data and coverage, a verified evaluator, compatible runtime evidence, successful metric computation, and no disqualifying blockers. These distinctions constrain reported claims to the available evidence, preventing successful execution or score production from being mistaken for benchmark validity.
Model and Benchmark Zoo
The Model Zoo is a manifest-driven registry that links model discovery to explicit, auditable integration states. Entries fall into five operational categories: video models for conditional generation, editing, animation, and audiovisual synthesis; generative world models for interactive environments, navigation, and simulation; 3D and 4D models for reconstruction, depth estimation, and static or dynamic scene representations; vision-language-action, vision-action, and world-action models for embodied prediction and control; and provider-hosted models accessed remotely. These categories reflect integration requirements rather than mutually exclusive capabilities. Each entry has a stable identifier and optional aliases, with a manifest documenting official sources, licensing and access conditions, tasks and variants, checkpoint references, input and artifact contracts, runtime bindings, hardware requirements, and blockers. Linked records specify runtime contracts, execution backends, and readiness; environment requirements, including Python and CUDA versions, package constraints, validation imports, and environment isolation; and the concrete Pipeline implementation. Evidence records distinguish source-reported results from those obtained through WorldFoundry, while checkpoints, credentials, and other protected or machine-dependent assets remain outside the repository and are resolved through explicit references. Catalog inclusion therefore establishes discoverability, not executable support: source availability, implementation status, asset readiness, runtime validation, and execution evidence are tracked independently, and entries remain planned or blocked until the relevant requirements are met.
The Benchmark Zoo is a parallel registry that links benchmark discovery to task construction, evaluator execution, result normalization, and evidence reporting, covering video quality, semantic alignment, physical and dynamical consistency, camera and spatial behavior, 3D and 4D understanding, interactive world modeling, safety, memory, and embodied control. Each entry has a stable identifier and optional aliases and documentation of target domains, official sources, datasets and splits, task definitions, sample coverage, artifact requirements, metrics, evaluator assets, judge requirements, result formats, runtime profiles, and blockers. Task manifests define how dataset records or prompts become reproducible sample requests, while benchmark manifests specify the evaluation protocol and the supported execution path that most closely follows it. This path may use an in-framework evaluator, an external official runtime, an official-result importer, a hosted judge, or a simulator interface. Integration verification is recorded separately from eligibility for benchmark comparison: importing a schema-conformant official result verifies parsing and normalization, whereas reproducing an official evaluation also requires the prescribed data, artifact coverage, dependencies, assets, credentials, runtime conditions, and protocol settings. Models and benchmarks are paired only when their declared task semantics and artifact requirements are compatible. Each pairing is then classified as diagnostic, benchmark-comparable, or eligible for an exact-reproduction claim based on provenance, coverage, protocol fidelity, and verification evidence.
Reading the table. Catalog inclusion, runnable integration, and runner/evidence verification are distinct maturity claims. The live Model recipes, Benchmark Hub, and Metrics registry remain authoritative for updates.
Empirical Validation and Analysis
We evaluate WorldFoundry using representative models for video generation, interactive world modeling, 3D and 4D generation, and embodied action prediction and control. The evaluation first verifies whether the framework preserves native input and output semantics and produces results consistent with native implementations under matched conditions. It then examines whether equivalent requests issued through the CLI, Python API, and web interface yield consistent configurations and execution behavior. Latency and GPU memory overhead are profiled last, together with the effort required to integrate a new model.
Experimental Setting
Experimental Setting
Experiments are conducted on 128 NVIDIA A100/H100 GPUs with 80 GB of memory each; evaluation on other hardware is left to future work. As model families differ in objectives, inputs, outputs, and execution semantics, we evaluate infrastructure capabilities rather than ranking performance across domains. Specifically, we assess whether the framework resolves declarative catalog entries into executable configurations, preserves model-native semantics across interfaces, stores heterogeneous outputs as persistent artifacts, and supports reported claims with traceable evidence. All workflows share a lifecycle of discovery, preparation, execution, inspection, and evaluation while retaining domain-specific inputs, controls, runtime requirements, and output representations. The evaluation examines whether this shared infrastructure supports reproducible and auditable workflows without imposing a common task, representation, or evaluation metric.
Experimental Results
We report results by model family: video generation (§ 6.3.1), interactive world models (§ 6.3.2), 3D and 4D generation (§ 6.3.3), and embodied action generation (§ 6.3.5). For each family, we describe the task setting, inputs and outputs, and representative systems executed through WorldFoundry. We then present inference validation results with visualizations of the generated outputs and summarize the empirical insights obtained from these executions, comparing qualitative behavior with published findings where noted.
Video Generation.
For video generation, we consider text-to-video synthesis, image-to-video synthesis, and video-to-video editing. These tasks assess whether a model can generate plausible visual dynamics, preserve subject identity and scene structure over time, and satisfy task-specific conditions, capabilities central to modeling how the visual world evolves. Successful generation therefore requires both fidelity to the supplied conditions and coherent temporal progression in appearance, spatial relationships, and motion. Depending on the task, inputs consist of a textual description alone or of visual conditions, such as reference images or source videos, paired with instructions specifying the desired content, motion, or transformation; additional reference modalities and structural constraints are supplied where the model supports them. Representative systems include CogVideoX [31], Vchitect-2.0 [32], HunyuanVideo 1.5 [33], LTX-2.5 [34], Wan2.1 [35], Wan2.1 VACE [36], Wan2.2 [37], LingBot-Video [38], and MiniMax H3 [39], which span diverse conditioning mechanisms and generation pipelines. WorldFoundry preserves each system’s native input processing and inference settings, so that results reflect its intended operation. Figure 7 shows representative text-to-video results from LTX-2.5, Wan2.2, and MiniMax H3, covering both camera motion through largely static scenes and object-centric dynamics such as running animals and a hovering bird.
Figure 7. (a) Ltx2.5 — Generated after the prompt “Zebras run across a grassy savanna, kicking up dust in warm daylight.” (b) Ltx2.5 — Generated after the prompt “Waves wash along a sunlit stone seawall as the camera glides forward.” (c) Wan 2.2 — Generated after the prompt “Pan across a sandy, rocky beach with gentle waves under a clear blue sky.” (d) Wan 2.2 — Generated after the prompt “Pan across blue water and misty mountain ridges, with trees in the foreground.” (e) Minimax-h3 — Generated after the prompt “Pan along a tropical beach with palms, seabirds on rocks, and a small boat.” (f) Minimax-h3 — Generated after the prompt “A hummingbird takes off and hovers beside sunlit leaves, rapidly beating its wings.” Demonstration of inference results from representative video generation models.
Qualitative analysis highlights distinct strengths across generation tasks. In text-conditioned synthesis, CogVideoX and Vchitect-2.0 capture scene composition and broad motion patterns, but complex motion and multi-object interactions expose inconsistencies in identity, fine detail, and spatial relationships. FLAG3D++ [40] provides detailed and professional language instructions to describe how to perform a specific activity, as well as the corresponding accurate and dense 3D human pose captured from an advanced MoCap system to handle complex activities and large movements. HunyuanVideo 1.5 and Wan2.1 produce coherent short sequences, particularly with image conditioning, although appearance preservation does not consistently translate into plausible motion under occlusion or substantial deformation. Published results for Wan2.2 [37] report improvements in motion generation and aesthetic control over Wan2.1, extending the range of visually convincing sequences. LingBot-Video provides a complementary advance: its reported post-training comparisons [38] show reduced object deformation, interpenetration, and premature object release, indicating gains in physical interaction beyond perceptual quality alone. MiniMax H3 broadens reference-based generation and editing, with official demonstrations combining appearance, motion, and audio cues from different sources [39]; these examples demonstrate compositional flexibility but do not establish reliable physical dynamics. Together, these observations suggest that progress unfolds along several distinct dimensions: visual refinement, preservation of conditioning information, and consistency of the depicted events. A sequence may remain visually smooth and recognizable while violating contact relations or failing to complete the requested transformation, which makes event-level consistency a necessary complement to frame-level quality in video-generation analysis.
Interactive World Model.
For interactive world modeling, we consider navigation, camera-controlled exploration, and action- or instruction-driven interaction within generated environments. These tasks assess whether a model can maintain a coherent environment as actions unfold, retain information beyond the immediate visual context, and carry the consequences of earlier interactions into subsequent predictions. Unlike offline video synthesis, the model produces an evolving observation stream conditioned on successive interventions, allowing later actions to respond to previously generated observations. Inputs comprise an initial image or image sequence, the available interaction history, and model-supported controls, including directional movement, camera rotation, discrete game actions, and textual instructions. Successful generation therefore requires both immediate responsiveness to controls and persistent consistency in scene geometry, object identity, and interaction outcomes, including when previously observed regions are revisited. Representative systems include the Matrix-Game series up to the recent 3.5 release [41], LingBot-World 1.0 and 2.0 [42, 43], Hunyuan GameCraft and GameCraft-2 [44, 45], YUME-1.5 [46], HY-WorldPlay [47], SCOPE [48], SANA-WM [49], and Echo-WM [50]. WorldFoundry preserves each system’s native action vocabulary, history-conditioning mechanism, and generation schedule, and records controls, observations, and runtime configurations to support reproducible trajectory-level analysis. Figure 8 shows 30-second rollouts from five representative systems driven by directional movement and mouse-based camera control across diverse scenes.
Figure 8. Demonstration of inference results from representative world models.
Qualitative comparison highlights progress along two complementary dimensions: scene persistence and interaction expressiveness. Matrix-Game 2.0 [51] enables responsive navigation but exhibits appearance drift over extended rollouts, whereas LingBot-World 1.0, Hunyuan GameCraft, YUME-1.5, and HY-WorldPlay produce visually coherent navigation sequences in the examined cases. Recent memory-augmented systems address consistency beyond the immediate context: Matrix-Game 3.0 [52] introduces long-horizon memory, while published results for Matrix-Game 3.5 [41] and ReWorld [53] demonstrate scene recall through geometry-aware retrieval and pose-indexed landmarks, respectively. These advances make revisitation more informative than uninterrupted forward motion for assessing whether a model retains previously observed content. In parallel, LingBot-World 2.0 [43] and Hunyuan-GameCraft-2 [45] broaden interaction to diverse actions and instruction-driven events, while SCOPE [48] demonstrates localized firing and reloading responses alongside continuous movement and camera control. However, scene recall and action responsiveness are distinct from persistent interaction outcomes: recovering a familiar view does not establish that an opened door remains open or that a displaced object retains its new position after leaving the field of view. The resulting evaluation priority is therefore to distinguish memory of scene appearance from memory of changes caused by actions. Reliable interactive world modeling requires both: stable recovery of unchanged content and consistent propagation of action-dependent state changes across subsequent observations.
3D & 4D Generation.
For 3D and 4D generation, we consider static scene reconstruction, scene synthesis, and dynamic scene modeling. These tasks assess whether a model can recover or generate spatial structure, maintain consistency across viewpoints, and represent changes over time. Successful reconstruction requires geometry and camera estimates consistent with the observed views, while conditional synthesis requires spatially coherent content that follows the supplied visual or textual conditions. Dynamic scene modeling additionally requires distinguishing camera motion from scene motion and preserving temporal correspondence through deformation, occlusion, and reappearance. Depending on the task, inputs comprise single images, multi-view observations, video sequences, or textual descriptions, and outputs include point maps, camera parameters, point clouds, meshes, Gaussian representations, and time-dependent geometry. Representative systems include DUSt3R [54], CUT3R [55], VGGT [56], VGGT-Omega [57], Splatt3R [58], and FlashWorld [59] for spatial reconstruction or synthesis, alongside MonST3R [60], NeoVerse [61], and Lyra [62] for complementary video-based and spatiotemporal workflows. Evaluation criteria include camera consistency, reconstruction completeness, novel-view quality, inference efficiency, and, where reference geometry is available, geometric accuracy, with additional attention to temporal stability in dynamic scenes. WorldFoundry preserves each method’s native inputs and representations, organizes comparisons within compatible task families, and records configurations, rendered views, and exported artifacts for reproducible spatial and temporal analysis. Figure 9 compares VGGT, VGGT-Omega, DUSt3R, and CUT3R on the same indoor scene, rendering each reconstruction from a common sequence of viewpoints.
Figure 9. Demonstration of inference results from representative 3D scene generation models.
Qualitative analysis highlights a distinction between reconstructing observed structure and generating plausible spatial completion. DUSt3R recovers geometry through pairwise point-map prediction, while VGGT jointly estimates camera parameters, depth, point maps, and point tracks from multiple views; VGGT-Omega further develops this feed-forward reconstruction paradigm. Splatt3R complements geometric reconstruction with Gaussian appearance attributes for novel-view rendering, although convincing rendered views do not establish accurate geometry in sparsely observed or occluded regions. FlashWorld enables efficient scene synthesis through direct Gaussian generation, while Lyra transfers video-generation priors into feed-forward 3D and 4D reconstruction, illustrating how learned priors can supply plausible content beyond the available observations. In dynamic scenes, MonST3R accommodates scene motion when estimating geometry and cameras, and NeoVerse extends monocular reconstruction to time-varying scenes and novel camera trajectories. The central ambiguity is whether image changes arise from camera motion, object motion, or newly visible surfaces: temporal smoothness alone cannot establish correct geometric correspondence. For extended exploration, Lyra 2.0 [63] reports improved consistency through geometry-guided retrieval and training that mitigates accumulated generation drift. Together, these developments motivate distinguishing three properties: agreement with observed evidence, cross-view consistency of inferred content, and temporal persistence of scene structure. A representation can be visually convincing and internally consistent while remaining inaccurate in unobserved regions; reconstructing a dynamic sequence likewise does not establish the ability to predict its evolution under alternative actions.
Embodied Action Generation.
Embodied Action Generation.
For embodied action generation, we consider visual policy learning, language-conditioned manipulation, and control informed by predicted environmental dynamics. These tasks assess whether a model can translate observations and task specifications into executable actions, coordinate perception with control, and adapt to feedback during execution. Inputs comprise visual observations and proprioceptive states, supplemented by natural-language instructions or predicted future observations where supported. Outputs comprise discrete actions, continuous control commands, or action sequences expressed in the target embodiment’s native action space. The integrated systems follow three complementary paradigms: Vision-Action (VA) policies, which generate actions from visual and robot-state observations; Vision-Language-Action (VLA) models, which additionally incorporate linguistic instructions; and World-Action Models (WAMs), which couple action generation with predictions of environmental evolution. Representative integrations include Action Chunking Transformer [64], OpenVLA [65], RoboFlamingo [66], MolmoAct2 [67], GigaBrain-0 [68], Being-H0.5 [69], StarVLA [19], LingBot-VLA 1.0 and 2.0 [70, 71], and LingBot-VA 1.0 and 2.0 [72, 73]. Evaluation combines offline action prediction with closed-loop execution in compatible manipulation environments, including LIBERO and RoboTwin [74, 75]; criteria include task completion, instruction adherence, control accuracy, trajectory smoothness, execution latency, and recovery from intermediate errors. WorldFoundry preserves each system’s observation history, camera configuration, action coordinates, normalization statistics, control frequency, and action-chunk horizon, with comparisons restricted to compatible embodiments and task protocols. Figure 10 shows closed-loop LIBERO-10 episodes of OpenVLA executed through WorldFoundry: the policy completes a single-object pick-and-place task but reaches the step limit on two multi-stage tasks, illustrating why closed-loop completion, rather than offline action accuracy, is the relevant measure.
Figure 10. Representative closed-loop embodied results. OpenVLA executed through WorldFoundry on LIBERO-10, with one recorded episode per task. The policy completes task (a) within 179 steps, whereas tasks (b) and (c) reach the 520-step limit before completion.
Published empirical findings indicate that semantic grounding, embodiment transfer, and predictive control contribute to embodied competence through distinct mechanisms. SuperMimic [76] demonstrates that a shared differentiable world model can efficiently scale humanoid whole-body control through short imagined rollouts and analytic policy gradients, substantially reducing the computational cost of policy learning. OpenVLA [65] and RoboFlamingo [66] demonstrate that pretrained vision-language representations can support language-conditioned manipulation, although transfer remains dependent on the target embodiment and adaptation data. MolmoAct2 [67] combines embodied reasoning with a dedicated action expert, exposing a trade-off between additional spatial reasoning and control latency. GigaBrain-0 [68] improves generalization through world-model-generated training data, whereas Being-H0.5 [69] aligns human and robot motion within a shared action representation. StarVLA [19] provides complementary policy variants for examining how action decoding and training choices affect execution, rather than defining a single control mechanism. LingBot-VLA 1.0 [70] demonstrates effective adaptation across robot platforms, while LingBot-VLA 2.0 [71] extends this approach to whole-body control and incorporates future semantic and geometric prediction as auxiliary supervision. LingBot-VA 1.0 [72] instead couples video and action prediction within a causal sequence model; LingBot-VA 2.0 [73] further introduces action-aligned visual representations and asynchronous inference that incorporates the latest observations during execution. These distinctions suggest that the value of world modeling depends on where prediction enters the control pipeline: as training-data augmentation, representation supervision, or an online component of action generation. In particular, plausible future observations are useful only insofar as they support executable actions and timely correction when reality departs from prediction. The decisive evidence is therefore improved closed-loop completion and recovery under comparable data, embodiment, and latency constraints, rather than predictive visual quality or offline action accuracy alone.
Limitations and Future Work
Although WorldFoundry provides a unified operational foundation for diverse generative world-modeling systems, its broad coverage encompasses different levels of execution maturity. Catalog inclusion, runnable integration, local asset readiness, official runtime validation, and benchmark eligibility remain distinct states, reflecting the external checkpoints, licensed datasets, hosted services, simulator assets, and dedicated environments required by individual models and benchmarks. Differences in architecture and hardware requirements also prevent a single optimization strategy from benefiting every video, spatial, interactive, and embodied model equally. In addition, the present framework focuses on inference, interaction, artifact management, and evaluation. A general training infrastructure remains a direction for future development, and upstream training code is intentionally separated from current runtime packages. This scope provides a consistent and auditable foundation for experimentation, while leaving opportunities to extend lifecycle reproducibility as public datasets and reproducible training recipes become available across domains.
The inference core already exposes a broad set of optimization mechanisms: configurable attention backends, including FlashAttention 2 [77] and 3 [78], SageAttention [79], xFormers, and SDPA; Triton kernel registration; compilation and inference caching; NVFP4 quantization where supported; hardware-aware GPU selection; and multi-GPU context and sequence parallelism. Their effects, however, have not yet been characterized systematically across model families, and validation has primarily used A100 and H100 GPUs; limited access to B200- and B300-class systems constrains assessment on newer accelerators. Future work will combine broader hardware validation with runtime profiling, automatic backend selection, improved compilation and cache reuse, fidelity-aware quantization, and more efficient memory management and distributed execution. We will also integrate dedicated serving engines through modular backend adapters and runtime profiles that preserve native input semantics, artifact contracts, and evaluation protocols. Improvements will be assessed by end-to-end latency, throughput, memory use, scaling efficiency, and reproducibility, since kernel-level acceleration alone does not establish application-level gains. An optimization will be adopted only if its outputs remain within a declared fidelity tolerance of the unoptimized route.
Beyond inference optimization, we plan to extend WorldFoundry to support data preparation and training. Initial development will focus on representative baselines and public datasets, reflecting the uneven availability of reproducible training recipes across domains. The extension will support versioned dataset manifests, distributed training, fine-tuning and post-training, checkpoint lineage, experiment recovery, and direct checkpoint registration in the Model Zoo. Existing contracts will link data and optimization configurations to checkpoints, inference artifacts, and benchmark results. Since reproducible training alone does not guarantee consistent deployment behavior, we will explicitly align preprocessing, conditioning, normalization, coordinate conventions, and checkpoint interpretation across training and inference. Reusable configurations, guided setup, and actionable diagnostics will ease adoption while retaining domain-specific controls. Training, execution, and evaluation will remain distinct responsibilities within an auditable research lifecycle, preserving architectural flexibility and traceability from data preparation to reported results.
Conclusion
We introduce WorldFoundry as open-source infrastructure built on an operational definition of generative world models as systems with persistent state, intervention-conditioned dynamics, and observational feedback. This definition shifts the criterion for world modeling from the quality of individual outputs to the behavior of the loop that produces them: a generated observation counts as evidence of a world model only insofar as it is constrained by a state that past interventions have altered and that new observations can revise. WorldFoundry relies on this view across video, 3D and 4D, interactive world, and embodied action systems, cataloging 270 model entries, 74 benchmarks, and 62 metrics under shared contracts that preserve native semantics. By recording catalog inclusion, executability, and validated performance as distinct claims, it keeps every reported result traceable to the artifacts and conditions that support it. Our cross-family analysis indicates that visual fidelity, scene consistency, and action responsiveness, although complementary, cannot by themselves certify world modeling; what current evaluations seldom measure is whether intervention effects persist and whether incoming observations correct the state. Future work will develop evaluations that directly probe intervention persistence and feedback assimilation, alongside broader inference optimization, hardware validation, and reproducible training with explicit training–inference consistency. By connecting capability definitions, executable workflows, and traceable evidence, WorldFoundry provides a shared foundation for measuring progress toward world intelligence.
References
Appendix
Author Contributions
OpenEnvision Team: Project Team.
Juanxi Tian (NTU): Project Leader & Core Contributor. Leads the project, designs and implements WorldFoundry, and writes the technical report.
Jiadong Hou (NTU): Core Contributor. Co-develops WorldFoundry and coordinates its updates.
Baiang Li (Princeton): Contributor. Reviews and helps refine WorldFoundry’s video generation pipeline.
Xin Lin (UCSD): Contributor. Reviews and helps refine WorldFoundry’s 3/4D generation pipeline.
Zhengbo Jiao (HKUST): Contributor. Tests WorldFoundry’s pipelines and supports outreach for the framework.
Weijia Zhang (Yale): Contributor. Tests WorldFoundry’s pipelines and supports outreach for the framework.
Yeying Jin (NUS): Advisor. Provides advice on writing the technical report.
Truong Nguyen (UCSD): Advisor. Provides advice on writing the technical report.
Haitian Zheng (Adobe): Advisor. Offers advice on WorldFoundry’s design and development.
Zhifei Zhang (Adobe): Advisor. Offers advice on WorldFoundry’s design and development.
Jianfei Yang (NTU): Advisor. Offers advice on WorldFoundry’s design and development.
Ming-Hsuan Yang (UC Merced): Advisor. Offers advice on WorldFoundry’s design and development.
Felix Heide (Princeton): Project Supervisor. Supervises the WorldFoundry project and provides guidance on the technical report.
Wei Xiong (Nvidia): Project Supervisor. Supervises the WorldFoundry project and provides guidance on the technical report.
Enze Xie (Nvidia): Project Supervisor. Supervises the WorldFoundry project and provides guidance on the technical report.
Wanhua Li (NTU): Project Supervisor. Supervises the WorldFoundry project and provides guidance on the technical report.
Supported Models
The WorldFoundry Model Zoo comprises 270 distinct registry entries, organized according to the five families in Table 2: 64 video generation models (including multimodal generation), 109 world models, 34 3D/4D models, 53 embodied models, and 10 hosted API models. Table 3–Table 7 enumerate every entry with its stable registry ID, upstream reference, and readiness status. Checkpoint variants and aliases are consolidated under a single entry unless the registry assigns them separate IDs. Model names link to the corresponding papers, technical reports, official releases, model cards, or API documentation. Where no upstream publication exists, the source is the official release or a registry record.
| Family | Representative integrations | Native tasks |
| MODEL ZOO | Executable systems and native artifacts | |
| Video generation | CogVideoX · HunyuanVideo series · LTX series · MiniMax H3 · SANA Video series · Wan series | Text-, visual-, and audio-conditioned video synthesis; video editing |
| Interactive world models | Matrix-Game series · LingBot-World series · Hunyuan GameCraft series · Cosmos 3 · SANA WM · YUME-1.5 · SCOPE | Action- and camera-conditioned rollouts; navigation and scene interaction |
| 3D & 4D | DUSt3R · VGGT / VGGT-Omega · Splatt3R · FlashWorld · MonST3R · NeoVerse · Lyra | 3D reconstruction and generation; dynamic scene modeling |
| Embodied action | ACT · OpenVLA · Lingbot-VLA / VA · MolmoAct2 · GigaBrain-0 · Being-H0.5 · StarVLA | Action prediction; policy execution; language-conditioned control |
| Hosted APIs | Seedance 2 / 2.5 · Kling API · Runway Gen-4.5 · Sora2 · Veo3 · World Labs API | API-based video and world generation |
| BENCHMARK ZOO | Evaluation protocols and evidence | |
| Domain | Representative benchmarks | Evaluation focus |
| Video generation | VBench series · T2V-CompBench · ChronoMagic-Bench · VideoScore · VideoPhy2 · T2VSafetyBench | Perceptual quality; semantic alignment; compositionality; temporal dynamics; physical plausibility; content safety |
| World models | WorldBench · WorldModelBench · WorldScore · 4DWorldBench · iWorld-Bench · WorldAtlas | Dynamics fidelity; 3D/4D consistency; memory retention; controllability; interactive task performance |
| Embodied AI | AI2-THOR · BEHAVIOR-1K · CALVIN · LIBERO · RoboTwin series · WorldArena series · RoboDojo | Closed-loop task completion in simulated environments |
| METRIC REGISTRY | In-tree scorers and compute APIs | |
| Family | Representative metrics | Measurement focus |
| Scorers & quality | CLIPScore · VQAScore · ITMScore · FaceScore · ArtScore · OpenS2V | Semantic alignment; identity consistency; aesthetic quality; subject-to-video quality |
| Distribution | FID · IS · KID · FVD · CMMD · Clean-FID · JEDi | Image/video distributional fidelity and sample diversity |
| Perceptual pairwise | LPIPS · SSIM / MS-SSIM · PSNR · DINO / DreamSim · mask accuracy · LQS | Perceptual and structural similarity; mask accuracy; layout quality |
| Editing & layout | SemSR · IRS · CAS · manipulation direction · VS similarity · object-wise consistency | Semantic edit success; manipulation direction; object-level consistency |
Table 2. Representative coverage of the WorldFoundry Model Zoo, Benchmark Zoo, and Metric Registry. Entries are illustrative rather than exhaustive or uniformly mature.
| Model | Registry ID | Readiness |
| Allegro | allegro | Runner verified |
| AnimateDiff | animatediff | Runner verified |
| Bernini | bernini | Runner verified |
| Bernini-R 1.3B | bernini-r-1.3b | Runner verified |
| Bernini-R 14B | bernini-r-14b | Runner verified |
| CogVideoX | cogvideox | Runner verified |
| DynamiCrafter | dynamicrafter | Runner verified |
| EasyAnimate | easyanimate | Runner verified |
| Emu3.5 | emu3.5 | Runner verified |
| FastVideo CausalWan2.2 | fastvideo-causal-wan2.2 | Runner verified |
| FramePack | framepack | Runner verified |
| Helios | helios | Runner verified |
| HunyuanVideo | hunyuanvideo | Runner verified |
| HunyuanVideo 1.5 | hunyuanvideo-1.5 | Runner verified |
| HyperFlow for MiniMax-H3 | hyperflow | Runner verified |
| I2VGen-XL | i2vgen-xl | Runner verified |
| JoyAI-Echo 1.5 / Echo-LongVideo | joyai-echo-longvideo | Runner verified |
| Kandinsky 6.0 lite | kandinsky6-lite | Runner verified |
| Kandinsky 6.0 lite-distill | kandinsky6-lite-distill | Runner verified |
| Kandinsky 6.0 lite-pretrain | kandinsky6-lite-pretrain | Runner verified |
| Kandinsky 6.0 pro | kandinsky6-pro | Runner verified |
| Kandinsky 6.0 pro-distill | kandinsky6-pro-distill | Runner verified |
| Kandinsky 6.0 pro-pretrain | kandinsky6-pro-pretrain | Runner verified |
| Krea Realtime Video 14B | krea-realtime-video | Runner verified |
| LingBot-Video | lingbot-video | Runner verified |
| LongCat-Video | longcat-video | Runner verified |
| LongVie | longvie-1 | Runner verified |
| LongVie 2 | longvie-2 | Runner verified |
| LTX-2.5 Distilled | ltx-2.5 | Runner verified |
| LTX-2.x | ltx-2.x | Runner verified |
| LTX-Video | ltx-video | Runner verified |
| MAGI-1 | magi-1 | Runner verified |
| MAGI-2 Preview | magi2-preview | Runner verified |
| MiniMax H3 | minimax-h3 | Runner verified |
| MMAudio | mmaudio | Runner verified |
| Mochi-1 | mochi-1 | Runner verified |
| ModelScopeT2V | modelscope-t2v | Runner verified |
| OmniVinci | omnivinci | Runner verified |
| Open-MAGVIT2 | open-magvit2 | Runner verified |
| Open-Sora | open-sora | Runner verified |
| Open-Sora-Plan | open-sora-plan | Runner verified |
| Pusa VidGen | pusa-vidgen | Runner verified |
| Qwen2.5-Omni | qwen2.5-omni | Runner verified |
| Model | Registry ID | Readiness |
| SAMA | sama-14b | Runner verified |
| Sana | sana | Runner verified |
| Show-o | show-o | Runner verified |
| SkyReels V3 | skyreels-v3 | Runner verified |
| SkyReels-V2 | skyreels-v2 | Runner verified |
| SpatialLadder | spatial-ladder | Runner verified |
| SpatialReasoner | spatial-reasoner | Runner verified |
| Stable Video Infinity 2.0 | stable-video-infinity | Runner verified |
| Step-Video-T2V | step-video-t2v | Runner verified |
| T2V Turbo T2V | t2v_turbo_t2v | Runner verified |
| UniAnimate-DiT | unianimate-dit | Runner verified |
| Vchitect-2.0 T2V | vchitect-2-t2v | Runner verified |
| VideoCrafter | videocrafter | Runner verified |
| VideoCrafter1 I2V | videocrafter1-i2v | Runner verified |
| VideoCrafter1 T2V | videocrafter1-t2v | Runner verified |
| VideoCrafter2 T2V | videocrafter2-t2v | Runner verified |
| VMem | vmem | Runner verified |
| Wan2.1 | wan2.1 | Runner verified |
| Wan2.1 VACE | wan2.1-vace | Runner verified |
| Wan2.2 | wan2.2 | Runner verified |
| ZeroScope | zeroscope | Runner verified |
Table 3. Video generation models (64 entries).
| Model | Registry ID | Readiness |
| Hailuo 2.3 | hailuo-2p3 | Runner verified |
| Kling API | kling-api | Runner verified |
| Luma Ray2 | luma-ray2 | Runner verified |
| Runway Gen-4.5 | runway-gen4p5 | Runner verified |
| Sora2 | sora2 | Runner verified |
| Veo3 | veo3 | Runner verified |
| Wan 2.5 API | wan-2p5 | Runner verified |
| Wan 2.6 API | wan-2p6 | Runner verified |
| Wan 2.7 API | wan-2p7 | Runner verified |
| World Labs API | worldlabs | Runner verified |
Table 7. Hosted API models (10 entries).
Readiness is evidence-aware; all 270 entries are marked Runner verified in this appendix. Verification scope is specific to the corresponding execution route, model variant, and configuration.
| Model | Registry ID | Readiness |
| ABot-World-0-5B-LF | abot-world-0-5b-lf | Runner verified |
| AC3D | ac3d | Runner verified |
| AdaWorld | adaworld | Runner verified |
| AlayaWorld | alayaworld | Runner verified |
| AlayaWorld v1.1 | alayaworld-v1.1 | Runner verified |
| Astra | astra | Runner verified |
| Astronex-World | astronex-world | Runner verified |
| ATI-Wan2.1-14B | ati-wan21-14b | Runner verified |
| BiWM Wan2.1 | biwm-wan21 | Runner verified |
| BiWM Wan2.2 | biwm-wan22 | Runner verified |
| CameraCtrl | cameractrl | Runner verified |
| Causal-Forcing | causal-forcing | Runner verified |
| Causal-rCM | causal-rcm | Runner verified |
| Cosmos Predict 2 | cosmos-predict-2 | Runner verified |
| Cosmos Predict 2.5 | cosmos-predict-2.5 | Runner verified |
| Cosmos Transfer 2.5 | cosmos-transfer-2.5 | Runner verified |
| Cosmos3 | cosmos3 | Runner verified |
| Ctrl-World | ctrl-world | Runner verified |
| DIAMOND | diamond | Runner verified |
| DINO-WM | dino-wm | Runner verified |
| DreamDojo | dreamdojo | Runner verified |
| DreamX-World 5B | dreamx-world-5b | Runner verified |
| DualCamCtrl | dualcamctrl | Runner verified |
| Echo-Infinity | echo-infinity | Runner verified |
| Echo-Memory | echo-memory-context-k1 | Runner verified |
| Echo-WM | joyai-echo-wm | Runner verified |
| EgoWM | egowm | Runner verified |
| Evoke | evoke | Runner verified |
| Gamma-World | gamma-world | Runner verified |
| GEN3C | gen3c | Runner verified |
| Genie Envisioner | genie-envisioner | Runner verified |
| Model | Registry ID | Readiness |
| GigaWorld-0 | giga-world-0 | Runner verified |
| HappyOyster | happyoyster | Runner verified |
| HelixWorld Preview v1 | helixworld | Runner verified |
| HMA | hma | Runner verified |
| Hunyuan Game Craft | hunyuan-game-craft | Runner verified |
| HunyuanWorld 1.0 | hunyuanworld-1 | Runner verified |
| HunyuanWorld-Mirror | hunyuanworld-mirror | Runner verified |
| HunyuanWorld-Voyager | hunyuanworld-voyager | Runner verified |
| HY-World 2.0 | hy-world-2.0 | Runner verified |
| HY-World 2.0 Trajectory Renderer | hyworld-worldgen | Runner verified |
| HY-WorldPlay | hy-worldplay | Runner verified |
| HyDRA | hydra | Runner verified |
| Infinite-World | infinite-world | Runner verified |
| InSpatio World 1.5 | inspatio-world-1p5 | Runner verified |
| InSpatio-World | inspatio-world | Runner verified |
| InSpatio-WorldFM | worldfm | Runner verified |
| IRASim | irasim | Runner verified |
| Kairos Sensenova | kairos-sensenova | Runner verified |
| LeWorldModel | leworldmodel | Runner verified |
| LingBot-World | lingbot-world | Runner verified |
| LingBot-World Base Act | lingbot-world-act | Runner verified |
| LingBot-World-V2 | lingbot-world-v2 | Runner verified |
| LiveWorld | liveworld | Runner verified |
| MagicWorld | magicworld | Runner verified |
| Matrix-Game 1.0 | matrix-game-1 | Runner verified |
| Matrix-Game 2.0 | matrix-game-2 | Runner verified |
| Matrix-Game 3.0 | matrix-game-3 | Runner verified |
| Matrix-Game 3.5 | matrix-game-3.5-first-person | Runner verified |
| MineWorld | mineworld | Runner verified |
| minWM | minwm-hy-action2v | Runner verified |
| MIRA | mira | Runner verified |
| MosaicMem | mosaicmem | Runner verified |
| MotionBricks | motionbricks | Runner verified |
| MotionCtrl | motionctrl | Runner verified |
| MoVerse | moverse | Runner verified |
| MultiWorld | multiworld | Runner verified |
| Navigation World Models | nwm | Runner verified |
| Oasis 500M | oasis-500m | Runner verified |
| OmniForcing | omniforcing | Runner verified |
| Open Dreamer | open-dreamer | Runner verified |
| Pandora | pandora | Runner verified |
| PointWorld | pointworld | Runner verified |
| RollingForcing | rolling-forcing | Runner verified |
| SANA-WM | sana-wm | Runner verified |
| SCOPE | scope | Runner verified |
| Self-Forcing | self-forcing | Runner verified |
| ShotStream | shotstream | Runner verified |
| SimWorld | simworld | Runner verified |
| Solaris | solaris | Runner verified |
| SolarWM | solarwm | Runner verified |
| Spatia | spatia | Runner verified |
| StarWM | starwm | Runner verified |
| TesserAct | tesseract | Runner verified |
| Uni3C | uni3c | Runner verified |
| Unified World Models | uwm | Runner verified |
| UniWorld-View | uniworld-view | Runner verified |
| VerseCrafter | versecrafter | Runner verified |
| VGGT-World | vggt-world | Runner verified |
| Vid2World | vid2world | Runner verified |
| Model | Registry ID | Readiness |
| Wan2.1-Fun V1.1 1.3B Control Camera | wan21-fun-1p3b-cam | Runner verified |
| Wan2.1-Fun V1.1 14B Control Camera | wan21-fun-14b-cam | Runner verified |
| Wan2.2-Fun 5B Control Camera | wan22-fun-5b-cam | Runner verified |
| Wan2.2-Fun A14B Control Camera | wan22-fun-a14b-cam | Runner verified |
| Warp-as-History | warp-as-history | Runner verified |
| WildDet3D | wilddet3d | Runner verified |
| WildWorld | wildworld | Runner verified |
| World Labs Marble 1.1 | worldlabs-marble-1.1 | Runner verified |
| WorldCam | worldcam | Runner verified |
| WorldCrafter | worldcrafter | Runner verified |
| WorldCrafter Base | worldcrafter-base | Runner verified |
| WorldCrafter Fast | worldcrafter-fast | Runner verified |
| WorldGrow | worldgrow | Runner verified |
| WorldMem | worldmem | Runner verified |
| WorldPlay2 | worldplay2 | Runner verified |
| WoW | wow | Runner verified |
| XGEN-JING Flash v1 | xgen-jing | Runner verified |
| YUME (1.5 paper) | yume | Runner verified |
| Zing-0.5 | zing | Runner verified |
Table 4. World models (109 entries).
| Model | Registry ID | Readiness |
| CUT3R | cut3r | Runner verified |
| Depth Any Panoramas | dap | Runner verified |
| Depth Anything 3 | depth-anything-v3 | Runner verified |
| Depth Anything 3 Metric Prior | depth-anything-v3-prior | Runner verified |
| Depth Anything V1 | depth-anything-v1 | Runner verified |
| Depth Anything V2 | depth-anything-v2-prior | Runner verified |
| DUSt3R | dust3r | Runner verified |
| Déjà View Looping Transformer | dvlt | Runner verified |
| FantasyWorld | fantasyworld | Runner verified |
| FlashWorld | flashworld | Runner verified |
| GeoCalib | geocalib-prior | Runner verified |
| Infinite VGGT | infinite-vggt | Runner verified |
| LagrNVS | lagernvs | Runner verified |
| LingBot-Map | lingbot-map | Runner verified |
| LoGeR | loger | Runner verified |
| Lyra (2.0 paper) | lyra | Runner verified |
| Metric3D | metric3d-prior | Runner verified |
| MonST3R | monst3r | Runner verified |
| MVDiffusion | mvdiffusion | Runner verified |
| NeoVerse | neoverse | Runner verified |
| Pi3 | pi3 | Runner verified |
| pixelSplat | pixelsplat | Runner verified |
| Prior Depth Anything | prior-depth-anything | Runner verified |
| ReCamMaster | recammaster | Runner verified |
| Segment and Track Anything | track-anything-prior | Runner verified |
| Splatt3R | splatt3r | Runner verified |
| Stable Virtual Camera | stable-virtual-camera | Runner verified |
| UniDepth V2 | unidepth-v2-prior | Runner verified |
| UniK3D | unik3d-prior | Runner verified |
| VGGT | vggt | Runner verified |
| VGGT-Omega | vggt-omega | Runner verified |
| Video Depth Anything | video-depth-anything-prior | Runner verified |
| WonderJourney | wonderjourney | Runner verified |
| Model | Registry ID | Readiness |
| WorldGen | worldgen | Runner verified |
Table 5. 3D and 4D models (34 entries).
| Model | Registry ID | Readiness |
| A1 | a1 | Runner verified |
| ABot-M0 | abot-m0 | Runner verified |
| Action Chunking Transformer | act | Runner verified |
| AHA-WAM | ahawam | Runner verified |
| Being-H0.5 | being-h05 | Runner verified |
| Being-H0.7 | being-h07 | Runner verified |
| CogACT | cogact | Runner verified |
| DB-CogACT / Dexbotic | db-cogact | Runner verified |
| Dexora 1B | dexora-1b | Runner verified |
| Diffusion Policy | diffusion-policy | Runner verified |
| DM0 | dm0 | Runner verified |
| EventVLA | eventvla | Runner verified |
| FastWAM | fastwam | Runner verified |
| Galaxea G0Plus | galaxea-vla | Runner verified |
| GigaBrain-0 | giga-brain-0 | Runner verified |
| GigaWorld-Policy-0.5 | giga-world-policy-0.5 | Runner verified |
| GO-1 | go1 | Runner verified |
| H-RDT | h-rdt | Runner verified |
| Hy-Embodied-0.5-VLA | hy-embodied-vla | Runner verified |
| InternVLA-A1-3B | internvla-a1 | Runner verified |
| LaST-R1 | last-r1 | Runner verified |
| Latent Action Pretraining | lapa | Runner verified |
| LDA-1B | lda-1b | Runner verified |
| LingBot-VA | lingbot-va | Runner verified |
| LingBot-VLA | lingbot-vla | Runner verified |
| LingBot-VLA 2.0 | lingbot-vla-v2 | Runner verified |
| Mem-0 | mem-0 | Runner verified |
| MME-VLA | mme-vla | Runner verified |
| MolmoAct2 | molmoact2 | Runner verified |
| MolmoBot | molmobot | Runner verified |
| NVIDIA DreamZero | dreamzero | Runner verified |
| NVIDIA GR00T | gr00t | Runner verified |
| Octo | octo | Runner verified |
| OpenPI | openpi | Runner verified |
| OpenVLA | openvla | Runner verified |
| OpenVLA-OFT | openvla-oft | Runner verified |
| pi0 | pi0 | Runner verified |
| pi0-FAST | pi0-fast | Runner verified |
| pi0.5 | pi05 | Runner verified |
| RDT-1B | rdt-1b | Runner verified |
| RISE | rise | Runner verified |
| RoboFlamingo | roboflamingo | Runner verified |
| RT-1 | rt-1 | Runner verified |
| SmolVLA | smolvla | Runner verified |
| Spatial-Forcing | spatial-forcing | Runner verified |
| Spirit-VLA v1.5 | spirit-v1.5 | Runner verified |
| StarVLA | starvla | Runner verified |
| TinyVLA | tinyvla | Runner verified |
| VLANeXt | vlanext | Runner verified |
| X-VLA | xvla | Runner verified |
| X-WAM | x-wam | Runner verified |
| Model | Registry ID | Readiness |
| Xiaomi-Robotics-0 | xiaomi-robotics-0 | Runner verified |
| Xiaomi-Robotics-1 | xiaomi-robotics-1 | Runner verified |
Table 6. Embodied models (53 entries).
Supported Benchmarks
The WorldFoundry Benchmark Hub documentation lists 74 benchmarks. Following the evaluation domains in Table 2, we organize them into 23 embodied AI, 33 video generation, and 18 world model benchmarks. Table 8–10 summarize each benchmark’s evaluation focus and reported status. Benchmark names link to their sources.
| Benchmark | Evaluation focus | Status |
| AI2-THOR | Interactive household navigation and manipulation | Integrated & Verified |
| BEHAVIOR-1K | 1,000 household activities and BDDL goals | Integrated & Verified |
| BridgeData V2 | Real robot trajectories; language and goal image conditioning | Integrated & Verified |
| CALVIN | Language instructed manipulation chains (length 1–5) | Integrated & Verified |
| Kinetix | Open ended 2D physics control tasks | Integrated & Verified |
| LARYBench | Latent action extraction, classification, and regression | Integrated & Verified |
| LIBERO | Spatial, Object, Goal, and LIBERO-100 lifelong manipulation | Integrated & Verified |
| LIBERO-Mem | Object motion, sequence, relation, and occlusion memory | Integrated & Verified |
| LIBERO-Plus | Seven perturbation factors; robustness at five difficulty levels | Integrated & Verified |
| LIBERO-Pro | Object, configuration, instruction, task, and environment robustness | Integrated & Verified |
| ManiSkill | GPU parallel SAPIEN manipulation skill families | Integrated & Verified |
| ManiSkill2 | 20 SAPIEN manipulation task families; multiple embodiments | Integrated & Verified |
| Meta-World | 50 manipulation tasks; multitask and meta learning suites | Integrated & Verified |
| MiKASA-Robo | Object, spatial, sequential, and capacity memory | Integrated & Verified |
| MolmoSpaces-Bench | Pick and place variants; stage and final task success | Integrated & Verified |
| RLBench | 100 vision guided manipulation tasks and procedural variations | Integrated & Verified |
| RoboCasa | Atomic and long horizon kitchen manipulation | Integrated & Verified |
| RoboCerebra | Long horizon manipulation with memory and disturbances | Integrated & Verified |
| RoboMME | Counting, permanence, reference, and imitation memory | Integrated & Verified |
| RoboTwin 2.0 | 50 dual arm tasks; clean and randomized settings | Integrated & Verified |
| SimplerEnv | Real to simulation evaluation on Google Robot and WidowX | Integrated & Verified |
| VLABench | Language conditioned manipulation with long horizon reasoning | Integrated & Verified |
| WorldArena | Embodied perceptual quality and functional utility | Integrated & Verified |
Table 8. Supported embodied AI benchmarks (23 entries).
All 74 entries are marked Integrated & Verified in this appendix. Verification scope is specific to the corresponding benchmark recipe and configuration, including its required datasets, checkpoints, judges, and execution environments.
| Benchmark | Evaluation focus | Status |
| AIGCBench | Image to video fidelity, alignment, motion, and consistency | Integrated & Verified |
| CameraBench | Camera motion classification, VQA, retrieval, and captioning | Integrated & Verified |
| Benchmark | Evaluation focus | Status |
| ChronoMagic-Bench | Time lapse coherence and metamorphic amplitude | Integrated & Verified |
| DEVIL Dynamics | Dynamics range, controllability, and quality | Integrated & Verified |
| EvalCrafter | Visual quality, alignment, motion, and temporal consistency | Integrated & Verified |
| FETV | Fine grained content, attribute control, and prompt complexity | Integrated & Verified |
| GenAI-Bench | Human preferences for image/video generation and editing | Integrated & Verified |
| IPV-Bench | Generation and recognition of impossible events | Integrated & Verified |
| LikePhys | Likelihood preference for valid versus impossible physics | Integrated & Verified |
| MemoBench | Visible–disappeared–reappeared visual memory | Integrated & Verified |
| MIND | Interactive memory consistency and action control | Integrated & Verified |
| MiraBench | Long video consistency, motion, quality, and caption alignment | Integrated & Verified |
| PhyEduVideo | Physics education across six scientific domains | Integrated & Verified |
| PhyFPS-Bench-Gen | Physical FPS alignment and temporal stability (100 prompts) | Integrated & Verified |
| PhyGenBench | Physical commonsense and adherence to physical laws | Integrated & Verified |
| PhyGround | Criteria grounded physics in text/image to video generation | Integrated & Verified |
| Physics-IQ Original | Five second real physics continuation; original protocol | Integrated & Verified |
| Physics-IQ Verified | Physics continuation under the distinct verified protocol | Integrated & Verified |
| PhysVidBench | Caption based physical commonsense QA and tool use | Integrated & Verified |
| RBench | Robot image to video task and embodiment fidelity | Integrated & Verified |
| T2V-CompBench | Seven compositional axes; attribute/action binding and numeracy | Integrated & Verified |
| T2VSafetyBench | Harmful content and temporal safety risks | Integrated & Verified |
| VBench | 16 dimensions of quality, temporal coherence, and semantics | Integrated & Verified |
| VBench-2.0 | 18 abilities across five intrinsic faithfulness pillars | Integrated & Verified |
| VBench++ | Image to video, long video, and trustworthiness extensions | Integrated & Verified |
| Video-Bench | Human preference aligned quality and condition alignment | Integrated & Verified |
| VideoPhy | Semantic adherence and physical commonsense | Integrated & Verified |
| VideoPhy2 | Action centric physical commonsense and rule judgements | Integrated & Verified |
| VideoScience-Bench | Scientific phenomena, dynamics, and spatiotemporal coherence | Integrated & Verified |
| VideoScore | Learned five aspect video quality/reward scoring | Integrated & Verified |
| VideoVerse | Event level temporal causality and world knowledge | Integrated & Verified |
| Visual Chronometer | Physical FPS prediction on arbitrary videos | Integrated & Verified |
| VMBench | Five perception aligned video motion dimensions | Integrated & Verified |
Table 9. Supported video generation benchmarks (33 entries).
| Benchmark | Evaluation focus | Status |
| 4DWorldBench | Perceptual quality, condition alignment, physics, and 4D consistency | Integrated & Verified |
| Apple-PI | Law grounded perception, formulation, and deduction | Integrated & Verified |
| EWMIBench | Embodied scene, motion, semantic consistency, and diversity | Integrated & Verified |
| HarnessEval-W | Transitions, drift resistance, revisits, and offscreen evolution | Integrated & Verified |
| iWorld-Bench | Interactive generation, trajectory following, and memory | Integrated & Verified |
| Benchmark | Evaluation focus | Status |
| PAWBench | Stochastic dynamics calibration and outcome coverage | Integrated & Verified |
| Physical AI Bench (PAI-Bench) | Generation and conditional generation tracks only | Integrated & Verified |
| SANA-WM 80-Scene Benchmark | 80 scenes per split; minute scale camera control and revisits | Integrated & Verified |
| STEVO-Bench | Control, physics, and unobserved state evolution | Integrated & Verified |
| T2VWorldBench | Six domains of world knowledge (1,200 prompts) | Integrated & Verified |
| WBench | Multi turn quality, setting, interaction, consistency, and physics | Integrated & Verified |
| World-in-World | Closed loop recognition, QA, navigation, and manipulation utility | Integrated & Verified |
| WorldBench | Diagnostic physical video segmentation and text QA | Integrated & Verified |
| WorldModelBench | Instruction following, commonsense, and physics adherence | Integrated & Verified |
| WorldOlympiad | Physical, interaction, and geometry triathlon | Integrated & Verified |
| WorldReasonBench | Process aware reasoning and multidimensional quality | Integrated & Verified |
| WorldScore | Camera conditioned controllability, quality, and dynamics | Integrated & Verified |
| WRBench | Camera control, visual integrity, and persistent state | Integrated & Verified |
Table 10. Supported world model benchmarks (18 entries).
Supported Metrics
Table 11 lists all 62 canonical entries in the WorldFoundry metric registry. The table is compiled from the metric documentation together with the registry and its module declarations, and therefore includes entries absent from the documentation summary, such as laion_aesthetic. Metrics are grouped by use into multimodal/quality scorers (10), image/feature distribution metrics (26), video distribution metrics (3), perceptual/layout metrics (11), editing metrics (7), and existing-result utilities (5). Aliases and variants map to their canonical entries and are not counted separately.
| Canonical metric ID | Assessment and required inputs | Dir. |
| Multimodal scorers and quality (10) | ||
clip_score | CLIPScore: image/video–text embedding alignment | |
vqa_score | VQAScore: affirmative-answer probability for a visual question | |
itm_score | Image/video–text matching with BLIP-2 or related ITM backbones | |
facescore | Learned facial realism and quality reward | |
artscore | Learned artness score for generated images | |
facesim_cur | Subject-to-video face identity similarity with CurricularFace | |
gme_score | GME-Qwen2-VL text–video relevance | |
nexus_score | Subject consistency with YOLO-World detection and GME | |
natural_score | API-based video naturalness rating | |
laion_aesthetic | Linear aesthetic predictor on normalized CLIP image/frame features | |
| Image and feature distribution (26) | ||
inception_score | Classifier confidence and label diversity of generated images | |
fid | Fréchet feature distance; Inception, CLIP, SwAV, frame, and object-crop/SceneFID variants | |
kid | Kernel Inception Distance between image feature sets | |
precision_recall | Feature-manifold fidelity and coverage of generated/reference sets | |
improved_precision_recall | Improved precision/recall and per-image realism score | |
fwd | Fréchet Wavelet Distance between image collections | |
cmmd | CLIP embedding maximum mean discrepancy | |
ppl | Perceptual path length of interpolated generator latents | |
mind | Monge Inception Distance between image feature sets | |
clean_fid | FID with standardized image resizing and preprocessing | |
vendi_score | Effective diversity from a feature similarity matrix | |
rarity_score | Uncommonness of generated samples relative to real features | |
trend | Divergence of fitted truncated generalized-normal feature densities | |
fld | Feature Likelihood Divergence using train, test, and generated features | |
multimodal_mid | Mutual Information Divergence for paired image–text features | |
| Canonical metric ID | Assessment and required inputs | Dir. |
fjd | Fréchet Joint Distance over conditional joint embeddings | |
crosslid | Cross Local Intrinsic Dimensionality of feature collections | |
cfid | Conditional Fréchet Inception Distance from paired embeddings | |
ssd | Semantic Similarity Distance for text–image consistency | |
linear_separability | Latent linear separability from a classifier confusion matrix | |
rke | Rényi kernel entropy mode count and relative entropy on features | |
fdd | Fréchet Denoised Distance using a pretrained denoising autoencoder | |
cis | Conditional Inception Score from condition-bucketed class probabilities | |
rnd | Random network distillation diversity on features or images | |
attribute_sad | Semantic Attribute Distance from precomputed HCS features | |
attribute_pad | Pairwise Attribute Distance from precomputed HCS features | |
| Video distribution (3) | ||
fvd | Fréchet distance between I3D features of reference/generated clips | |
fvmd | Fréchet Video Motion Distance from tracked motion features | |
jedi | Video JEPA feature distance with a polynomial-kernel MMD | |
| Perceptual, reconstruction, and layout (11) | ||
lpips | Learned perceptual distance between reference/generated images | |
ssim | Structural Similarity Index for paired images | |
ms_ssim | Multi-scale structural similarity for paired images | |
psnr | Peak signal-to-noise ratio from paired-image reconstruction error | |
dino_similarity | Cosine similarity of DINO/DINOv2 reference/generated features | |
dreamsim | DreamSim perceptual distance between paired images | |
cpbd | No-reference sharpness from cumulative blur-detection probability | |
fsim | Feature Similarity Index for paired images | |
mask_accuracy | Pixel-wise mask accuracy and intersection-over-union (FCN) | |
object_detection | Phrase-matching object-detection success rate | |
lqs | Layout Quality Score: recall, precision, location, and area consistency | |
| Editing and semantic consistency (7) | ||
semsr | Semantic Shift Rate for trigger/origin/target image triplets | |
irs | Image Realism Score calibrated using real-image statistics | |
cas | Classification Accuracy Score: synthetic-data training, real-data testing | |
manipulation_direction | Alignment of image and text change vectors for an edit | |
vs_similarity | HDGAN visual–semantic similarity of paired embeddings | |
quality_loss | CLIP alignment multiplied by unintended text-presence probability | |
object_wise_consistency | Object-wise IoU and detection success against guidance boxes | |
| Existing-result utilities (5) | ||
artifact_count | Counts artifacts recorded on a generation result | — |
required_artifacts_present | Returns one when all required artifacts are present | |
has_artifact | Parameterized presence check: has_artifact:<name> | |
numeric | Emits numeric scores and timings stored in result metadata | — |
numeric_value | Parameterized numeric field: numeric:<name> | — |
Table 11. Complete supported metric catalog. Metrics are grouped by evaluation use; the underlying registry families comprise 30 distribution, 14 scorer, 13 perceptual, and 5 existing-result entries.
The registry specifies scoring implementations; benchmark scorecard dimensions and official evaluator outputs are defined by their respective benchmark protocols. and denote the preferred direction declared for each assessment; “–” denotes a result utility with no universal quality ordering. Numeric-field direction follows the underlying reported score. Metric IDs link to their sources or implementations. Learned metrics require their declared pretrained backbones, FVD uses I3D weights, JEDi uses compatible video-encoder/probe weights, and natural_score requires an API judge. Precomputed-feature and layout routines instead consume the specified embeddings, probabilities, masks, or boxes. Support for a registry entry does not by itself establish equivalence to every published evaluator configuration.
References
- [1]David Ha and Jürgen Schmidhuber. World Models. arXiv preprint arXiv:1803.10122, 2018. URL https://arxiv.org/abs/1803.10122. 3, 5, 7
- [2]Danijar Hafner, Timothy Lillicrap, Ian Fischer, et al. Learning Latent Dynamics for Planning from Pixels. In International Conference on Machine Learning (ICML), 2019. URL https://arxiv.org/abs/1811.04551. 5, 7
- [3]Danijar Hafner, Jurgis Pasukonis, Jimmy Ba, et al. Mastering diverse control tasks through world models. Nature, 640:647–653, 2025. doi: 10.1038/s41586-025-08744-2. URL https://www.nature.com/articles/s41586-025-08744-2. 3, 5, 7DOI
- [4]Mido Assran, Adrian Bardes, David Fan, et al. V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning. arXiv preprint arXiv:2506.09985, 2025. URL https://arxiv.org/abs/2506.09985. 3
- [5]DataFlow Team, Bohan Zeng, Daili Hua, et al. OpenWorldLib: A Unified Codebase and Definition of Advanced World Models. arXiv preprint arXiv:2604.04707, 2026. URL https://arxiv.org/abs/2604.04707. 3, 5, 10
- [6]Dian Zheng, Ziqi Huang, Hongbo Liu, et al. VBench-2.0: Advancing Video Generation Benchmark Suite for Intrinsic Faithfulness. arXiv preprint arXiv:2503.21755, 2025. URL https://arxiv.org/abs/2503.21755. 4, 5, 6, 9
- [7]Hritik Bansal, Clark Peng, Yonatan Bitton, et al. VideoPhy-2: A Challenging Action-Centric Physical Common-sense Evaluation in Video Generation. arXiv preprint arXiv:2503.06800, 2025. URL https://arxiv.org/abs/2503.06800.
- [8]Dacheng Li, Yunhao Fang, Yukang Chen, et al. WorldModelBench: Judging Video Generation Models As World Models. arXiv preprint arXiv:2502.20694, 2025. URL https://arxiv.org/abs/2502.20694. 6, 9
- [9]Yu Shang, Zhuohang Li, Yiding Ma, et al. WorldArena: A Unified Benchmark for Evaluating Perception and Functional Utility of Embodied World Models. arXiv preprint arXiv:2602.08971, 2026. URL https://arxiv.org/abs/2602.08971. 4, 5, 6, 9
- [10]OpenWorldLib Contributors. OpenWorldLib: Official Implementation and Documentation. Official source repository, 2026. URL https://github.com/OpenDCAI/OpenWorldLib. 5
- [11]Niket Agarwal, Arslan Ali, Maciej Bala, et al. Cosmos World Foundation Model Platform for Physical AI. arXiv preprint arXiv:2501.03575, 2025. URL https://arxiv.org/abs/2501.03575. 5, 10
- [12]NVIDIA. Cosmos: Official Implementation and Documentation. Official source repository, n.d.. URL https://github.com/NVIDIA/cosmos. 5
- [13]NVIDIA. Cosmos Evaluator: Automated Evaluation and Grading for Synthetic Video. Official source repository, n.d.. URL https://github.com/NVIDIA/cosmos-evaluator. 5
- [14]Yue Liao, Pengfei Zhou, Siyuan Huang, et al. Genie Envisioner: A Unified World Foundation Platform for Robotic Manipulation. arXiv preprint arXiv:2508.05635, 2025. URL https://arxiv.org/abs/2508.05635. 5
- [15]Boxiang Qiu, Liliang Chen, Yue Liao, et al. GE-Sim 2.0: A Roadmap Towards Comprehensive Closed-loop Video World Simulators for Robotic Manipulation. arXiv preprint arXiv:2605.27491, 2026. URL https://arxiv.org/abs/2605.27491. 5
- [16]Min Zhao, Hongzhou Zhu, Bokai Yan, et al. minWM: A Full-Stack Open-Source Framework for Real-Time Interactive Video World Models. arXiv preprint arXiv:2605.30263, 2026. URL https://arxiv.org/abs/2605.30263. 5
- [17]Patrick von Platen, Suraj Patil, Anton Lozhkov, et al. Diffusers: State-of-the-art diffusion models. Official source repository, 2022. URL https://github.com/huggingface/diffusers. 5
- [18]Remi Cadene, Simon Alibert, Francesco Capuano, et al. LeRobot: An Open-Source Library for End-to-End Robot Learning. In International Conference on Learning Representations (ICLR), 2026. URL https://arxiv.org/abs/2602.22818. 5
- [19]StarVLA Community. StarVLA: A Lego-like Codebase for Vision-Language-Action Model Developing. arXiv preprint arXiv:2604.05014, 2026. URL https://arxiv.org/abs/2604.05014. 5, 10, 21
- [20]Ziqi Huang, Yinan He, Jiashuo Yu, et al. VBench: Comprehensive Benchmark Suite for Video Generative Models. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024. URL https://arxiv.org/abs/2311.17982. 5, 6
- [21]Ziqi Huang, Fan Zhang, Xiaojie Xu, et al. VBench++: Comprehensive and Versatile Benchmark Suite for Video Generative Models. arXiv preprint arXiv:2411.13503, 2024. URL https://arxiv.org/abs/2411.13503. 5, 6
- [22]Haoyi Duan, Hong-Xing Yu, Sirui Chen, et al. WorldScore: A Unified Evaluation Benchmark for World Generation. In IEEE/CVF International Conference on Computer Vision (ICCV), 2025. URL https://arxiv.org/abs/2504.00983. 5, 6, 9
- [23]Yu Shang, Yinzhou Tang, Yiding Ma, et al. WorldArena 2.0: Extending Embodied World Model Benchmarking on Modality, Functionality and Platform. arXiv preprint arXiv:2605.17912, 2026. URL https://arxiv.org/abs/2605.17912. 5, 6, 9
- [24]Jiahan Zhang, Muqing Jiang, Nanru Dai, et al. World-in-World: World Models in a Closed-Loop World. arXiv preprint arXiv:2510.18135, 2025. URL https://arxiv.org/abs/2510.18135. 5, 6
- [25]Matthew Tancik, Ethan Weber, Evonne Ng, et al. Nerfstudio: A Modular Framework for Neural Radiance Field Development. In ACM SIGGRAPH 2023 Conference Proceedings, 2023. URL https://arxiv.org/abs/2302.04264. 5
- [26]Mayank Mittal, Pascal Roth, James Tigue, et al. Isaac Lab: A GPU-Accelerated Simulation Framework for Multi-Modal Robot Learning. arXiv preprint arXiv:2511.04831, 2025. URL https://arxiv.org/abs/2511.04831. 5
- [27]Hritik Bansal, Zongyu Lin, Tianyi Xie, et al. VideoPhy: Evaluating Physical Commonsense for Video Generation. arXiv preprint arXiv:2406.03520, 2024. URL https://arxiv.org/abs/2406.03520. 6
- [28]Jianjie Fang, Yingshan Lei, Qin Wan, et al. iWorld-Bench: A Benchmark for Interactive World Models with a Unified Action Generation Framework. arXiv preprint arXiv:2605.03941, 2026. URL https://arxiv.org/abs/2605.03941. 6, 9
- [29]Xinhao Xiang, Xiao Liu, Zizhong Li, Zhuosheng Liu, and Jiawei Zhang. AIGVE-Tool: AI-Generated Video Evaluation Toolkit with Multifaceted Benchmark. arXiv preprint arXiv:2503.14064, 2025. URL https://arxiv.org/abs/2503.14064. 6
- [30]Weiliang Chen, Haowen Sun, Jun Gao, et al. HarnessEval-W: Agentifying the Evaluation of Visual Worlds. arXiv preprint arXiv:2608.16859, 2026. URL https://arxiv.org/abs/2608.16859. 6
- [31]Zhuoyi Yang, Jiayan Teng, Wendi Zheng, et al. CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer. arXiv preprint arXiv:2408.06072, 2024. URL https://arxiv.org/abs/2408.06072. 17
- [32]Weichen Fan, Chenyang Si, Junhao Song, et al. Vchitect-2.0: Parallel Transformer for Scaling Up Video Diffusion Models. arXiv preprint arXiv:2501.08453, 2025. URL https://arxiv.org/abs/2501.08453. 17
- [33]Bing Wu, Chang Zou, Changlin Li, et al. HunyuanVideo 1.5 Technical Report. arXiv preprint arXiv:2511.18870, 2025. URL https://arxiv.org/abs/2511.18870. 17
- [34]Lighttricks. LTX-2.5: Official Distilled Model Release. Official model release or documentation, n.d. URL https://github.com/Lighttricks/LTX-2. Accessed October 13, 2026. 17
- [35]Team Wan, Ang Wang, Baole Ai, et al. Wan: Open and Advanced Large-Scale Video Generative Models. arXiv preprint arXiv:2503.20314, 2025. URL https://arxiv.org/abs/2503.20314. 17
- [36]Zeyinzi Jiang, Zhen Han, Chaojie Mao, et al. VACE: All-in-One Video Creation and Editing. arXiv preprint arXiv:2503.07598, 2025. URL https://arxiv.org/abs/2503.07598. 17
- [37]Wan Team. Wan2.2: Official Model Release. Official model repository, 2025. URL https://github.com/Wan-Video/Wan2.2. Accessed September 9, 2026. 17
- [38]Shuailei Ma, Jiaqi Liao, Xinyang Wang, et al. Scaling Mixture-of-Experts Video Pretraining for Embodied Intelligence. arXiv preprint arXiv:2607.07675, 2026. URL https://arxiv.org/abs/2607.07675. 17
- [39]MiniMax. MiniMax H3. Official model release, 2026. URL https://www.minimax.cn/blog/minimax-h3. Accessed September 9, 2026. 17
- [40]Yansong Tang, Aoyang Liu, Jinpeng Liu, Shiyi Zhang, Wenxun Dai, Jie Zhou, Xiu Li, and Jiwen Lu. Flag3d++: A benchmark for 3d fitness activity comprehension with language instruction. IEEE Transactions on Pattern Analysis and Machine Intelligence, 47(11):9731–9748, 2025. 17
- [41]Runjia Qian, Zile Wang, Jihai Zhang, et al. Matrix-Game 3.5: Enhancing Real-Time Streaming Interactive World Models with Patch Memory. arXiv preprint arXiv:2608.29910, 2026. URL https://arxiv.org/abs/2608.29910. 18, 19
- [42]Robbyant Team, Zelin Gao, Qiuyu Wang, et al. Advancing Open-source World Models. arXiv preprint arXiv:2601.20540, 2026. URL https://arxiv.org/abs/2601.20540. 18
- [43]Zelin Gao, Qiuyu Wang, Jiapeng Zhu, et al. Infinite Worlds with Versatile Interactions. arXiv preprint arXiv:2607.07534, 2026. URL https://arxiv.org/abs/2607.07534. 18, 19
- [44]Jiaqi Li, Junshu Tang, Zhiyong Xu, et al. Hunyuan-GameCraft: High-dynamic Interactive Game Video Generation with Hybrid History Condition. arXiv preprint arXiv:2506.17201, 2025. URL https://arxiv.org/abs/2506.17201. 18
- [45]Junshu Tang, Jiacheng Liu, Jiaqi Li, et al. Hunyuan-GameCraft-2: Instruction-following Interactive Game World Model. arXiv preprint arXiv:2511.23429, 2025. URL https://arxiv.org/abs/2511.23429. 18, 19
- [46]Xiaofeng Mao, Zhen Li, Chuanhao Li, et al. Yume-1.5: A Text-Controlled Interactive World Generation Model. arXiv preprint arXiv:2512.22096, 2025. URL https://arxiv.org/abs/2512.22096. 18
- [47]Wenqiang Sun, Haiyu Zhang, Haoyuan Wang, et al. WorldPlay: Towards Long-Term Geometric Consistency for Real-Time Interactive World Modeling. arXiv preprint arXiv:2512.14614, 2025. URL https://arxiv.org/abs/2512.14614. 18
- [48]Zizhao Tong, Yeying Jin, Hongfeng Lai, et al. SCOPE: Simulating Cross-game Operations in Playable Environments for FPS World Models. arXiv preprint arXiv:2605.23345, 2026. URL https://arxiv.org/abs/2605.23345. 18, 19
- [49]Haoyi Zhu, Haozhe Liu, Yuyang Zhao, et al. SANA-WM: Efficient Minute-Scale World Modeling with Hybrid Linear Diffusion Transformer. arXiv preprint arXiv:2605.15178, 2026. URL https://arxiv.org/abs/2605.15178. 18
- [50]Songchun Zhang, Yaowei Li, Junhao Zhuang, et al. EchoWM: Open and Enterable Omnimodal World Models. arXiv preprint arXiv:2608.23189, 2026. URL https://arxiv.org/abs/2608.23189. 18
- [51]Xianglong He, Chunli Peng, Zexiang Liu, et al. Matrix-game 2.0: An open-source real-time and streaming interactive world model. arXiv preprint arXiv:2508.13009, 2025. URL https://arxiv.org/abs/2508.13009. 19
- [52]Zile Wang, Zexiang Liu, Jiaxing Li, et al. MatrixGame 3.0: Real-Time and Streaming Interactive World Model with Long-Horizon Memory. arXiv preprint arXiv:2604.08995, 2026. URL https://arxiv.org/abs/2604.08995. 19
- [53]Zhifei Chen, Luozhou Wang, Guibao Shen, et al. ReWorld: An Interactive World Model with Long-Horizon Memory. arXiv preprint arXiv:2608.23565, 2026. URL https://arxiv.org/abs/2608.23565. 19
- [54]Shuzhe Wang, Vincent Leroy, Yohann Cabon, et al. DUSt3R: Geometric 3D Vision Made Easy. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024. URL https://arxiv.org/abs/2312.14132. 19
- [55]Qianqian Wang, Yifei Zhang, Aleksander Holynski, et al. Continuous 3D Perception Model with Persistent State. arXiv preprint arXiv:2501.12387, 2025. URL https://arxiv.org/abs/2501.12387. 20
- [56]Jianyuan Wang, Minghao Chen, Nikita Karaev, et al. VGGT: Visual Geometry Grounded Transformer. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025. URL https://arxiv.org/abs/2503.11651. 20
- [57]Jianyuan Wang, Minghao Chen, Shangzhan Zhang, et al. VGGT-Ω. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2026. URL https://arxiv.org/abs/2605.15195. 20
- [58]Brandon Smart, Chuanxia Zheng, Iro Laina, et al. Splat3R: Zero-shot Gaussian Splatting from Uncalibrated Image Pairs. arXiv preprint arXiv:2408.13912, 2024. URL https://arxiv.org/abs/2408.13912. 20
- [59]Xinyang Li, Tengfei Wang, Zixiao Gu, et al. FlashWorld: High-quality 3D Scene Generation within Seconds. arXiv preprint arXiv:2510.13678, 2025. URL https://arxiv.org/abs/2510.13678. 20
- [60]Junyi Zhang, Charles Herrmann, Junhwa Hur, et al. MonST3R: A Simple Approach for Estimating Geometry in the Presence of Motion. arXiv preprint arXiv:2410.03825, 2024. URL https://arxiv.org/abs/2410.03825. 20
- [61]Yuxue Yang, Lue Fan, Ziqi Shi, et al. NeoVerse: Enhancing 4D World Model with in-the-wild Monocular Videos. arXiv preprint arXiv:2601.00393, 2026. URL https://arxiv.org/abs/2601.00393. 20
- [62]Sherwin Bahmani, Tianchang Shen, Jiawei Ren, et al. Lyra: Generative 3D Scene Reconstruction via Video Diffusion Model Self-Distillation. In International Conference on Learning Representations (ICLR), 2026. URL https://research.nvidia.com/labs/toronto-ai/lyra/. 20arxiv.org/abs/2509.19296
- [63]Tianchang Shen, Sherwin Bahmani, Kai He, et al. Lyra 2.0: Explorable Generative 3D Worlds. arXiv preprint arXiv:2604.13036, 2026. URL https://arxiv.org/abs/2604.13036. 20
- [64]Tony Z. Zhao, Vikash Kumar, Sergey Levine, et al. Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware. arXiv preprint arXiv:2304.13705, 2023. URL https://arxiv.org/abs/2304.13705. 21
- [65]Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, et al. OpenVLA: An Open-Source Vision-Language-Action Model. arXiv preprint arXiv:2406.09246, 2024. URL https://arxiv.org/abs/2406.09246. 21
- [66]Xinghang Li, Minghuan Liu, Hanbo Zhang, et al. Vision-Language Foundation Models as Effective Robot Imitators. arXiv preprint arXiv:2311.01378, 2023. URL https://arxiv.org/abs/2311.01378. 21
- [67]Haoquan Fang, Jiafei Duan, Donovan Clay, et al. MolmoAct2: Action Reasoning Models for Real-world Deployment. arXiv preprint arXiv:2605.02881, 2026. URL https://arxiv.org/abs/2605.02881. 21
- [68]GigaBrain Team, Angen Ye, Boyuan Wang, et al. GigaBrain-0: A World Model-Powered Vision-Language-Action Model. arXiv preprint arXiv:2510.19430, 2025. URL https://arxiv.org/abs/2510.19430. 21
- [69]Hao Luo, Ye Wang, Wanpeng Zhang, et al. Being-H0.5: Scaling Human-Centric Robot Learning for Cross-Embodiment Generalization. arXiv preprint arXiv:2601.12993, 2026. URL https://arxiv.org/abs/2601.12993. 21
- [70]Wei Wu, Fan Lu, Yunnan Wang, et al. A Pragmatic VLA Foundation Model. arXiv preprint arXiv:2601.18692, 2026. URL https://arxiv.org/abs/2601.18692. 21
- [71]Wei Wu, Fangjing Wang, Fan Lu, et al. From Foundation to Application: Improving VLA Models in Practice. arXiv preprint arXiv:2607.06403, 2026. URL https://arxiv.org/abs/2607.06403. 21
- [72]Lin Li, Qihang Zhang, Yiming Luo, et al. Causal World Modeling for Robot Control. arXiv preprint arXiv:2601.21998, 2026. URL https://arxiv.org/abs/2601.21998. 21
- [73]Qihang Zhang, Lin Li, Luyao Zhang, et al. Native Video-Action Pretraining for Generalizable Robot Control. arXiv preprint arXiv:2607.08639, 2026. URL https://arxiv.org/abs/2607.08639. 21
- [74]Bo Liu, Yifeng Zhu, Chongkai Gao, et al. LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning. arXiv preprint arXiv:2306.03310, 2023. URL https://arxiv.org/abs/2306.03310. 21
- [75]Tianxing Chen, Zanxin Chen, Baijun Chen, et al. RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulation. arXiv preprint arXiv:2506.18088, 2025. URL https://arxiv.org/abs/2506.18088. 21
- [76]Qiyaun Liao, Yiyang Shao, Junfeng Long, Kevin Zakka, Roberto Horowtiz, Pieter Abbeel, and Koushil Sreenath. Supermimic: Scaling humanoid whole-body control with one differentiable world model. arXiv preprint arXiv:2610.28480, 2026. 21arxiv.org/abs/2610.28480
- [77]Tri Dao. FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning. In International Conference on Learning Representations (ICLR), 2024. URL https://arxiv.org/abs/2307.08691. 23
- [78]Jay Shah, Ganesh Bikshandi, Ying Zhang, et al. FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precision. arXiv preprint arXiv:2407.08608, 2024. URL https://arxiv.org/abs/2407.08608. 23
- [79]Jintao Zhang, Jia Wei, Haofeng Huang, et al. SageAttention: Accurate 8-Bit Attention for Plug-and-play Inference Acceleration. In International Conference on Learning Representations (ICLR), 2025. URL https://arxiv.org/abs/2410.02367. 23






