betaFeedback

WorksComputer Science

WorldFoundry: Towards Unified Infrastructure for World Intelligence

Abstract

Visual generative models increasingly blur the line between synthesizing observations and simulating environments. Yet visual plausibility alone does not constitute successful world modeling: compelling outputs may lack persistent state, physical dynamics, or causally consistent responses to interactions. We define a generative world model as a predictive dynamical system that maintains a persistent, history-conditioned representation of an environment, updates that representation in response to actions or interventions, and generates temporally coherent trajectories that reflect outcomes under observed and counterfactual interventions. From this definition, we derive an intelligence-oriented framework spanning perceptual grounding, persistent state modeling, controllable transformation, dynamics-aware prediction, and closed-loop interaction. We present WorldFoundry, open-source infrastructure for reproducible inference and evaluation across video, 3D and 4D, embodied action, and interactive world systems. WorldFoundry unifies model and benchmark discovery, runtime and asset preparation, execution orchestration, artifact management, and interactive visualization, while modality-specific adapters preserve native inputs, execution semantics, output representations, and evaluation protocols. Explicit manifests, durable artifact contracts, and evidence-aware scorecards separate execution readiness from evaluative claims, making each claim traceable to its supporting artifacts. The catalog contains 270 model entries, 74 benchmarks, and 62 registered metric entries, with readiness and execution evidence recorded separately. WorldFoundry enables reproducible comparison of persistent, controllable dynamics across heterogeneous systems, advancing the field from perceptual synthesis toward measurable world intelligence.

See Appendix A for author contributions.

Date: November 7, 2026

Email: [email protected]

Project Page: https://openenvision.org/WorldFoundry/

Code: https://github.com/OpenEnvision/WorldFoundry

Introduction

Recent advances in generative modeling are moving beyond synthesizing individual objects or scenes toward modeling worlds that evolve over time and respond to interventions. This shift calls for a broader notion of world intelligence, in which systems must perceive observations, infer structured states, predict their evolution, express resulting consequences through task-specific outputs, and exploit these predictions for interaction [1, 2, 3, 4]. Video generators, neural scene representations, and embodied policies address complementary parts of this problem. However, incompatible representations, runtimes, interfaces, and evaluation protocols make capabilities difficult to compose, results difficult to reproduce, and progress toward genuine world models difficult to measure. The bottleneck is therefore not only algorithmic, but also infrastructural. Recent efforts such as OpenWorldLib [5] address complementary aspects of this integration problem. Progress therefore requires shared infrastructure grounded in a common functional abstraction that makes heterogeneous models composable, reproducible, and comparable across modalities while preserving their native semantics.

To establish common ground without constraining how different systems represent the world, we formulate the abstraction at the level of function. Building on insights from earlier definitions in reinforcement learning [1, 2, 3] and recent efforts to delineate their scope [5], we identify three tightly coupled components. State inference estimates a latent environmental state from the observation history; forward simulation predicts its evolution under endogenous dynamics and external interventions; and interactive response produces distinct, actionconditioned futures as visual, geometric, or task-specific outputs. This decomposition is functional rather than architectural and defines a shared interface, not a requirement that every system implement all three operations. Static 3D methods emphasize persistent-state inference and geometry decoding, video models primarily learn implicit transitions and observation generation, 4D methods explicitly represent time-varying structure, and embodied models connect state estimation and action prediction to decision-making. Characterizing systems by the operations they implement and the capability profiles they support provides a common vocabulary for comparing them.

These operations alone, however, describe what a system does without determining whether it qualifies as a world model. Qualification is decided not by any single operation but by the state that persists across all of them: we define a generative world model as a learned, history- and control-conditioned dynamical system that infers and maintains a persistent environmental state, evolves that state under endogenous dynamics and external interventions, and assimilates observational feedback into coherent future trajectories. Persistence in this sense is operational rather than nominal: an intervention must leave a trace in the state that constrains later predictions, and each incoming observation must in turn revise that state. Only when both hold is the world-modeling loop closed. Visual plausibility, by contrast, reflects only the observation decoder; it is neither necessary nor sufficient evidence that the underlying state persists, a gap that recent physics- and world-oriented benchmarks repeatedly expose [6, 7, 8, 9]. It is closure, not plausibility, that draws a principled boundary between foundational generative models, which synthesize or transform observations without closing the loop, and generative world models, which couple generation to an evolving environmental state through closed-loop updates. Measuring progress from foundational generation toward world intelligence therefore calls for evidence of closure, which in turn depends on shared infrastructure for reproducible execution and comprehensive, fair evaluation.

We realize this infrastructure in WorldFoundry, an open-source system for integrating, executing, and evaluating video, 3D, 4D, interactive world, and embodied-action models. WorldFoundry spans the research lifecycle, from model and benchmark discovery and runtime and asset preparation to execution orchestration, artifact management, evaluation, and interactive visualization. Modality-specific adapters preserve native execution and evaluation semantics; declarative manifests specify dependencies, assets, and interfaces; durable artifact contracts decouple execution from evaluation; and evidence-aware scorecards separate declared support, operational readiness, and validated performance. The current release integrates 270 models and over 100 benchmark and metric implementations, with every model runner validated through end-to-end execution of a documented reference workflow. By combining our theory of world modeling with the proposed infrastructure, WorldFoundry transforms isolated generative demonstrations into cumulative, auditable evidence and advances the field from perceptual synthesis toward measurable world intelligence.

Our key contributions can be summarized as follows:

  • World Model Definition. We formally define world models through a minimal abstraction comprising state inference, forward simulation, and interactive response. We then characterize the core capabilities of perception, modeling, manipulation, dynamics, and interaction within a structured functional framework, thereby organizing and differentiating existing generative and world-modeling systems.

  • Unified Inference and Evaluation Infrastructure. We introduce WorldFoundry, an open-source infrastructure that standardizes execution and evaluation across video, 3D, 4D, embodied-action, and interactive world systems, making inference reproducible and configurable without overriding native model semantics, and placing heterogeneous systems under a common operational definition of world modeling.

  • Validation at Scale. We integrate 270 generative models and over 100 benchmark and metric implementations. We validate every model runner through a documented end-to-end reference workflow, yielding an auditable record of operational readiness under the tested configurations.

Related Work

SystemVideo3D/4DInter. WMEmbodiedModel runnerHosted APIClosed loopTrainingProtocolsEvidenceWM def.Interfaces
WORLD-MODEL INFRASTRUCTURE
OpenWorldLib [5, 10]✓✓✓✓✓✓✓×✓◐✓Py · CLI
Cosmos [11, 12, 13]✓×✓✓◐✓✓✓✓◐◐Py · CLI · Web
Genie Envisioner [14, 15]◐×✓✓◐×✓✓◐◐◐Py · CLI<br>Web (monitor)
minWM [16]◐×✓×◐××✓×◐◐Py · CLI
MODALITY-SPECIFIC TOOLKITS
Diffusers [17]✓◐◐◐✓××✓◐◐×Py
LeRobot [18]◐×✓✓××✓✓◐◐×Py · CLI
StarVLA [19]×××✓◐×✓✓◐◐×Py · CLI
EVALUATION SUITES
VBench [6, 20, 21]✓×××××××◐◐×Py · CLI · LB
WorldScore [22]✓✓××◐◐××◐◐◐Py · CLI · LB
WorldArena [9, 23]◐×◐✓◐×✓◐◐◐◐Py · CLI · LB
World-in-World [24]◐×◐✓◐◐✓◐◐◐◐Py · CLI
WorldFoundry✓✓✓✓✓✓✓×✓✓✓TUI · CLI · Web<br>Py · MCP

Table 1. Comparison with world-model infrastructures, modality-specific toolkits, and evaluation suites. ✓: supported; ◐: specialized support; ×: no documented workflow. Column definitions follow the table.

Reading the table. Coverage: ◐ restricted task or domain; Inter. WM requires action-conditioned dynamics, not camera-path rendering alone. Model runner: ✓ independently released families behind shared interfaces; ◐ first-party models or benchmark-specific adapters. Hosted API: provider-hosted generation or policy services. Closed loop: controls depend on prior outputs or environment feedback. Training: ◐ protocol-specific recipe only. Protocols: ✓ independent benchmarks across task families; ◐ a single suite or domain. Evidence: ✓ links entry readiness to result provenance; ◐ stores configurations or scores only. WM def.: ✓ framework-level criterion and taxonomy; ◐ domain-specific formulation. VBench, WorldArena, and Genie Envisioner rows include VBench++/2.0, WorldArena 2.0, and GE-Sim 2.0. LB: leaderboard; MCP: Model Context Protocol. Marks describe documented workflows; entry-level readiness and validation scope are reported in the appendices. Validation applies to the documented route and configuration.

World Models and Research Infrastructure

Learned world models link environmental prediction to planning and control. World Models [1] combined compact visual representations with recurrent dynamics and policy learning; PlaNet [2] planned through latent dynamics, while DreamerV3 [3] learned policies from imagined trajectories. Research on video generation, spatial reconstruction, and embodied control introduces diverse conditioning schemes, state representations, and execution requirements. Specialized toolkits address these requirements within individual domains: Diffusers [17] organizes generative pipelines, Nerfstudio [25] supports neural reconstruction and rendering, LeRobot [18] and StarVLA [19] support policy development and deployment, and Isaac Lab [26] provides simulation and learning workflows. World-model platforms cover broader development lifecycles. Cosmos [11, 12] combines data preparation, foundation models, and post-training, complemented by scoring orchestration and durable result storage in Cosmos Evaluator [13]. Genie Envisioner and GE-Sim 2.0 [14, 15] connect action-conditioned video simulation with robot-policy evaluation and learning, while minWM [16] supports fine-tuning and distillation for controllable autoregressive generation. OpenWorldLib [5, 10] is the most closely related infrastructure for heterogeneous integration: its pipeline, operator, representation, reasoning, and memory modules unify video, spatial, and embodied systems through common interfaces, supporting multi-turn execution and evaluation from saved results. Its formulation emphasizes perception, interaction, and long-term memory. WorldFoundry couples an operational criterion grounded in persistent state, intervention effects, and observational feedback (§ 4–§ 4.1) with explicit execution contracts: manifests specify runtime and asset requirements, serializable requests preserve native controls, and typed artifact references link outputs to execution provenance. These contracts connect model invocation, artifact management, and evaluation within a shared experimental lifecycle; general training remains future work.

Evaluation Protocols and Experimental Evidence

Evaluation suites operationalize complementary notions of generative quality, environmental consistency, and interactive utility. VBench and its extensions [6, 20, 21] provide a multidimensional assessment of perceptual quality, semantic alignment, and intrinsic faithfulness, while VideoPhy [27] and WorldModelBench [8] examine whether generated events and interactions satisfy physical constraints and commonsense expectations. WorldScore [22] evaluates video and spatial generation under camera-conditioned protocols, connecting assessment to specified controls over the generated scene. iWorld-Bench [28] focuses on interactive control and memory, where model responses must be interpreted in relation to preceding inputs and interactions. WorldArena [9, 23] and World-in-World [24] extend assessment to embodied utility, policy performance, and closed-loop planning, linking world-model capabilities to their effectiveness within agent workflows. Evaluation infrastructure supports these assessments through complementary mechanisms: AIGVE-Tool [29] provides modular scoring components, and HarnessEval-W [30] organizes judgments around structured supporting evidence. The evidence required for evaluation consequently varies across protocols. Some assessments can operate on fixed generated artifacts, whereas interactive and embodied tasks may require control histories, environment feedback, and executed rollouts. Differences in these requirements and in score aggregation make execution conditions and protocol compliance central to meaningful comparison. WorldFoundry records resolved configurations, artifact references, sample coverage, failures, and validation scope for each evaluation (§ 5.5). Its scorecards distinguish diagnostic results, benchmark-comparable results, and complete reproductions under the corresponding protocol. Compatible evaluators can rescore saved outputs; protocols requiring new interventions or policy rollouts retain their execution requirements. These records make evaluative claims traceable to their supporting artifacts, execution conditions, and validation evidence.

Unified Definition of World Modeling

Figure 2. World Modeling. State inference maintains a persistent environment representation from observations; forward simulation predicts coherent state transitions; interactive response produces action-conditioned futures. Observation feedback updates the state for subsequent predictions.

A fundamental obstacle to progress in generative world modeling research is the absence of a precise and shared conceptual foundation. Without an operational definition of what constitutes a world model and a principled criterion for distinguishing it from related systems, the field risks conflating fundamentally different modeling objectives. This section aims to address this gap. We first establish an operational definition of world modeling; we then situate the model landscape along an intelligence-oriented capability progression that traces increasingly demanding functional behaviors; finally, we formulate a principled boundary between foundational generative models that synthesize observations and generative world models that simulate world.

What is World Model: An Operational Definition

Motivated by previous work on learned world models in reinforcement learning [1, 2, 3], we characterize world modeling by three tightly coupled operations over task-relevant latent state rather than observations alone: (i) state inference, which estimates latent environmental state from observations; (ii) forward simulation, which predicts state evolution under autonomous dynamics, physical constraints, and external actions; and (iii) interactive response, which adapts predicted futures coherently under alternative actions or interventions. Let oto_t denote an observation, ctc_t an action or intervention, and ht=(o≤t,c<t)h_t = (o_{\leq t}, c_{<t}) the interaction history. A minimal, modality-agnostic abstraction is

zt∼q(zt∣ht),zt+1∼p(zt+1∣zt,ct),o^t+1∼g(⋅∣zt+1).(1)z_t \sim q(z_t \mid h_t), \qquad z_{t+1} \sim p(z_{t+1} \mid z_t, c_t), \qquad\hat{o}_{t+1} \sim g(\cdot\mid z_{t+1}). \tag*{(1)}

Here ztz_t denotes a task-relevant environmental state, while qq, pp, and gg denote state inference, transition, and observation models. Repeated application of pp yields predictive rollouts under alternative control sequences; once ot+1o_{t+1} is observed, the state is re-inferred as zt+1∼q(⋅∣ht+1)z_{t+1} \sim q(\cdot\mid h_{t+1}) before subsequent predictions, closing the loop between prediction and feedback, following the latent-dynamics perspective used in model-based planning [2]. This formulation is functional rather than architectural: ztz_t may be recurrent, object-centric, geometric, token-based, or otherwise structured. Coupling these generic world-modeling operations into a single persistent process likewise yields our foundational definition of generative world models:

Definition: Generative World Model. A Generative World Model is a learned, history- and control-conditioned latent dynamical system that infers and maintains a persistent environmental state, evolves that state under endogenous dynamics and external interventions, and assimilates feedback to generate coherent future trajectories.

With this definition, a generative world model synthesizes possible future trajectories of observations grounded in a persistent, history-conditioned environmental state. Generation under alternative interventions branches from a shared state, with the consequences of each intervention constraining subsequent outputs. Newly acquired observations update the maintained state and inform further generation. The defining property is therefore the coupling of observation synthesis with persistent state, intervention-conditioned dynamics, and observational feedback. These requirements motivate the capability progression below, which characterizes how constituent capabilities jointly support coherent world generation and interaction.

What World Models Can Do: Capability Progression

Figure 3. Capability progression from generative modeling toward world modeling. Perceptual grounding provides the foundation; persistent state modeling, dynamics-aware prediction, and closed-loop interaction form the world-model spine; controllable transformation provides an orthogonal capability for manipulation.

To situate existing generative systems with respect to this definition, we organize them along a capability progression in which generation becomes increasingly grounded in an evolving environmental state. Perceptual Grounding constitutes the foundation: the capability to interpret, reconstruct, or synthesize observations without maintaining a persistent environmental state. The World-Model Spine comprises three successive stages, each building on its predecessor: Persistent State Modeling infers and maintains a structured environmental state across observations; Dynamics-Aware Prediction predicts physically and causally coherent transitions of that state under endogenous dynamics and external interventions; and Closed-Loop Interaction sustains repeated cycles of action, observation, and state update, in which each observation revises the state that conditions subsequent predictions. Controllable Transformation, the capability to modify represented entities, attributes, or configurations as specified while preserving content outside the scope of the edit, is orthogonal to the spine and can complement any of its stages; by itself, however, it entails none of the spine capabilities. Rather than assigning each system to a single discrete level, the progression characterizes it by a capability profile: which capabilities it exhibits and the degree to which they are integrated.

Foundational Capability: Perceptual Grounding

Perceptual Grounding (Perception) is the capability to interpret, reconstruct, or synthesize observable content. It captures structure at the observation level, supplying the perceptual basis for state inference without itself maintaining a persistent representation of the underlying environment.

World-Model Spine

Persistent State → Dynamics Prediction → Interactive Feedback

Spine I: Persistent State Modeling

Persistent State Modeling (Modeling) is the capability to infer, maintain, and recursively update a structured representation of environmental state from sequential observations. This representation tracks task-relevant entities and their identities, geometry, spatial relations, and latent attributes across viewpoint changes and occlusions under partial observability.

Spine II: Dynamics-Aware Prediction

Dynamics-Aware Prediction (Dynamics) is the capability to predict environmental state transitions in accordance with physical and causal regularities. It propagates the effects of endogenous processes, entity interactions, and external interventions through the evolving state, producing temporally coherent trajectories in which earlier events constrain later predictions through their persisting consequences.

Spine III: Closed-Loop Interaction

Closed-Loop Interaction (Interaction) is the capability to sustain recurrent cycles of action, environmental transition, observation, and state update within a persistent environment. Feedback from observed action outcomes revises the maintained state, which in turn conditions subsequent predictions and actions, preserving causal continuity across successive interactions.

Orthogonal Capability: Controllable Transformation

Controllable Transformation (Manipulation) is the capability to modify represented content or state according to specified controls while preserving properties and structural relations outside the intended scope of the edit. As an orthogonal capability, it can complement any stage of the progression, but on its own establishes neither persistent state modeling nor coherent dynamics or closed-loop interaction.

Taken together, these capabilities characterize world intelligence through their integration rather than their individual strength: observations update persistent state, state constrains predictions, and action outcomes inform subsequent inference and interaction. Current systems occupy this progression unevenly. Video generators achieve strong perceptual grounding with largely implicit state; 3D and 4D methods maintain explicit spatial structure but rarely predict its evolution under actions; interactive world models close the loop over short horizons and have begun to address longer-range memory; and embodied policies act within persistent environments under narrow task and embodiment coverage. Evaluating where a system sits therefore requires evidence about the cycle itself, not only about output quality. Existing benchmarks test complementary segments of this progression: visual and physical fidelity [6, 8], camera-controlled world generation [22], interactive memory [28], and downstream embodied utility [9, 23].

Taxonomy: Foundational Generative Models and Generative World Models

Building on the definition and capability progression above, we draw a single boundary through contemporary generative systems: whether a system closes the world-modeling loop. Models that primarily generate or transform observations count as foundational generative models, even with partial, local, or implicit dynamics, unless they maintain persistent state, support intervention-conditioned dynamics, and assimilate observational feedback. Generative world models, by contrast, meet all three conditions: they maintain latent world states that evolve under intervention and are revised by incoming observations.

Foundational Generative Models learn visual, geometric, or spatiotemporal priors for synthesis, reconstruction, and controllable manipulation, but do not integrate state, dynamics, and feedback into a complete world-modeling loop. Their defining limitation is the inability to maintain an evolving environmental state that preserves intervention effects and incorporates observational feedback into subsequent predictions. Strong individual capabilities therefore do not suffice to sustain a coherent, revisable model of the world.

Generative World Models, by contrast, integrate persistent state, intervention-conditioned dynamics, and observational feedback into a complete world-modeling loop. Their defining property is functional coupling: intervention effects persist across state transitions, while each new observation revises the state governing subsequent predictions. Generation is thereby made accountable to an evolving environmental model, supplying a functional basis for world intelligence, the capacity to anticipate the consequences of intervention and to revise those expectations through interaction.

WorldFoundry Design

Assessing the capabilities defined above requires reproducible experiments that tie observed behavior to explicitly specified tasks and evaluation conditions. Existing implementations, however, rely on incompatible environments, input conventions, execution procedures, artifact formats, and evaluation protocols, making it difficult to tell whether observed differences arise from model capabilities or from the surrounding experimental pipelines. This fragmentation also blurs the distinctions among catalog inclusion, successful inference, partial validation, and benchmark reproduction, which are often conflated under a single notion of “model support.” Related infrastructures address complementary aspects of this problem, including OpenWorldLib for world-model inference [5], Cosmos for physical-AI model development [11], and StarVLA for modular embodied-policy research [19]. Common to all of these settings is a central design challenge: preserving model-specific execution semantics while making explicit what has been executed, under what conditions, and which conclusions the resulting evidence supports.

We introduce WorldFoundry as open-source infrastructure organized around a common lifecycle of discovery, runtime preparation, execution, inspection, and evaluation. It serves three functions. (I) It integrates declarative model and benchmark catalogs with explicit runtime, dependency, and asset management, allowing researchers to discover, configure, and execute foundational generative models for video, 3D, and 4D alongside generative world models, embodied simulators, and vision-language-action, vision-action, and world-action models, while preserving native execution semantics. (II) It decouples execution from evaluation through durable artifacts and serializable request and result contracts, so that generated videos, spatial representations, actions, trajectories, and execution traces can be inspected, reused, compared, and scored without re-instantiating the originating model or runtime; run manifests, sample results, metric summaries, reports, and scorecards record the provenance and evidential scope of each experiment. (III) It provides interactive interfaces for configuring runs, issuing control inputs, visualizing multimodal outputs, comparing results across model families, and diagnosing failures throughout generation, interaction, and evaluation.

System Overview

WorldFoundry is organized around a user-oriented, capability-aware research lifecycle that brings heterogeneous visual generation and world-modeling systems into a coherent operational framework. Rather than requiring users to navigate fragmented, model-specific inference scripts and disconnected evaluation pipelines, it establishes shared conventions for model discovery, runtime preparation, execution, artifact inspection, and evidence-based evaluation while preserving model-specific semantics. The framework comprises five components (Figure 1), each described in a subsequent subsection. (1) _Surface Layer_ provides consistent user-facing access through a terminal interface, command-line tools, a browser workspace, a Python API, and programmatic interfaces. (2) _Unified Inference and Interaction_ resolves model configurations, runtime profiles, and required assets, and supports both stateless generation and stateful interaction through shared request and execution pathways without altering model-native inputs, parameters, or execution semantics. (3) _Artifact and Representation Contracts_ specify how requests, results, generated outputs, state traces, and execution records are materialized as durable, inspectable artifacts with explicit identity, semantics, metadata, and provenance. (4) Evaluation and Evidence applies compatible metrics and benchmark protocols to these artifacts and records per-sample results, aggregate scores, coverage, validation status, unresolved blockers, and comparison eligibility. (5) Model and Benchmark Zoo maintains manifest-driven registries of models, benchmarks, tasks, runtime profiles, assets, evaluators, readiness states, and known blockers.

Figure 1. Overview of WorldFoundry. Open-source infrastructure for generative world models. (1) one run is reachable from every interactive and programmatic surface; (2) preflight requests execute through a unified runner, one-shot or session-resident; (3) outputs are persisted behind durable artifact contracts with typed references; (4) benchmarks score persist in artifacts reported in an evidence scorecard; and (5) declarative manifests bind discovery, runtime and evaluation, keeping claims traceable to evidence.

Across these components, persistent artifacts form the explicit boundary between execution and assessment. This separation allows computationally expensive outputs to be inspected, compared, rescored, and audited without re-executing the originating model, and keeps catalog inclusion, runnable integration, bounded validation, and complete benchmark reproduction as distinct claims rather than a single notion of support.

Surface Layer

The Surface Layer offers multiple user-facing interfaces that share a single orchestration path, ensuring consistent execution and evaluation across access modes. The terminal user interface (TUI) guides discovery by querying model and benchmark catalogs, checking requirements and readiness, and resolving an executable command before resources are allocated. The command-line interface (CLI) covers discovery, preparation, inference, evaluation, validation, and reporting, and is designed for automated experimentation and reproducible benchmarking. The browser-based Workspace supports multi-model job configuration, model-specific input specification, runtime selection, execution monitoring, artifact inspection, and cross-run comparison, while the single-model Studio offers tailored controls for interactive use and detailed inspection. Python and MCP interfaces extend this functionality to external programs and agents. Every surface translates user intent into normalized requests that are resolved against shared catalogs, runtime profiles, Pipelines, Operators, and result contracts; this shared resolution keeps results comparable regardless of how a run is launched. Figure 4–Figure 6 illustrate the complementary roles of the three primary human-facing surfaces.

WorldFoundry CLI discover, verify, execute, and report

Figure 4. Command-line interface (CLI). WorldFoundry CLI is a unified command-line interface for configuring and executing model inference and evaluation workflows. Screenshot from the CLI reference.

Terminal user interface (TUI)

Figure 6. Terminal user interface (TUI). WorldFoundry TUI is an interactive terminal interface for configuring and executing model inference and evaluation workflows through guided menus. Screenshot from the TUI guide.

Workspace and Studio

Figure 5. Workspace and Studio. WorldFoundry Workspace provides a unified browser interface for managing model inference and evaluation workflows, while Studio offers interactive environments tailored to different modalities. Screenshots from the Studio guide.

Studio additionally provides modality-aware visualization and interaction. It displays generated videos, temporal previews, galleries, and artifact references; renders point clouds and meshes in dedicated Viser views; and presents Gaussian splats in the Spark viewer. For interactive world models, a resident model session allows navigation, camera, and text controls to update the environment state without reinitializing the Pipeline for each input. Execution status, runtime logs, generation progress, and performance telemetry help users diagnose model-loading failures, interaction latency, generation instability, and invalid artifacts. Embodied systems that require native simulation connect through an external simulator bridge, which isolates simulator-specific dependencies from the browser-based Workspace. Together, these features unify discovery, configuration, execution control, qualitative inspection, and runtime diagnosis, while preserving generated outputs as durable artifacts for subsequent evaluation.

Unified Inference and Interaction

WorldFoundry resolves inference and interaction requests through a manifest-driven execution sequence. Each model identifier or alias is mapped to a canonical catalog entry whose manifest specifies provenance, supported tasks and variants, input and output modalities, required assets, runtime configuration, execution binding, and readiness constraints. Given the requested task and variant, the resolver verifies environment and asset compatibility and availability, reports blockers, and selects an execution backend. Once resolution succeeds, the input is encoded as a GenerationRequest, which defines common fields for sample identity, task context, conditioning inputs, control signals, generation parameters, and output specifications while retaining model-specific payloads. This contract standardizes orchestration without imposing uniform input semantics: video systems retain temporal and media-conditioning semantics; 3D and 4D systems retain camera, coordinate-frame, geometry, and rendering semantics; embodied systems retain observation–action schemas and simulator contracts; and interactive world models retain session state and sequential controls. The corresponding GenerationResult records execution status, timing, errors, the resolved configuration, and artifact references, establishing a common basis for traceability and downstream evaluation.

Given a resolved GenerationRequest, the WorldModelRunner coordinates execution through two complementary abstractions: an Operator and a Pipeline. The Operator handles task-specific adaptation, including input validation, preprocessing, translation into model-native arguments, postprocessing, and conversion of native outputs into framework-compatible artifacts. The Pipeline manages the model lifecycle and inference runtime, including model construction, checkpoint loading, device placement, runtime configuration, distributed execution, and invocation of native inference procedures. This division separates model adaptation from runtime management while accommodating domain-specific dependencies and simulator integration. Interactive systems may retain a resident Pipeline and session state, allowing successive actions or conditions to update the environment without repeated checkpoint loading or runtime initialization. Artifact references and execution metadata populate the GenerationResult; benchmark runners subsequently consume the materialized outputs and provenance records, decoupling evaluation from model loading and inference.

Artifact and Representation Contracts

WorldFoundry defines a two-level contract system that separates sample-level execution from run-level organization. A GenerationRequest represents a single execution unit: inputs contains text, source media, or task-specific observations; controls specifies action sequences, camera trajectories, or other control signals; generation_kwargs retains model-native parameters such as random seeds and frame rates; and output_schema declares expected artifact types and structural requirements. The corresponding GenerationResult records stable sample and model identifiers, execution status, errors, timing, resolved metadata, and output references. Each materialized output is described by an ArtifactRef, which identifies externally stored content by its semantic role and URI, with optional MIME type, media properties, file size, and content hash. Being JSON-serializable, these contracts persist, transfer, and can be inspected across process and environment boundaries independently of the originating model implementation.

The contracts standardize artifact identification, provenance, references, and run organization while preserving native encodings and modality-specific semantics across media, geometry, dynamic 4D representations, actions, and state traces. At the run level, execution_plan.json specifies intended stages and sample identifiers; requests.jsonl and results.jsonl retain corresponding requests and outcomes; sample_ledger.jsonl tracks per-stage status and failures; artifacts.jsonl indexes materialized outputs; and run_manifest.json records the resolved run identity, configuration, runtime context, and output locations. Failed requests remain explicit in result and ledger records, preserving traceability and coverage accounting. This persistent record structure decouples artifact inspection and evaluation from model execution, enabling outputs to be transferred, rescored, compared, and audited without reloading the originating model. It also distinguishes structural conformance, artifact availability, qualitative validity, and benchmark eligibility as separate claims requiring appropriate evidence.

Evaluation and Evidence

WorldFoundry treats evaluation as an artifact-centered process, allowing scoring to proceed independently of the originating model runtime once the required outputs are materialized. The evaluation pipeline resolves the model, benchmark, metrics, data source, runtime profile, and output directory, and maps task or dataset entries to GenerationRequest records with stable sample identifiers, split assignments, and input keys. It supports scoring existing GenerationResult records, generating outputs for registered metrics, executing a complete model–benchmark evaluation, and applying benchmark-specific evaluators to existing artifact directories. Metrics and benchmarks carry distinct responsibilities: metrics compute per-sample measurements and aggregate statistics from compatible artifacts, spanning semantic alignment, quality, distributional similarity, perceptual comparison, and editing or layout assessment, whereas benchmarks additionally prescribe data, sample coverage, preprocessing, artifact layout, evaluator assets, judge configuration, and aggregation rules, with benchmark runners implementing protocol-specific logic and normalizing external evaluator outputs into a shared result structure. Dependencies and evaluator checkpoints are declared explicitly, and unavailable resources yield explicit skipped or failed states rather than undeclared substitutions.

The evidence layer records evaluation outcomes and the conditions under which they support scientific comparisons. Each evaluation preserves its execution plan, configuration, environment and code revision, request and result records, sample coverage, stage-level errors, metric outputs, artifact references, and unresolved blockers. Per-sample measurements are stored in metrics/per_sample.jsonl, aggregate values and failed or skipped sample identifiers in metrics/summary.json, and run metadata and a reproducibility fingerprint in run_manifest.json. These records inform scorecard.json, the authoritative machine-readable record of score validity, provenance, protocol fidelity, coverage, and comparison eligibility. Derived summary.json and report.md files provide concise machine-readable and human-readable views consistent with the scorecard. Evidence is classified as diagnostic, benchmark-comparable, or an exact benchmark reproduction according to fidelity to the prescribed generation settings, data, evaluator, runtime, and reference configuration. Importing official results validates result normalization, while a bounded evaluator run demonstrates integration operability; neither alone establishes complete benchmark reproduction. Leaderboard eligibility further requires prescribed data and coverage, a verified evaluator, compatible runtime evidence, successful metric computation, and no disqualifying blockers. These distinctions constrain reported claims to the available evidence, preventing successful execution or score production from being mistaken for benchmark validity.

Model and Benchmark Zoo

The Model Zoo is a manifest-driven registry that links model discovery to explicit, auditable integration states. Entries fall into five operational categories: video models for conditional generation, editing, animation, and audiovisual synthesis; generative world models for interactive environments, navigation, and simulation; 3D and 4D models for reconstruction, depth estimation, and static or dynamic scene representations; vision-language-action, vision-action, and world-action models for embodied prediction and control; and provider-hosted models accessed remotely. These categories reflect integration requirements rather than mutually exclusive capabilities. Each entry has a stable identifier and optional aliases, with a manifest documenting official sources, licensing and access conditions, tasks and variants, checkpoint references, input and artifact contracts, runtime bindings, hardware requirements, and blockers. Linked records specify runtime contracts, execution backends, and readiness; environment requirements, including Python and CUDA versions, package constraints, validation imports, and environment isolation; and the concrete Pipeline implementation. Evidence records distinguish source-reported results from those obtained through WorldFoundry, while checkpoints, credentials, and other protected or machine-dependent assets remain outside the repository and are resolved through explicit references. Catalog inclusion therefore establishes discoverability, not executable support: source availability, implementation status, asset readiness, runtime validation, and execution evidence are tracked independently, and entries remain planned or blocked until the relevant requirements are met.

The Benchmark Zoo is a parallel registry that links benchmark discovery to task construction, evaluator execution, result normalization, and evidence reporting, covering video quality, semantic alignment, physical and dynamical consistency, camera and spatial behavior, 3D and 4D understanding, interactive world modeling, safety, memory, and embodied control. Each entry has a stable identifier and optional aliases and documentation of target domains, official sources, datasets and splits, task definitions, sample coverage, artifact requirements, metrics, evaluator assets, judge requirements, result formats, runtime profiles, and blockers. Task manifests define how dataset records or prompts become reproducible sample requests, while benchmark manifests specify the evaluation protocol and the supported execution path that most closely follows it. This path may use an in-framework evaluator, an external official runtime, an official-result importer, a hosted judge, or a simulator interface. Integration verification is recorded separately from eligibility for benchmark comparison: importing a schema-conformant official result verifies parsing and normalization, whereas reproducing an official evaluation also requires the prescribed data, artifact coverage, dependencies, assets, credentials, runtime conditions, and protocol settings. Models and benchmarks are paired only when their declared task semantics and artifact requirements are compatible. Each pairing is then classified as diagnostic, benchmark-comparable, or eligible for an exact-reproduction claim based on provenance, coverage, protocol fidelity, and verification evidence.

Reading the table. Catalog inclusion, runnable integration, and runner/evidence verification are distinct maturity claims. The live Model recipes, Benchmark Hub, and Metrics registry remain authoritative for updates.

Empirical Validation and Analysis

We evaluate WorldFoundry using representative models for video generation, interactive world modeling, 3D and 4D generation, and embodied action prediction and control. The evaluation first verifies whether the framework preserves native input and output semantics and produces results consistent with native implementations under matched conditions. It then examines whether equivalent requests issued through the CLI, Python API, and web interface yield consistent configurations and execution behavior. Latency and GPU memory overhead are profiled last, together with the effort required to integrate a new model.

Experimental Setting

Experimental Setting

Experiments are conducted on 128 NVIDIA A100/H100 GPUs with 80 GB of memory each; evaluation on other hardware is left to future work. As model families differ in objectives, inputs, outputs, and execution semantics, we evaluate infrastructure capabilities rather than ranking performance across domains. Specifically, we assess whether the framework resolves declarative catalog entries into executable configurations, preserves model-native semantics across interfaces, stores heterogeneous outputs as persistent artifacts, and supports reported claims with traceable evidence. All workflows share a lifecycle of discovery, preparation, execution, inspection, and evaluation while retaining domain-specific inputs, controls, runtime requirements, and output representations. The evaluation examines whether this shared infrastructure supports reproducible and auditable workflows without imposing a common task, representation, or evaluation metric.

Experimental Results

We report results by model family: video generation (§ 6.3.1), interactive world models (§ 6.3.2), 3D and 4D generation (§ 6.3.3), and embodied action generation (§ 6.3.5). For each family, we describe the task setting, inputs and outputs, and representative systems executed through WorldFoundry. We then present inference validation results with visualizations of the generated outputs and summarize the empirical insights obtained from these executions, comparing qualitative behavior with published findings where noted.

Video Generation.

For video generation, we consider text-to-video synthesis, image-to-video synthesis, and video-to-video editing. These tasks assess whether a model can generate plausible visual dynamics, preserve subject identity and scene structure over time, and satisfy task-specific conditions, capabilities central to modeling how the visual world evolves. Successful generation therefore requires both fidelity to the supplied conditions and coherent temporal progression in appearance, spatial relationships, and motion. Depending on the task, inputs consist of a textual description alone or of visual conditions, such as reference images or source videos, paired with instructions specifying the desired content, motion, or transformation; additional reference modalities and structural constraints are supplied where the model supports them. Representative systems include CogVideoX [31], Vchitect-2.0 [32], HunyuanVideo 1.5 [33], LTX-2.5 [34], Wan2.1 [35], Wan2.1 VACE [36], Wan2.2 [37], LingBot-Video [38], and MiniMax H3 [39], which span diverse conditioning mechanisms and generation pipelines. WorldFoundry preserves each system’s native input processing and inference settings, so that results reflect its intended operation. Figure 7 shows representative text-to-video results from LTX-2.5, Wan2.2, and MiniMax H3, covering both camera motion through largely static scenes and object-centric dynamics such as running animals and a hovering bird.

Figure 7. (a) Ltx2.5 — Generated after the prompt “Zebras run across a grassy savanna, kicking up dust in warm daylight.” (b) Ltx2.5 — Generated after the prompt “Waves wash along a sunlit stone seawall as the camera glides forward.” (c) Wan 2.2 — Generated after the prompt “Pan across a sandy, rocky beach with gentle waves under a clear blue sky.” (d) Wan 2.2 — Generated after the prompt “Pan across blue water and misty mountain ridges, with trees in the foreground.” (e) Minimax-h3 — Generated after the prompt “Pan along a tropical beach with palms, seabirds on rocks, and a small boat.” (f) Minimax-h3 — Generated after the prompt “A hummingbird takes off and hovers beside sunlit leaves, rapidly beating its wings.” Demonstration of inference results from representative video generation models.

Qualitative analysis highlights distinct strengths across generation tasks. In text-conditioned synthesis, CogVideoX and Vchitect-2.0 capture scene composition and broad motion patterns, but complex motion and multi-object interactions expose inconsistencies in identity, fine detail, and spatial relationships. FLAG3D++ [40] provides detailed and professional language instructions to describe how to perform a specific activity, as well as the corresponding accurate and dense 3D human pose captured from an advanced MoCap system to handle complex activities and large movements. HunyuanVideo 1.5 and Wan2.1 produce coherent short sequences, particularly with image conditioning, although appearance preservation does not consistently translate into plausible motion under occlusion or substantial deformation. Published results for Wan2.2 [37] report improvements in motion generation and aesthetic control over Wan2.1, extending the range of visually convincing sequences. LingBot-Video provides a complementary advance: its reported post-training comparisons [38] show reduced object deformation, interpenetration, and premature object release, indicating gains in physical interaction beyond perceptual quality alone. MiniMax H3 broadens reference-based generation and editing, with official demonstrations combining appearance, motion, and audio cues from different sources [39]; these examples demonstrate compositional flexibility but do not establish reliable physical dynamics. Together, these observations suggest that progress unfolds along several distinct dimensions: visual refinement, preservation of conditioning information, and consistency of the depicted events. A sequence may remain visually smooth and recognizable while violating contact relations or failing to complete the requested transformation, which makes event-level consistency a necessary complement to frame-level quality in video-generation analysis.

Interactive World Model.

For interactive world modeling, we consider navigation, camera-controlled exploration, and action- or instruction-driven interaction within generated environments. These tasks assess whether a model can maintain a coherent environment as actions unfold, retain information beyond the immediate visual context, and carry the consequences of earlier interactions into subsequent predictions. Unlike offline video synthesis, the model produces an evolving observation stream conditioned on successive interventions, allowing later actions to respond to previously generated observations. Inputs comprise an initial image or image sequence, the available interaction history, and model-supported controls, including directional movement, camera rotation, discrete game actions, and textual instructions. Successful generation therefore requires both immediate responsiveness to controls and persistent consistency in scene geometry, object identity, and interaction outcomes, including when previously observed regions are revisited. Representative systems include the Matrix-Game series up to the recent 3.5 release [41], LingBot-World 1.0 and 2.0 [42, 43], Hunyuan GameCraft and GameCraft-2 [44, 45], YUME-1.5 [46], HY-WorldPlay [47], SCOPE [48], SANA-WM [49], and Echo-WM [50]. WorldFoundry preserves each system’s native action vocabulary, history-conditioning mechanism, and generation schedule, and records controls, observations, and runtime configurations to support reproducible trajectory-level analysis. Figure 8 shows 30-second rollouts from five representative systems driven by directional movement and mouse-based camera control across diverse scenes.

Figure 8. Demonstration of inference results from representative world models.

Qualitative comparison highlights progress along two complementary dimensions: scene persistence and interaction expressiveness. Matrix-Game 2.0 [51] enables responsive navigation but exhibits appearance drift over extended rollouts, whereas LingBot-World 1.0, Hunyuan GameCraft, YUME-1.5, and HY-WorldPlay produce visually coherent navigation sequences in the examined cases. Recent memory-augmented systems address consistency beyond the immediate context: Matrix-Game 3.0 [52] introduces long-horizon memory, while published results for Matrix-Game 3.5 [41] and ReWorld [53] demonstrate scene recall through geometry-aware retrieval and pose-indexed landmarks, respectively. These advances make revisitation more informative than uninterrupted forward motion for assessing whether a model retains previously observed content. In parallel, LingBot-World 2.0 [43] and Hunyuan-GameCraft-2 [45] broaden interaction to diverse actions and instruction-driven events, while SCOPE [48] demonstrates localized firing and reloading responses alongside continuous movement and camera control. However, scene recall and action responsiveness are distinct from persistent interaction outcomes: recovering a familiar view does not establish that an opened door remains open or that a displaced object retains its new position after leaving the field of view. The resulting evaluation priority is therefore to distinguish memory of scene appearance from memory of changes caused by actions. Reliable interactive world modeling requires both: stable recovery of unchanged content and consistent propagation of action-dependent state changes across subsequent observations.

3D & 4D Generation.

For 3D and 4D generation, we consider static scene reconstruction, scene synthesis, and dynamic scene modeling. These tasks assess whether a model can recover or generate spatial structure, maintain consistency across viewpoints, and represent changes over time. Successful reconstruction requires geometry and camera estimates consistent with the observed views, while conditional synthesis requires spatially coherent content that follows the supplied visual or textual conditions. Dynamic scene modeling additionally requires distinguishing camera motion from scene motion and preserving temporal correspondence through deformation, occlusion, and reappearance. Depending on the task, inputs comprise single images, multi-view observations, video sequences, or textual descriptions, and outputs include point maps, camera parameters, point clouds, meshes, Gaussian representations, and time-dependent geometry. Representative systems include DUSt3R [54], CUT3R [55], VGGT [56], VGGT-Omega [57], Splatt3R [58], and FlashWorld [59] for spatial reconstruction or synthesis, alongside MonST3R [60], NeoVerse [61], and Lyra [62] for complementary video-based and spatiotemporal workflows. Evaluation criteria include camera consistency, reconstruction completeness, novel-view quality, inference efficiency, and, where reference geometry is available, geometric accuracy, with additional attention to temporal stability in dynamic scenes. WorldFoundry preserves each method’s native inputs and representations, organizes comparisons within compatible task families, and records configurations, rendered views, and exported artifacts for reproducible spatial and temporal analysis. Figure 9 compares VGGT, VGGT-Omega, DUSt3R, and CUT3R on the same indoor scene, rendering each reconstruction from a common sequence of viewpoints.

Figure 9. Demonstration of inference results from representative 3D scene generation models.

Qualitative analysis highlights a distinction between reconstructing observed structure and generating plausible spatial completion. DUSt3R recovers geometry through pairwise point-map prediction, while VGGT jointly estimates camera parameters, depth, point maps, and point tracks from multiple views; VGGT-Omega further develops this feed-forward reconstruction paradigm. Splatt3R complements geometric reconstruction with Gaussian appearance attributes for novel-view rendering, although convincing rendered views do not establish accurate geometry in sparsely observed or occluded regions. FlashWorld enables efficient scene synthesis through direct Gaussian generation, while Lyra transfers video-generation priors into feed-forward 3D and 4D reconstruction, illustrating how learned priors can supply plausible content beyond the available observations. In dynamic scenes, MonST3R accommodates scene motion when estimating geometry and cameras, and NeoVerse extends monocular reconstruction to time-varying scenes and novel camera trajectories. The central ambiguity is whether image changes arise from camera motion, object motion, or newly visible surfaces: temporal smoothness alone cannot establish correct geometric correspondence. For extended exploration, Lyra 2.0 [63] reports improved consistency through geometry-guided retrieval and training that mitigates accumulated generation drift. Together, these developments motivate distinguishing three properties: agreement with observed evidence, cross-view consistency of inferred content, and temporal persistence of scene structure. A representation can be visually convincing and internally consistent while remaining inaccurate in unobserved regions; reconstructing a dynamic sequence likewise does not establish the ability to predict its evolution under alternative actions.

Embodied Action Generation.

Embodied Action Generation.

For embodied action generation, we consider visual policy learning, language-conditioned manipulation, and control informed by predicted environmental dynamics. These tasks assess whether a model can translate observations and task specifications into executable actions, coordinate perception with control, and adapt to feedback during execution. Inputs comprise visual observations and proprioceptive states, supplemented by natural-language instructions or predicted future observations where supported. Outputs comprise discrete actions, continuous control commands, or action sequences expressed in the target embodiment’s native action space. The integrated systems follow three complementary paradigms: Vision-Action (VA) policies, which generate actions from visual and robot-state observations; Vision-Language-Action (VLA) models, which additionally incorporate linguistic instructions; and World-Action Models (WAMs), which couple action generation with predictions of environmental evolution. Representative integrations include Action Chunking Transformer [64], OpenVLA [65], RoboFlamingo [66], MolmoAct2 [67], GigaBrain-0 [68], Being-H0.5 [69], StarVLA [19], LingBot-VLA 1.0 and 2.0 [70, 71], and LingBot-VA 1.0 and 2.0 [72, 73]. Evaluation combines offline action prediction with closed-loop execution in compatible manipulation environments, including LIBERO and RoboTwin [74, 75]; criteria include task completion, instruction adherence, control accuracy, trajectory smoothness, execution latency, and recovery from intermediate errors. WorldFoundry preserves each system’s observation history, camera configuration, action coordinates, normalization statistics, control frequency, and action-chunk horizon, with comparisons restricted to compatible embodiments and task protocols. Figure 10 shows closed-loop LIBERO-10 episodes of OpenVLA executed through WorldFoundry: the policy completes a single-object pick-and-place task but reaches the step limit on two multi-stage tasks, illustrating why closed-loop completion, rather than offline action accuracy, is the relevant measure.

Figure 10. Representative closed-loop embodied results. OpenVLA executed through WorldFoundry on LIBERO-10, with one recorded episode per task. The policy completes task (a) within 179 steps, whereas tasks (b) and (c) reach the 520-step limit before completion.

Published empirical findings indicate that semantic grounding, embodiment transfer, and predictive control contribute to embodied competence through distinct mechanisms. SuperMimic [76] demonstrates that a shared differentiable world model can efficiently scale humanoid whole-body control through short imagined rollouts and analytic policy gradients, substantially reducing the computational cost of policy learning. OpenVLA [65] and RoboFlamingo [66] demonstrate that pretrained vision-language representations can support language-conditioned manipulation, although transfer remains dependent on the target embodiment and adaptation data. MolmoAct2 [67] combines embodied reasoning with a dedicated action expert, exposing a trade-off between additional spatial reasoning and control latency. GigaBrain-0 [68] improves generalization through world-model-generated training data, whereas Being-H0.5 [69] aligns human and robot motion within a shared action representation. StarVLA [19] provides complementary policy variants for examining how action decoding and training choices affect execution, rather than defining a single control mechanism. LingBot-VLA 1.0 [70] demonstrates effective adaptation across robot platforms, while LingBot-VLA 2.0 [71] extends this approach to whole-body control and incorporates future semantic and geometric prediction as auxiliary supervision. LingBot-VA 1.0 [72] instead couples video and action prediction within a causal sequence model; LingBot-VA 2.0 [73] further introduces action-aligned visual representations and asynchronous inference that incorporates the latest observations during execution. These distinctions suggest that the value of world modeling depends on where prediction enters the control pipeline: as training-data augmentation, representation supervision, or an online component of action generation. In particular, plausible future observations are useful only insofar as they support executable actions and timely correction when reality departs from prediction. The decisive evidence is therefore improved closed-loop completion and recovery under comparable data, embodiment, and latency constraints, rather than predictive visual quality or offline action accuracy alone.

Limitations and Future Work

Although WorldFoundry provides a unified operational foundation for diverse generative world-modeling systems, its broad coverage encompasses different levels of execution maturity. Catalog inclusion, runnable integration, local asset readiness, official runtime validation, and benchmark eligibility remain distinct states, reflecting the external checkpoints, licensed datasets, hosted services, simulator assets, and dedicated environments required by individual models and benchmarks. Differences in architecture and hardware requirements also prevent a single optimization strategy from benefiting every video, spatial, interactive, and embodied model equally. In addition, the present framework focuses on inference, interaction, artifact management, and evaluation. A general training infrastructure remains a direction for future development, and upstream training code is intentionally separated from current runtime packages. This scope provides a consistent and auditable foundation for experimentation, while leaving opportunities to extend lifecycle reproducibility as public datasets and reproducible training recipes become available across domains.

The inference core already exposes a broad set of optimization mechanisms: configurable attention backends, including FlashAttention 2 [77] and 3 [78], SageAttention [79], xFormers, and SDPA; Triton kernel registration; compilation and inference caching; NVFP4 quantization where supported; hardware-aware GPU selection; and multi-GPU context and sequence parallelism. Their effects, however, have not yet been characterized systematically across model families, and validation has primarily used A100 and H100 GPUs; limited access to B200- and B300-class systems constrains assessment on newer accelerators. Future work will combine broader hardware validation with runtime profiling, automatic backend selection, improved compilation and cache reuse, fidelity-aware quantization, and more efficient memory management and distributed execution. We will also integrate dedicated serving engines through modular backend adapters and runtime profiles that preserve native input semantics, artifact contracts, and evaluation protocols. Improvements will be assessed by end-to-end latency, throughput, memory use, scaling efficiency, and reproducibility, since kernel-level acceleration alone does not establish application-level gains. An optimization will be adopted only if its outputs remain within a declared fidelity tolerance of the unoptimized route.

Beyond inference optimization, we plan to extend WorldFoundry to support data preparation and training. Initial development will focus on representative baselines and public datasets, reflecting the uneven availability of reproducible training recipes across domains. The extension will support versioned dataset manifests, distributed training, fine-tuning and post-training, checkpoint lineage, experiment recovery, and direct checkpoint registration in the Model Zoo. Existing contracts will link data and optimization configurations to checkpoints, inference artifacts, and benchmark results. Since reproducible training alone does not guarantee consistent deployment behavior, we will explicitly align preprocessing, conditioning, normalization, coordinate conventions, and checkpoint interpretation across training and inference. Reusable configurations, guided setup, and actionable diagnostics will ease adoption while retaining domain-specific controls. Training, execution, and evaluation will remain distinct responsibilities within an auditable research lifecycle, preserving architectural flexibility and traceability from data preparation to reported results.

Conclusion

We introduce WorldFoundry as open-source infrastructure built on an operational definition of generative world models as systems with persistent state, intervention-conditioned dynamics, and observational feedback. This definition shifts the criterion for world modeling from the quality of individual outputs to the behavior of the loop that produces them: a generated observation counts as evidence of a world model only insofar as it is constrained by a state that past interventions have altered and that new observations can revise. WorldFoundry relies on this view across video, 3D and 4D, interactive world, and embodied action systems, cataloging 270 model entries, 74 benchmarks, and 62 metrics under shared contracts that preserve native semantics. By recording catalog inclusion, executability, and validated performance as distinct claims, it keeps every reported result traceable to the artifacts and conditions that support it. Our cross-family analysis indicates that visual fidelity, scene consistency, and action responsiveness, although complementary, cannot by themselves certify world modeling; what current evaluations seldom measure is whether intervention effects persist and whether incoming observations correct the state. Future work will develop evaluations that directly probe intervention persistence and feedback assimilation, alongside broader inference optimization, hardware validation, and reproducible training with explicit training–inference consistency. By connecting capability definitions, executable workflows, and traceable evidence, WorldFoundry provides a shared foundation for measuring progress toward world intelligence.

References

Appendix

Author Contributions

  • OpenEnvision Team: Project Team.

  • Juanxi Tian (NTU): Project Leader & Core Contributor. Leads the project, designs and implements WorldFoundry, and writes the technical report.

  • Jiadong Hou (NTU): Core Contributor. Co-develops WorldFoundry and coordinates its updates.

  • Baiang Li (Princeton): Contributor. Reviews and helps refine WorldFoundry’s video generation pipeline.

  • Xin Lin (UCSD): Contributor. Reviews and helps refine WorldFoundry’s 3/4D generation pipeline.

  • Zhengbo Jiao (HKUST): Contributor. Tests WorldFoundry’s pipelines and supports outreach for the framework.

  • Weijia Zhang (Yale): Contributor. Tests WorldFoundry’s pipelines and supports outreach for the framework.

  • Yeying Jin (NUS): Advisor. Provides advice on writing the technical report.

  • Truong Nguyen (UCSD): Advisor. Provides advice on writing the technical report.

  • Haitian Zheng (Adobe): Advisor. Offers advice on WorldFoundry’s design and development.

  • Zhifei Zhang (Adobe): Advisor. Offers advice on WorldFoundry’s design and development.

  • Jianfei Yang (NTU): Advisor. Offers advice on WorldFoundry’s design and development.

  • Ming-Hsuan Yang (UC Merced): Advisor. Offers advice on WorldFoundry’s design and development.

  • Felix Heide (Princeton): Project Supervisor. Supervises the WorldFoundry project and provides guidance on the technical report.

  • Wei Xiong (Nvidia): Project Supervisor. Supervises the WorldFoundry project and provides guidance on the technical report.

  • Enze Xie (Nvidia): Project Supervisor. Supervises the WorldFoundry project and provides guidance on the technical report.

  • Wanhua Li (NTU): Project Supervisor. Supervises the WorldFoundry project and provides guidance on the technical report.

Supported Models

The WorldFoundry Model Zoo comprises 270 distinct registry entries, organized according to the five families in Table 2: 64 video generation models (including multimodal generation), 109 world models, 34 3D/4D models, 53 embodied models, and 10 hosted API models. Table 3–Table 7 enumerate every entry with its stable registry ID, upstream reference, and readiness status. Checkpoint variants and aliases are consolidated under a single entry unless the registry assigns them separate IDs. Model names link to the corresponding papers, technical reports, official releases, model cards, or API documentation. Where no upstream publication exists, the source is the official release or a registry record.

FamilyRepresentative integrationsNative tasks
MODEL ZOOExecutable systems and native artifacts
Video generationCogVideoX · HunyuanVideo series · LTX series · MiniMax H3 · SANA Video series · Wan seriesText-, visual-, and audio-conditioned video synthesis; video editing
Interactive world modelsMatrix-Game series · LingBot-World series · Hunyuan GameCraft series · Cosmos 3 · SANA WM · YUME-1.5 · SCOPEAction- and camera-conditioned rollouts; navigation and scene interaction
3D & 4DDUSt3R · VGGT / VGGT-Omega · Splatt3R · FlashWorld · MonST3R · NeoVerse · Lyra3D reconstruction and generation; dynamic scene modeling
Embodied actionACT · OpenVLA · Lingbot-VLA / VA · MolmoAct2 · GigaBrain-0 · Being-H0.5 · StarVLAAction prediction; policy execution; language-conditioned control
Hosted APIsSeedance 2 / 2.5 · Kling API · Runway Gen-4.5 · Sora2 · Veo3 · World Labs APIAPI-based video and world generation
BENCHMARK ZOOEvaluation protocols and evidence
DomainRepresentative benchmarksEvaluation focus
Video generationVBench series · T2V-CompBench · ChronoMagic-Bench · VideoScore · VideoPhy2 · T2VSafetyBenchPerceptual quality; semantic alignment; compositionality; temporal dynamics; physical plausibility; content safety
World modelsWorldBench · WorldModelBench · WorldScore · 4DWorldBench · iWorld-Bench · WorldAtlasDynamics fidelity; 3D/4D consistency; memory retention; controllability; interactive task performance
Embodied AIAI2-THOR · BEHAVIOR-1K · CALVIN · LIBERO · RoboTwin series · WorldArena series · RoboDojoClosed-loop task completion in simulated environments
METRIC REGISTRYIn-tree scorers and compute APIs
FamilyRepresentative metricsMeasurement focus
Scorers & qualityCLIPScore · VQAScore · ITMScore · FaceScore · ArtScore · OpenS2VSemantic alignment; identity consistency; aesthetic quality; subject-to-video quality
DistributionFID · IS · KID · FVD · CMMD · Clean-FID · JEDiImage/video distributional fidelity and sample diversity
Perceptual pairwiseLPIPS · SSIM / MS-SSIM · PSNR · DINO / DreamSim · mask accuracy · LQSPerceptual and structural similarity; mask accuracy; layout quality
Editing & layoutSemSR · IRS · CAS · manipulation direction · VS similarity · object-wise consistencySemantic edit success; manipulation direction; object-level consistency

Table 2. Representative coverage of the WorldFoundry Model Zoo, Benchmark Zoo, and Metric Registry. Entries are illustrative rather than exhaustive or uniformly mature.

ModelRegistry IDReadiness
AllegroallegroRunner verified
AnimateDiffanimatediffRunner verified
BerniniberniniRunner verified
Bernini-R 1.3Bbernini-r-1.3bRunner verified
Bernini-R 14Bbernini-r-14bRunner verified
CogVideoXcogvideoxRunner verified
DynamiCrafterdynamicrafterRunner verified
EasyAnimateeasyanimateRunner verified
Emu3.5emu3.5Runner verified
FastVideo CausalWan2.2fastvideo-causal-wan2.2Runner verified
FramePackframepackRunner verified
HeliosheliosRunner verified
HunyuanVideohunyuanvideoRunner verified
HunyuanVideo 1.5hunyuanvideo-1.5Runner verified
HyperFlow for MiniMax-H3hyperflowRunner verified
I2VGen-XLi2vgen-xlRunner verified
JoyAI-Echo 1.5 / Echo-LongVideojoyai-echo-longvideoRunner verified
Kandinsky 6.0 litekandinsky6-liteRunner verified
Kandinsky 6.0 lite-distillkandinsky6-lite-distillRunner verified
Kandinsky 6.0 lite-pretrainkandinsky6-lite-pretrainRunner verified
Kandinsky 6.0 prokandinsky6-proRunner verified
Kandinsky 6.0 pro-distillkandinsky6-pro-distillRunner verified
Kandinsky 6.0 pro-pretrainkandinsky6-pro-pretrainRunner verified
Krea Realtime Video 14Bkrea-realtime-videoRunner verified
LingBot-Videolingbot-videoRunner verified
LongCat-Videolongcat-videoRunner verified
LongVielongvie-1Runner verified
LongVie 2longvie-2Runner verified
LTX-2.5 Distilledltx-2.5Runner verified
LTX-2.xltx-2.xRunner verified
LTX-Videoltx-videoRunner verified
MAGI-1magi-1Runner verified
MAGI-2 Previewmagi2-previewRunner verified
MiniMax H3minimax-h3Runner verified
MMAudiommaudioRunner verified
Mochi-1mochi-1Runner verified
ModelScopeT2Vmodelscope-t2vRunner verified
OmniVinciomnivinciRunner verified
Open-MAGVIT2open-magvit2Runner verified
Open-Soraopen-soraRunner verified
Open-Sora-Planopen-sora-planRunner verified
Pusa VidGenpusa-vidgenRunner verified
Qwen2.5-Omniqwen2.5-omniRunner verified
ModelRegistry IDReadiness
SAMAsama-14bRunner verified
SanasanaRunner verified
Show-oshow-oRunner verified
SkyReels V3skyreels-v3Runner verified
SkyReels-V2skyreels-v2Runner verified
SpatialLadderspatial-ladderRunner verified
SpatialReasonerspatial-reasonerRunner verified
Stable Video Infinity 2.0stable-video-infinityRunner verified
Step-Video-T2Vstep-video-t2vRunner verified
T2V Turbo T2Vt2v_turbo_t2vRunner verified
UniAnimate-DiTunianimate-ditRunner verified
Vchitect-2.0 T2Vvchitect-2-t2vRunner verified
VideoCraftervideocrafterRunner verified
VideoCrafter1 I2Vvideocrafter1-i2vRunner verified
VideoCrafter1 T2Vvideocrafter1-t2vRunner verified
VideoCrafter2 T2Vvideocrafter2-t2vRunner verified
VMemvmemRunner verified
Wan2.1wan2.1Runner verified
Wan2.1 VACEwan2.1-vaceRunner verified
Wan2.2wan2.2Runner verified
ZeroScopezeroscopeRunner verified

Table 3. Video generation models (64 entries).

ModelRegistry IDReadiness
Hailuo 2.3hailuo-2p3Runner verified
Kling APIkling-apiRunner verified
Luma Ray2luma-ray2Runner verified
Runway Gen-4.5runway-gen4p5Runner verified
Sora2sora2Runner verified
Veo3veo3Runner verified
Wan 2.5 APIwan-2p5Runner verified
Wan 2.6 APIwan-2p6Runner verified
Wan 2.7 APIwan-2p7Runner verified
World Labs APIworldlabsRunner verified

Table 7. Hosted API models (10 entries).

Readiness is evidence-aware; all 270 entries are marked Runner verified in this appendix. Verification scope is specific to the corresponding execution route, model variant, and configuration.

ModelRegistry IDReadiness
ABot-World-0-5B-LFabot-world-0-5b-lfRunner verified
AC3Dac3dRunner verified
AdaWorldadaworldRunner verified
AlayaWorldalayaworldRunner verified
AlayaWorld v1.1alayaworld-v1.1Runner verified
AstraastraRunner verified
Astronex-Worldastronex-worldRunner verified
ATI-Wan2.1-14Bati-wan21-14bRunner verified
BiWM Wan2.1biwm-wan21Runner verified
BiWM Wan2.2biwm-wan22Runner verified
CameraCtrlcameractrlRunner verified
Causal-Forcingcausal-forcingRunner verified
Causal-rCMcausal-rcmRunner verified
Cosmos Predict 2cosmos-predict-2Runner verified
Cosmos Predict 2.5cosmos-predict-2.5Runner verified
Cosmos Transfer 2.5cosmos-transfer-2.5Runner verified
Cosmos3cosmos3Runner verified
Ctrl-Worldctrl-worldRunner verified
DIAMONDdiamondRunner verified
DINO-WMdino-wmRunner verified
DreamDojodreamdojoRunner verified
DreamX-World 5Bdreamx-world-5bRunner verified
DualCamCtrldualcamctrlRunner verified
Echo-Infinityecho-infinityRunner verified
Echo-Memoryecho-memory-context-k1Runner verified
Echo-WMjoyai-echo-wmRunner verified
EgoWMegowmRunner verified
EvokeevokeRunner verified
Gamma-Worldgamma-worldRunner verified
GEN3Cgen3cRunner verified
Genie Envisionergenie-envisionerRunner verified
ModelRegistry IDReadiness
GigaWorld-0giga-world-0Runner verified
HappyOysterhappyoysterRunner verified
HelixWorld Preview v1helixworldRunner verified
HMAhmaRunner verified
Hunyuan Game Crafthunyuan-game-craftRunner verified
HunyuanWorld 1.0hunyuanworld-1Runner verified
HunyuanWorld-Mirrorhunyuanworld-mirrorRunner verified
HunyuanWorld-Voyagerhunyuanworld-voyagerRunner verified
HY-World 2.0hy-world-2.0Runner verified
HY-World 2.0 Trajectory Rendererhyworld-worldgenRunner verified
HY-WorldPlayhy-worldplayRunner verified
HyDRAhydraRunner verified
Infinite-Worldinfinite-worldRunner verified
InSpatio World 1.5inspatio-world-1p5Runner verified
InSpatio-Worldinspatio-worldRunner verified
InSpatio-WorldFMworldfmRunner verified
IRASimirasimRunner verified
Kairos Sensenovakairos-sensenovaRunner verified
LeWorldModelleworldmodelRunner verified
LingBot-Worldlingbot-worldRunner verified
LingBot-World Base Actlingbot-world-actRunner verified
LingBot-World-V2lingbot-world-v2Runner verified
LiveWorldliveworldRunner verified
MagicWorldmagicworldRunner verified
Matrix-Game 1.0matrix-game-1Runner verified
Matrix-Game 2.0matrix-game-2Runner verified
Matrix-Game 3.0matrix-game-3Runner verified
Matrix-Game 3.5matrix-game-3.5-first-personRunner verified
MineWorldmineworldRunner verified
minWMminwm-hy-action2vRunner verified
MIRAmiraRunner verified
MosaicMemmosaicmemRunner verified
MotionBricksmotionbricksRunner verified
MotionCtrlmotionctrlRunner verified
MoVersemoverseRunner verified
MultiWorldmultiworldRunner verified
Navigation World ModelsnwmRunner verified
Oasis 500Moasis-500mRunner verified
OmniForcingomniforcingRunner verified
Open Dreameropen-dreamerRunner verified
PandorapandoraRunner verified
PointWorldpointworldRunner verified
RollingForcingrolling-forcingRunner verified
SANA-WMsana-wmRunner verified
SCOPEscopeRunner verified
Self-Forcingself-forcingRunner verified
ShotStreamshotstreamRunner verified
SimWorldsimworldRunner verified
SolarissolarisRunner verified
SolarWMsolarwmRunner verified
SpatiaspatiaRunner verified
StarWMstarwmRunner verified
TesserActtesseractRunner verified
Uni3Cuni3cRunner verified
Unified World ModelsuwmRunner verified
UniWorld-Viewuniworld-viewRunner verified
VerseCrafterversecrafterRunner verified
VGGT-Worldvggt-worldRunner verified
Vid2Worldvid2worldRunner verified
ModelRegistry IDReadiness
Wan2.1-Fun V1.1 1.3B Control Camerawan21-fun-1p3b-camRunner verified
Wan2.1-Fun V1.1 14B Control Camerawan21-fun-14b-camRunner verified
Wan2.2-Fun 5B Control Camerawan22-fun-5b-camRunner verified
Wan2.2-Fun A14B Control Camerawan22-fun-a14b-camRunner verified
Warp-as-Historywarp-as-historyRunner verified
WildDet3Dwilddet3dRunner verified
WildWorldwildworldRunner verified
World Labs Marble 1.1worldlabs-marble-1.1Runner verified
WorldCamworldcamRunner verified
WorldCrafterworldcrafterRunner verified
WorldCrafter Baseworldcrafter-baseRunner verified
WorldCrafter Fastworldcrafter-fastRunner verified
WorldGrowworldgrowRunner verified
WorldMemworldmemRunner verified
WorldPlay2worldplay2Runner verified
WoWwowRunner verified
XGEN-JING Flash v1xgen-jingRunner verified
YUME (1.5 paper)yumeRunner verified
Zing-0.5zingRunner verified

Table 4. World models (109 entries).

ModelRegistry IDReadiness
CUT3Rcut3rRunner verified
Depth Any PanoramasdapRunner verified
Depth Anything 3depth-anything-v3Runner verified
Depth Anything 3 Metric Priordepth-anything-v3-priorRunner verified
Depth Anything V1depth-anything-v1Runner verified
Depth Anything V2depth-anything-v2-priorRunner verified
DUSt3Rdust3rRunner verified
Déjà View Looping TransformerdvltRunner verified
FantasyWorldfantasyworldRunner verified
FlashWorldflashworldRunner verified
GeoCalibgeocalib-priorRunner verified
Infinite VGGTinfinite-vggtRunner verified
LagrNVSlagernvsRunner verified
LingBot-Maplingbot-mapRunner verified
LoGeRlogerRunner verified
Lyra (2.0 paper)lyraRunner verified
Metric3Dmetric3d-priorRunner verified
MonST3Rmonst3rRunner verified
MVDiffusionmvdiffusionRunner verified
NeoVerseneoverseRunner verified
Pi3pi3Runner verified
pixelSplatpixelsplatRunner verified
Prior Depth Anythingprior-depth-anythingRunner verified
ReCamMasterrecammasterRunner verified
Segment and Track Anythingtrack-anything-priorRunner verified
Splatt3Rsplatt3rRunner verified
Stable Virtual Camerastable-virtual-cameraRunner verified
UniDepth V2unidepth-v2-priorRunner verified
UniK3Dunik3d-priorRunner verified
VGGTvggtRunner verified
VGGT-Omegavggt-omegaRunner verified
Video Depth Anythingvideo-depth-anything-priorRunner verified
WonderJourneywonderjourneyRunner verified
ModelRegistry IDReadiness
WorldGenworldgenRunner verified

Table 5. 3D and 4D models (34 entries).

ModelRegistry IDReadiness
A1a1Runner verified
ABot-M0abot-m0Runner verified
Action Chunking TransformeractRunner verified
AHA-WAMahawamRunner verified
Being-H0.5being-h05Runner verified
Being-H0.7being-h07Runner verified
CogACTcogactRunner verified
DB-CogACT / Dexboticdb-cogactRunner verified
Dexora 1Bdexora-1bRunner verified
Diffusion Policydiffusion-policyRunner verified
DM0dm0Runner verified
EventVLAeventvlaRunner verified
FastWAMfastwamRunner verified
Galaxea G0Plusgalaxea-vlaRunner verified
GigaBrain-0giga-brain-0Runner verified
GigaWorld-Policy-0.5giga-world-policy-0.5Runner verified
GO-1go1Runner verified
H-RDTh-rdtRunner verified
Hy-Embodied-0.5-VLAhy-embodied-vlaRunner verified
InternVLA-A1-3Binternvla-a1Runner verified
LaST-R1last-r1Runner verified
Latent Action PretraininglapaRunner verified
LDA-1Blda-1bRunner verified
LingBot-VAlingbot-vaRunner verified
LingBot-VLAlingbot-vlaRunner verified
LingBot-VLA 2.0lingbot-vla-v2Runner verified
Mem-0mem-0Runner verified
MME-VLAmme-vlaRunner verified
MolmoAct2molmoact2Runner verified
MolmoBotmolmobotRunner verified
NVIDIA DreamZerodreamzeroRunner verified
NVIDIA GR00Tgr00tRunner verified
OctooctoRunner verified
OpenPIopenpiRunner verified
OpenVLAopenvlaRunner verified
OpenVLA-OFTopenvla-oftRunner verified
pi0pi0Runner verified
pi0-FASTpi0-fastRunner verified
pi0.5pi05Runner verified
RDT-1Brdt-1bRunner verified
RISEriseRunner verified
RoboFlamingoroboflamingoRunner verified
RT-1rt-1Runner verified
SmolVLAsmolvlaRunner verified
Spatial-Forcingspatial-forcingRunner verified
Spirit-VLA v1.5spirit-v1.5Runner verified
StarVLAstarvlaRunner verified
TinyVLAtinyvlaRunner verified
VLANeXtvlanextRunner verified
X-VLAxvlaRunner verified
X-WAMx-wamRunner verified
ModelRegistry IDReadiness
Xiaomi-Robotics-0xiaomi-robotics-0Runner verified
Xiaomi-Robotics-1xiaomi-robotics-1Runner verified

Table 6. Embodied models (53 entries).

Supported Benchmarks

The WorldFoundry Benchmark Hub documentation lists 74 benchmarks. Following the evaluation domains in Table 2, we organize them into 23 embodied AI, 33 video generation, and 18 world model benchmarks. Table 8–10 summarize each benchmark’s evaluation focus and reported status. Benchmark names link to their sources.

BenchmarkEvaluation focusStatus
AI2-THORInteractive household navigation and manipulationIntegrated & Verified
BEHAVIOR-1K1,000 household activities and BDDL goalsIntegrated & Verified
BridgeData V2Real robot trajectories; language and goal image conditioningIntegrated & Verified
CALVINLanguage instructed manipulation chains (length 1–5)Integrated & Verified
KinetixOpen ended 2D physics control tasksIntegrated & Verified
LARYBenchLatent action extraction, classification, and regressionIntegrated & Verified
LIBEROSpatial, Object, Goal, and LIBERO-100 lifelong manipulationIntegrated & Verified
LIBERO-MemObject motion, sequence, relation, and occlusion memoryIntegrated & Verified
LIBERO-PlusSeven perturbation factors; robustness at five difficulty levelsIntegrated & Verified
LIBERO-ProObject, configuration, instruction, task, and environment robustnessIntegrated & Verified
ManiSkillGPU parallel SAPIEN manipulation skill familiesIntegrated & Verified
ManiSkill220 SAPIEN manipulation task families; multiple embodimentsIntegrated & Verified
Meta-World50 manipulation tasks; multitask and meta learning suitesIntegrated & Verified
MiKASA-RoboObject, spatial, sequential, and capacity memoryIntegrated & Verified
MolmoSpaces-BenchPick and place variants; stage and final task successIntegrated & Verified
RLBench100 vision guided manipulation tasks and procedural variationsIntegrated & Verified
RoboCasaAtomic and long horizon kitchen manipulationIntegrated & Verified
RoboCerebraLong horizon manipulation with memory and disturbancesIntegrated & Verified
RoboMMECounting, permanence, reference, and imitation memoryIntegrated & Verified
RoboTwin 2.050 dual arm tasks; clean and randomized settingsIntegrated & Verified
SimplerEnvReal to simulation evaluation on Google Robot and WidowXIntegrated & Verified
VLABenchLanguage conditioned manipulation with long horizon reasoningIntegrated & Verified
WorldArenaEmbodied perceptual quality and functional utilityIntegrated & Verified

Table 8. Supported embodied AI benchmarks (23 entries).

All 74 entries are marked Integrated & Verified in this appendix. Verification scope is specific to the corresponding benchmark recipe and configuration, including its required datasets, checkpoints, judges, and execution environments.

BenchmarkEvaluation focusStatus
AIGCBenchImage to video fidelity, alignment, motion, and consistencyIntegrated & Verified
CameraBenchCamera motion classification, VQA, retrieval, and captioningIntegrated & Verified
BenchmarkEvaluation focusStatus
ChronoMagic-BenchTime lapse coherence and metamorphic amplitudeIntegrated & Verified
DEVIL DynamicsDynamics range, controllability, and qualityIntegrated & Verified
EvalCrafterVisual quality, alignment, motion, and temporal consistencyIntegrated & Verified
FETVFine grained content, attribute control, and prompt complexityIntegrated & Verified
GenAI-BenchHuman preferences for image/video generation and editingIntegrated & Verified
IPV-BenchGeneration and recognition of impossible eventsIntegrated & Verified
LikePhysLikelihood preference for valid versus impossible physicsIntegrated & Verified
MemoBenchVisible–disappeared–reappeared visual memoryIntegrated & Verified
MINDInteractive memory consistency and action controlIntegrated & Verified
MiraBenchLong video consistency, motion, quality, and caption alignmentIntegrated & Verified
PhyEduVideoPhysics education across six scientific domainsIntegrated & Verified
PhyFPS-Bench-GenPhysical FPS alignment and temporal stability (100 prompts)Integrated & Verified
PhyGenBenchPhysical commonsense and adherence to physical lawsIntegrated & Verified
PhyGroundCriteria grounded physics in text/image to video generationIntegrated & Verified
Physics-IQ OriginalFive second real physics continuation; original protocolIntegrated & Verified
Physics-IQ VerifiedPhysics continuation under the distinct verified protocolIntegrated & Verified
PhysVidBenchCaption based physical commonsense QA and tool useIntegrated & Verified
RBenchRobot image to video task and embodiment fidelityIntegrated & Verified
T2V-CompBenchSeven compositional axes; attribute/action binding and numeracyIntegrated & Verified
T2VSafetyBenchHarmful content and temporal safety risksIntegrated & Verified
VBench16 dimensions of quality, temporal coherence, and semanticsIntegrated & Verified
VBench-2.018 abilities across five intrinsic faithfulness pillarsIntegrated & Verified
VBench++Image to video, long video, and trustworthiness extensionsIntegrated & Verified
Video-BenchHuman preference aligned quality and condition alignmentIntegrated & Verified
VideoPhySemantic adherence and physical commonsenseIntegrated & Verified
VideoPhy2Action centric physical commonsense and rule judgementsIntegrated & Verified
VideoScience-BenchScientific phenomena, dynamics, and spatiotemporal coherenceIntegrated & Verified
VideoScoreLearned five aspect video quality/reward scoringIntegrated & Verified
VideoVerseEvent level temporal causality and world knowledgeIntegrated & Verified
Visual ChronometerPhysical FPS prediction on arbitrary videosIntegrated & Verified
VMBenchFive perception aligned video motion dimensionsIntegrated & Verified

Table 9. Supported video generation benchmarks (33 entries).

BenchmarkEvaluation focusStatus
4DWorldBenchPerceptual quality, condition alignment, physics, and 4D consistencyIntegrated & Verified
Apple-PILaw grounded perception, formulation, and deductionIntegrated & Verified
EWMIBenchEmbodied scene, motion, semantic consistency, and diversityIntegrated & Verified
HarnessEval-WTransitions, drift resistance, revisits, and offscreen evolutionIntegrated & Verified
iWorld-BenchInteractive generation, trajectory following, and memoryIntegrated & Verified
BenchmarkEvaluation focusStatus
PAWBenchStochastic dynamics calibration and outcome coverageIntegrated & Verified
Physical AI Bench (PAI-Bench)Generation and conditional generation tracks onlyIntegrated & Verified
SANA-WM 80-Scene Benchmark80 scenes per split; minute scale camera control and revisitsIntegrated & Verified
STEVO-BenchControl, physics, and unobserved state evolutionIntegrated & Verified
T2VWorldBenchSix domains of world knowledge (1,200 prompts)Integrated & Verified
WBenchMulti turn quality, setting, interaction, consistency, and physicsIntegrated & Verified
World-in-WorldClosed loop recognition, QA, navigation, and manipulation utilityIntegrated & Verified
WorldBenchDiagnostic physical video segmentation and text QAIntegrated & Verified
WorldModelBenchInstruction following, commonsense, and physics adherenceIntegrated & Verified
WorldOlympiadPhysical, interaction, and geometry triathlonIntegrated & Verified
WorldReasonBenchProcess aware reasoning and multidimensional qualityIntegrated & Verified
WorldScoreCamera conditioned controllability, quality, and dynamicsIntegrated & Verified
WRBenchCamera control, visual integrity, and persistent stateIntegrated & Verified

Table 10. Supported world model benchmarks (18 entries).

Supported Metrics

Table 11 lists all 62 canonical entries in the WorldFoundry metric registry. The table is compiled from the metric documentation together with the registry and its module declarations, and therefore includes entries absent from the documentation summary, such as laion_aesthetic. Metrics are grouped by use into multimodal/quality scorers (10), image/feature distribution metrics (26), video distribution metrics (3), perceptual/layout metrics (11), editing metrics (7), and existing-result utilities (5). Aliases and variants map to their canonical entries and are not counted separately.

Canonical metric IDAssessment and required inputsDir.
Multimodal scorers and quality (10)
clip_scoreCLIPScore: image/video–text embedding alignment↑\uparrow
vqa_scoreVQAScore: affirmative-answer probability for a visual question↑\uparrow
itm_scoreImage/video–text matching with BLIP-2 or related ITM backbones↑\uparrow
facescoreLearned facial realism and quality reward↑\uparrow
artscoreLearned artness score for generated images↑\uparrow
facesim_curSubject-to-video face identity similarity with CurricularFace↑\uparrow
gme_scoreGME-Qwen2-VL text–video relevance↑\uparrow
nexus_scoreSubject consistency with YOLO-World detection and GME↑\uparrow
natural_scoreAPI-based video naturalness rating↑\uparrow
laion_aestheticLinear aesthetic predictor on normalized CLIP image/frame features↑\uparrow
Image and feature distribution (26)
inception_scoreClassifier confidence and label diversity of generated images↑\uparrow
fidFréchet feature distance; Inception, CLIP, SwAV, frame, and object-crop/SceneFID variants↓\downarrow
kidKernel Inception Distance between image feature sets↓\downarrow
precision_recallFeature-manifold fidelity and coverage of generated/reference sets↑\uparrow
improved_precision_recallImproved precision/recall and per-image realism score↑\uparrow
fwdFréchet Wavelet Distance between image collections↓\downarrow
cmmdCLIP embedding maximum mean discrepancy↓\downarrow
pplPerceptual path length of interpolated generator latents↓\downarrow
mindMonge Inception Distance between image feature sets↓\downarrow
clean_fidFID with standardized image resizing and preprocessing↓\downarrow
vendi_scoreEffective diversity from a feature similarity matrix↑\uparrow
rarity_scoreUncommonness of generated samples relative to real features↑\uparrow
trendDivergence of fitted truncated generalized-normal feature densities↓\downarrow
fldFeature Likelihood Divergence using train, test, and generated features↓\downarrow
multimodal_midMutual Information Divergence for paired image–text features↑\uparrow
Canonical metric IDAssessment and required inputsDir.
fjdFréchet Joint Distance over conditional joint embeddings↓\downarrow
crosslidCross Local Intrinsic Dimensionality of feature collections↓\downarrow
cfidConditional Fréchet Inception Distance from paired embeddings↓\downarrow
ssdSemantic Similarity Distance for text–image consistency↓\downarrow
linear_separabilityLatent linear separability from a classifier confusion matrix↑\uparrow
rkeRényi kernel entropy mode count and relative entropy on features↑\uparrow
fddFréchet Denoised Distance using a pretrained denoising autoencoder↓\downarrow
cisConditional Inception Score from condition-bucketed class probabilities↑\uparrow
rndRandom network distillation diversity on features or images↑\uparrow
attribute_sadSemantic Attribute Distance from precomputed HCS features↓\downarrow
attribute_padPairwise Attribute Distance from precomputed HCS features↓\downarrow
Video distribution (3)
fvdFréchet distance between I3D features of reference/generated clips↓\downarrow
fvmdFréchet Video Motion Distance from tracked motion features↓\downarrow
jediVideo JEPA feature distance with a polynomial-kernel MMD↓\downarrow
Perceptual, reconstruction, and layout (11)
lpipsLearned perceptual distance between reference/generated images↓\downarrow
ssimStructural Similarity Index for paired images↑\uparrow
ms_ssimMulti-scale structural similarity for paired images↑\uparrow
psnrPeak signal-to-noise ratio from paired-image reconstruction error↑\uparrow
dino_similarityCosine similarity of DINO/DINOv2 reference/generated features↑\uparrow
dreamsimDreamSim perceptual distance between paired images↓\downarrow
cpbdNo-reference sharpness from cumulative blur-detection probability↑\uparrow
fsimFeature Similarity Index for paired images↑\uparrow
mask_accuracyPixel-wise mask accuracy and intersection-over-union (FCN)↑\uparrow
object_detectionPhrase-matching object-detection success rate↑\uparrow
lqsLayout Quality Score: recall, precision, location, and area consistency↑\uparrow
Editing and semantic consistency (7)
semsrSemantic Shift Rate for trigger/origin/target image triplets↑\uparrow
irsImage Realism Score calibrated using real-image statistics↑\uparrow
casClassification Accuracy Score: synthetic-data training, real-data testing↑\uparrow
manipulation_directionAlignment of image and text change vectors for an edit↑\uparrow
vs_similarityHDGAN visual–semantic similarity of paired embeddings↑\uparrow
quality_lossCLIP alignment multiplied by unintended text-presence probability↓\downarrow
object_wise_consistencyObject-wise IoU and detection success against guidance boxes↑\uparrow
Existing-result utilities (5)
artifact_countCounts artifacts recorded on a generation result—
required_artifacts_presentReturns one when all required artifacts are present↑\uparrow
has_artifactParameterized presence check: has_artifact:<name>↑\uparrow
numericEmits numeric scores and timings stored in result metadata—
numeric_valueParameterized numeric field: numeric:<name>—

Table 11. Complete supported metric catalog. Metrics are grouped by evaluation use; the underlying registry families comprise 30 distribution, 14 scorer, 13 perceptual, and 5 existing-result entries.

The registry specifies scoring implementations; benchmark scorecard dimensions and official evaluator outputs are defined by their respective benchmark protocols. ↑\uparrow and ↓\downarrow denote the preferred direction declared for each assessment; “–” denotes a result utility with no universal quality ordering. Numeric-field direction follows the underlying reported score. Metric IDs link to their sources or implementations. Learned metrics require their declared pretrained backbones, FVD uses I3D weights, JEDi uses compatible video-encoder/probe weights, and natural_score requires an API judge. Precomputed-feature and layout routines instead consume the specified embeddings, probabilities, masks, or boxes. Support for a registry entry does not by itself establish equivalence to every published evaluator configuration.

References

  1. [1]David Ha and Jürgen Schmidhuber. World Models. arXiv preprint arXiv:1803.10122, 2018. URL https://arxiv.org/abs/1803.10122. 3, 5, 7
  2. [2]Danijar Hafner, Timothy Lillicrap, Ian Fischer, et al. Learning Latent Dynamics for Planning from Pixels. In International Conference on Machine Learning (ICML), 2019. URL https://arxiv.org/abs/1811.04551. 5, 7
  3. [3]Danijar Hafner, Jurgis Pasukonis, Jimmy Ba, et al. Mastering diverse control tasks through world models. Nature, 640:647–653, 2025. doi: 10.1038/s41586-025-08744-2. URL https://www.nature.com/articles/s41586-025-08744-2. 3, 5, 7DOI
  4. [4]Mido Assran, Adrian Bardes, David Fan, et al. V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning. arXiv preprint arXiv:2506.09985, 2025. URL https://arxiv.org/abs/2506.09985. 3
  5. [5]DataFlow Team, Bohan Zeng, Daili Hua, et al. OpenWorldLib: A Unified Codebase and Definition of Advanced World Models. arXiv preprint arXiv:2604.04707, 2026. URL https://arxiv.org/abs/2604.04707. 3, 5, 10
  6. [6]Dian Zheng, Ziqi Huang, Hongbo Liu, et al. VBench-2.0: Advancing Video Generation Benchmark Suite for Intrinsic Faithfulness. arXiv preprint arXiv:2503.21755, 2025. URL https://arxiv.org/abs/2503.21755. 4, 5, 6, 9
  7. [7]Hritik Bansal, Clark Peng, Yonatan Bitton, et al. VideoPhy-2: A Challenging Action-Centric Physical Common-sense Evaluation in Video Generation. arXiv preprint arXiv:2503.06800, 2025. URL https://arxiv.org/abs/2503.06800.
  8. [8]Dacheng Li, Yunhao Fang, Yukang Chen, et al. WorldModelBench: Judging Video Generation Models As World Models. arXiv preprint arXiv:2502.20694, 2025. URL https://arxiv.org/abs/2502.20694. 6, 9
  9. [9]Yu Shang, Zhuohang Li, Yiding Ma, et al. WorldArena: A Unified Benchmark for Evaluating Perception and Functional Utility of Embodied World Models. arXiv preprint arXiv:2602.08971, 2026. URL https://arxiv.org/abs/2602.08971. 4, 5, 6, 9
  10. [10]OpenWorldLib Contributors. OpenWorldLib: Official Implementation and Documentation. Official source repository, 2026. URL https://github.com/OpenDCAI/OpenWorldLib. 5
  11. [11]Niket Agarwal, Arslan Ali, Maciej Bala, et al. Cosmos World Foundation Model Platform for Physical AI. arXiv preprint arXiv:2501.03575, 2025. URL https://arxiv.org/abs/2501.03575. 5, 10
  12. [12]NVIDIA. Cosmos: Official Implementation and Documentation. Official source repository, n.d.. URL https://github.com/NVIDIA/cosmos. 5
  13. [13]NVIDIA. Cosmos Evaluator: Automated Evaluation and Grading for Synthetic Video. Official source repository, n.d.. URL https://github.com/NVIDIA/cosmos-evaluator. 5
  14. [14]Yue Liao, Pengfei Zhou, Siyuan Huang, et al. Genie Envisioner: A Unified World Foundation Platform for Robotic Manipulation. arXiv preprint arXiv:2508.05635, 2025. URL https://arxiv.org/abs/2508.05635. 5
  15. [15]Boxiang Qiu, Liliang Chen, Yue Liao, et al. GE-Sim 2.0: A Roadmap Towards Comprehensive Closed-loop Video World Simulators for Robotic Manipulation. arXiv preprint arXiv:2605.27491, 2026. URL https://arxiv.org/abs/2605.27491. 5
  16. [16]Min Zhao, Hongzhou Zhu, Bokai Yan, et al. minWM: A Full-Stack Open-Source Framework for Real-Time Interactive Video World Models. arXiv preprint arXiv:2605.30263, 2026. URL https://arxiv.org/abs/2605.30263. 5
  17. [17]Patrick von Platen, Suraj Patil, Anton Lozhkov, et al. Diffusers: State-of-the-art diffusion models. Official source repository, 2022. URL https://github.com/huggingface/diffusers. 5
  18. [18]Remi Cadene, Simon Alibert, Francesco Capuano, et al. LeRobot: An Open-Source Library for End-to-End Robot Learning. In International Conference on Learning Representations (ICLR), 2026. URL https://arxiv.org/abs/2602.22818. 5
  19. [19]StarVLA Community. StarVLA: A Lego-like Codebase for Vision-Language-Action Model Developing. arXiv preprint arXiv:2604.05014, 2026. URL https://arxiv.org/abs/2604.05014. 5, 10, 21
  20. [20]Ziqi Huang, Yinan He, Jiashuo Yu, et al. VBench: Comprehensive Benchmark Suite for Video Generative Models. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024. URL https://arxiv.org/abs/2311.17982. 5, 6
  21. [21]Ziqi Huang, Fan Zhang, Xiaojie Xu, et al. VBench++: Comprehensive and Versatile Benchmark Suite for Video Generative Models. arXiv preprint arXiv:2411.13503, 2024. URL https://arxiv.org/abs/2411.13503. 5, 6
  22. [22]Haoyi Duan, Hong-Xing Yu, Sirui Chen, et al. WorldScore: A Unified Evaluation Benchmark for World Generation. In IEEE/CVF International Conference on Computer Vision (ICCV), 2025. URL https://arxiv.org/abs/2504.00983. 5, 6, 9
  23. [23]Yu Shang, Yinzhou Tang, Yiding Ma, et al. WorldArena 2.0: Extending Embodied World Model Benchmarking on Modality, Functionality and Platform. arXiv preprint arXiv:2605.17912, 2026. URL https://arxiv.org/abs/2605.17912. 5, 6, 9
  24. [24]Jiahan Zhang, Muqing Jiang, Nanru Dai, et al. World-in-World: World Models in a Closed-Loop World. arXiv preprint arXiv:2510.18135, 2025. URL https://arxiv.org/abs/2510.18135. 5, 6
  25. [25]Matthew Tancik, Ethan Weber, Evonne Ng, et al. Nerfstudio: A Modular Framework for Neural Radiance Field Development. In ACM SIGGRAPH 2023 Conference Proceedings, 2023. URL https://arxiv.org/abs/2302.04264. 5
  26. [26]Mayank Mittal, Pascal Roth, James Tigue, et al. Isaac Lab: A GPU-Accelerated Simulation Framework for Multi-Modal Robot Learning. arXiv preprint arXiv:2511.04831, 2025. URL https://arxiv.org/abs/2511.04831. 5
  27. [27]Hritik Bansal, Zongyu Lin, Tianyi Xie, et al. VideoPhy: Evaluating Physical Commonsense for Video Generation. arXiv preprint arXiv:2406.03520, 2024. URL https://arxiv.org/abs/2406.03520. 6
  28. [28]Jianjie Fang, Yingshan Lei, Qin Wan, et al. iWorld-Bench: A Benchmark for Interactive World Models with a Unified Action Generation Framework. arXiv preprint arXiv:2605.03941, 2026. URL https://arxiv.org/abs/2605.03941. 6, 9
  29. [29]Xinhao Xiang, Xiao Liu, Zizhong Li, Zhuosheng Liu, and Jiawei Zhang. AIGVE-Tool: AI-Generated Video Evaluation Toolkit with Multifaceted Benchmark. arXiv preprint arXiv:2503.14064, 2025. URL https://arxiv.org/abs/2503.14064. 6
  30. [30]Weiliang Chen, Haowen Sun, Jun Gao, et al. HarnessEval-W: Agentifying the Evaluation of Visual Worlds. arXiv preprint arXiv:2608.16859, 2026. URL https://arxiv.org/abs/2608.16859. 6
  31. [31]Zhuoyi Yang, Jiayan Teng, Wendi Zheng, et al. CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer. arXiv preprint arXiv:2408.06072, 2024. URL https://arxiv.org/abs/2408.06072. 17
  32. [32]Weichen Fan, Chenyang Si, Junhao Song, et al. Vchitect-2.0: Parallel Transformer for Scaling Up Video Diffusion Models. arXiv preprint arXiv:2501.08453, 2025. URL https://arxiv.org/abs/2501.08453. 17
  33. [33]Bing Wu, Chang Zou, Changlin Li, et al. HunyuanVideo 1.5 Technical Report. arXiv preprint arXiv:2511.18870, 2025. URL https://arxiv.org/abs/2511.18870. 17
  34. [34]Lighttricks. LTX-2.5: Official Distilled Model Release. Official model release or documentation, n.d. URL https://github.com/Lighttricks/LTX-2. Accessed October 13, 2026. 17
  35. [35]Team Wan, Ang Wang, Baole Ai, et al. Wan: Open and Advanced Large-Scale Video Generative Models. arXiv preprint arXiv:2503.20314, 2025. URL https://arxiv.org/abs/2503.20314. 17
  36. [36]Zeyinzi Jiang, Zhen Han, Chaojie Mao, et al. VACE: All-in-One Video Creation and Editing. arXiv preprint arXiv:2503.07598, 2025. URL https://arxiv.org/abs/2503.07598. 17
  37. [37]Wan Team. Wan2.2: Official Model Release. Official model repository, 2025. URL https://github.com/Wan-Video/Wan2.2. Accessed September 9, 2026. 17
  38. [38]Shuailei Ma, Jiaqi Liao, Xinyang Wang, et al. Scaling Mixture-of-Experts Video Pretraining for Embodied Intelligence. arXiv preprint arXiv:2607.07675, 2026. URL https://arxiv.org/abs/2607.07675. 17
  39. [39]MiniMax. MiniMax H3. Official model release, 2026. URL https://www.minimax.cn/blog/minimax-h3. Accessed September 9, 2026. 17
  40. [40]Yansong Tang, Aoyang Liu, Jinpeng Liu, Shiyi Zhang, Wenxun Dai, Jie Zhou, Xiu Li, and Jiwen Lu. Flag3d++: A benchmark for 3d fitness activity comprehension with language instruction. IEEE Transactions on Pattern Analysis and Machine Intelligence, 47(11):9731–9748, 2025. 17
  41. [41]Runjia Qian, Zile Wang, Jihai Zhang, et al. Matrix-Game 3.5: Enhancing Real-Time Streaming Interactive World Models with Patch Memory. arXiv preprint arXiv:2608.29910, 2026. URL https://arxiv.org/abs/2608.29910. 18, 19
  42. [42]Robbyant Team, Zelin Gao, Qiuyu Wang, et al. Advancing Open-source World Models. arXiv preprint arXiv:2601.20540, 2026. URL https://arxiv.org/abs/2601.20540. 18
  43. [43]Zelin Gao, Qiuyu Wang, Jiapeng Zhu, et al. Infinite Worlds with Versatile Interactions. arXiv preprint arXiv:2607.07534, 2026. URL https://arxiv.org/abs/2607.07534. 18, 19
  44. [44]Jiaqi Li, Junshu Tang, Zhiyong Xu, et al. Hunyuan-GameCraft: High-dynamic Interactive Game Video Generation with Hybrid History Condition. arXiv preprint arXiv:2506.17201, 2025. URL https://arxiv.org/abs/2506.17201. 18
  45. [45]Junshu Tang, Jiacheng Liu, Jiaqi Li, et al. Hunyuan-GameCraft-2: Instruction-following Interactive Game World Model. arXiv preprint arXiv:2511.23429, 2025. URL https://arxiv.org/abs/2511.23429. 18, 19
  46. [46]Xiaofeng Mao, Zhen Li, Chuanhao Li, et al. Yume-1.5: A Text-Controlled Interactive World Generation Model. arXiv preprint arXiv:2512.22096, 2025. URL https://arxiv.org/abs/2512.22096. 18
  47. [47]Wenqiang Sun, Haiyu Zhang, Haoyuan Wang, et al. WorldPlay: Towards Long-Term Geometric Consistency for Real-Time Interactive World Modeling. arXiv preprint arXiv:2512.14614, 2025. URL https://arxiv.org/abs/2512.14614. 18
  48. [48]Zizhao Tong, Yeying Jin, Hongfeng Lai, et al. SCOPE: Simulating Cross-game Operations in Playable Environments for FPS World Models. arXiv preprint arXiv:2605.23345, 2026. URL https://arxiv.org/abs/2605.23345. 18, 19
  49. [49]Haoyi Zhu, Haozhe Liu, Yuyang Zhao, et al. SANA-WM: Efficient Minute-Scale World Modeling with Hybrid Linear Diffusion Transformer. arXiv preprint arXiv:2605.15178, 2026. URL https://arxiv.org/abs/2605.15178. 18
  50. [50]Songchun Zhang, Yaowei Li, Junhao Zhuang, et al. EchoWM: Open and Enterable Omnimodal World Models. arXiv preprint arXiv:2608.23189, 2026. URL https://arxiv.org/abs/2608.23189. 18
  51. [51]Xianglong He, Chunli Peng, Zexiang Liu, et al. Matrix-game 2.0: An open-source real-time and streaming interactive world model. arXiv preprint arXiv:2508.13009, 2025. URL https://arxiv.org/abs/2508.13009. 19
  52. [52]Zile Wang, Zexiang Liu, Jiaxing Li, et al. MatrixGame 3.0: Real-Time and Streaming Interactive World Model with Long-Horizon Memory. arXiv preprint arXiv:2604.08995, 2026. URL https://arxiv.org/abs/2604.08995. 19
  53. [53]Zhifei Chen, Luozhou Wang, Guibao Shen, et al. ReWorld: An Interactive World Model with Long-Horizon Memory. arXiv preprint arXiv:2608.23565, 2026. URL https://arxiv.org/abs/2608.23565. 19
  54. [54]Shuzhe Wang, Vincent Leroy, Yohann Cabon, et al. DUSt3R: Geometric 3D Vision Made Easy. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024. URL https://arxiv.org/abs/2312.14132. 19
  55. [55]Qianqian Wang, Yifei Zhang, Aleksander Holynski, et al. Continuous 3D Perception Model with Persistent State. arXiv preprint arXiv:2501.12387, 2025. URL https://arxiv.org/abs/2501.12387. 20
  56. [56]Jianyuan Wang, Minghao Chen, Nikita Karaev, et al. VGGT: Visual Geometry Grounded Transformer. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025. URL https://arxiv.org/abs/2503.11651. 20
  57. [57]Jianyuan Wang, Minghao Chen, Shangzhan Zhang, et al. VGGT-Ω. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2026. URL https://arxiv.org/abs/2605.15195. 20
  58. [58]Brandon Smart, Chuanxia Zheng, Iro Laina, et al. Splat3R: Zero-shot Gaussian Splatting from Uncalibrated Image Pairs. arXiv preprint arXiv:2408.13912, 2024. URL https://arxiv.org/abs/2408.13912. 20
  59. [59]Xinyang Li, Tengfei Wang, Zixiao Gu, et al. FlashWorld: High-quality 3D Scene Generation within Seconds. arXiv preprint arXiv:2510.13678, 2025. URL https://arxiv.org/abs/2510.13678. 20
  60. [60]Junyi Zhang, Charles Herrmann, Junhwa Hur, et al. MonST3R: A Simple Approach for Estimating Geometry in the Presence of Motion. arXiv preprint arXiv:2410.03825, 2024. URL https://arxiv.org/abs/2410.03825. 20
  61. [61]Yuxue Yang, Lue Fan, Ziqi Shi, et al. NeoVerse: Enhancing 4D World Model with in-the-wild Monocular Videos. arXiv preprint arXiv:2601.00393, 2026. URL https://arxiv.org/abs/2601.00393. 20
  62. [62]Sherwin Bahmani, Tianchang Shen, Jiawei Ren, et al. Lyra: Generative 3D Scene Reconstruction via Video Diffusion Model Self-Distillation. In International Conference on Learning Representations (ICLR), 2026. URL https://research.nvidia.com/labs/toronto-ai/lyra/. 20arxiv.org/abs/2509.19296
  63. [63]Tianchang Shen, Sherwin Bahmani, Kai He, et al. Lyra 2.0: Explorable Generative 3D Worlds. arXiv preprint arXiv:2604.13036, 2026. URL https://arxiv.org/abs/2604.13036. 20
  64. [64]Tony Z. Zhao, Vikash Kumar, Sergey Levine, et al. Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware. arXiv preprint arXiv:2304.13705, 2023. URL https://arxiv.org/abs/2304.13705. 21
  65. [65]Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, et al. OpenVLA: An Open-Source Vision-Language-Action Model. arXiv preprint arXiv:2406.09246, 2024. URL https://arxiv.org/abs/2406.09246. 21
  66. [66]Xinghang Li, Minghuan Liu, Hanbo Zhang, et al. Vision-Language Foundation Models as Effective Robot Imitators. arXiv preprint arXiv:2311.01378, 2023. URL https://arxiv.org/abs/2311.01378. 21
  67. [67]Haoquan Fang, Jiafei Duan, Donovan Clay, et al. MolmoAct2: Action Reasoning Models for Real-world Deployment. arXiv preprint arXiv:2605.02881, 2026. URL https://arxiv.org/abs/2605.02881. 21
  68. [68]GigaBrain Team, Angen Ye, Boyuan Wang, et al. GigaBrain-0: A World Model-Powered Vision-Language-Action Model. arXiv preprint arXiv:2510.19430, 2025. URL https://arxiv.org/abs/2510.19430. 21
  69. [69]Hao Luo, Ye Wang, Wanpeng Zhang, et al. Being-H0.5: Scaling Human-Centric Robot Learning for Cross-Embodiment Generalization. arXiv preprint arXiv:2601.12993, 2026. URL https://arxiv.org/abs/2601.12993. 21
  70. [70]Wei Wu, Fan Lu, Yunnan Wang, et al. A Pragmatic VLA Foundation Model. arXiv preprint arXiv:2601.18692, 2026. URL https://arxiv.org/abs/2601.18692. 21
  71. [71]Wei Wu, Fangjing Wang, Fan Lu, et al. From Foundation to Application: Improving VLA Models in Practice. arXiv preprint arXiv:2607.06403, 2026. URL https://arxiv.org/abs/2607.06403. 21
  72. [72]Lin Li, Qihang Zhang, Yiming Luo, et al. Causal World Modeling for Robot Control. arXiv preprint arXiv:2601.21998, 2026. URL https://arxiv.org/abs/2601.21998. 21
  73. [73]Qihang Zhang, Lin Li, Luyao Zhang, et al. Native Video-Action Pretraining for Generalizable Robot Control. arXiv preprint arXiv:2607.08639, 2026. URL https://arxiv.org/abs/2607.08639. 21
  74. [74]Bo Liu, Yifeng Zhu, Chongkai Gao, et al. LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning. arXiv preprint arXiv:2306.03310, 2023. URL https://arxiv.org/abs/2306.03310. 21
  75. [75]Tianxing Chen, Zanxin Chen, Baijun Chen, et al. RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulation. arXiv preprint arXiv:2506.18088, 2025. URL https://arxiv.org/abs/2506.18088. 21
  76. [76]Qiyaun Liao, Yiyang Shao, Junfeng Long, Kevin Zakka, Roberto Horowtiz, Pieter Abbeel, and Koushil Sreenath. Supermimic: Scaling humanoid whole-body control with one differentiable world model. arXiv preprint arXiv:2610.28480, 2026. 21arxiv.org/abs/2610.28480
  77. [77]Tri Dao. FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning. In International Conference on Learning Representations (ICLR), 2024. URL https://arxiv.org/abs/2307.08691. 23
  78. [78]Jay Shah, Ganesh Bikshandi, Ying Zhang, et al. FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precision. arXiv preprint arXiv:2407.08608, 2024. URL https://arxiv.org/abs/2407.08608. 23
  79. [79]Jintao Zhang, Jia Wei, Haofeng Huang, et al. SageAttention: Accurate 8-Bit Attention for Plug-and-play Inference Acceleration. In International Conference on Learning Representations (ICLR), 2025. URL https://arxiv.org/abs/2410.02367. 23

Paper details

vlnc:W_kt4f6bju
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 4

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents