betaFeedback

WorksComputer Science

PhysResponse: Probing the Physics of Video Generation with Controlled Interventions

Abstract

Video generation models are increasingly viewed as world simulators, which requires not only plausible motion but dynamics that respond correctly to specified physical parameters. Existing models, datasets and benchmarks, however, largely target physical plausibility, leaving this physical parameter responsiveness insufficiently explored. We introduce PhysResponse, a benchmark that measures it at two levels: scene-conditioned simulator inversion recovers the effective gravity, friction and initial speed of generated motion, and event evaluation tests outcomes near and across critical transitions. Evaluating ten image-to-video models, we find that even the lowest parameter errors exceed those of physically correct videos by over an order of magnitude, that models rarely reproduce both outcomes of a transition pair, and that the two levels expose complementary failures that visual metrics such as PSNR miss. To supply the missing training signal, we construct 72,000 simulated videos of 20 multi-object interaction types with controlled parameters; fine-tuning on them lowers parameter errors, even on held-out interaction types, and transfers to real videos and an external physics benchmark. Continued training with a numerical encoder further lowers all three errors on held-out types. PhysResponse provides a foundation for world simulators that follow physical conditions, not just appearances.

One video and prompt excerpt per interaction family

Figure 1. We present PhysResponse, 72,000 simulated videos of 20 multi-object interactions with set physical conditions, and a benchmark that recovers gravity, friction and speed from generated videos. One video and prompt excerpt per interaction family.

[email protected]

Introduction

Rapid advances in video generation [29, 38, 39, 55] have enabled unprecedented visual fidelity and realism, motivating a growing body of research to develop video generation models into world simulators [32, 34, 53, 61] capable of understanding and predicting real-world physical dynamics. Alongside these advances, physics-focused datasets [25, 31] and benchmarks have emerged to improve and evaluate models’ ability to generate physically plausible motion and interactions. Together, these efforts have established physical plausibility as an important criterion for assessing progress toward world simulation.

However, physical plausibility alone is insufficient for a world simulator. Beyond producing realistic motion, a simulator must capture how dynamics change in response to specified physical parameters. In embodied AI, for example, it should predict how variations in gripper friction affect object manipulation. Such capabilities are essential for simulating interactions across scenarios and physical conditions. Yet existing models [11, 53], datasets [25, 64], and benchmarks [17, 30, 42] largely focus on a common question: Do the generated outcomes appear consistent with physical common sense? This leaves a more precise question insufficiently explored: Do the generated dynamics respond correctly to the specified physical parameters? We refer to this capability as physical parameter responsiveness. Addressing it requires evaluating not only whether a video depicts plausible dynamics, but also whether those dynamics are consistent with the particular parameter values provided.

To address this gap, we first introduce a benchmark and evaluation protocol for measuring physical parameter responsiveness at both the event and parameter levels. Using a physics simulator, we render multi-object interactions across 20 scenarios under varying combinations of gravitational acceleration, friction, and initial speed. Our event-level metrics assess whether generated videos exhibit the expected events under those conditions, while our parameter-level metrics measure how closely physical parameters recovered from the generated dynamics match their specified values. Together, these metrics evaluate both event outcomes and quantitative consistency with the conditioning parameters. Our evaluation shows that current models struggle to respond accurately to specified physical parameters (Figure 2).

Figure 2. Speed decides the outcome in simulation but often not in generation. In simulation only the faster book tips the can onto the shelf, but PhysAlign, even told so, leaves it on the table at both speeds. ✓/✗: match/contradict the simulation.

One factor that may contribute to this limitation is the lack of training data that systematically captures how physical parameters shape observable dynamics. We therefore develop a data generation pipeline with control over physical parameters and construct a dataset of 72,000 videos spanning 20 scenarios. The dataset depicts object motion and interactions under diverse parameter combinations, with particular emphasis on capturing variations that produce distinct motion patterns or event outcomes. This diversity provides training examples that connect parameter values to their observable physical effects. Experiments show that fine-tuning on our dataset improves physical parameter responsiveness, supporting the value of targeted data for learning dynamics.

Beyond data, we investigate how model design can further improve physical parameter responsiveness. Specifically, we augment the baseline model with a numerical encoder and fine-tune the resulting model on our dataset. Continued training with the numerical encoder further improves parameter responsiveness, lowering all three errors on held-out interaction types, suggesting that strengthening numerical conditioning is a promising direction for enabling world simulators to accurately reproduce dynamics under specified physical conditions. Taken together, our benchmark, dataset, and model exploration provide a foundation for studying world simulation beyond general physical plausibility. Our contributions are threefold:

  • A benchmark for physical parameter responsiveness. We introduce a benchmark spanning 20 scenarios, together with event-level and parameter-level metrics that assess whether generated dynamics correctly reflect specified physical parameters. Our evaluation identifies substantial limitations in current state-of-the-art models.

  • A dataset with controlled physical parameters. We develop a configurable simulation pipeline and construct a dataset of 72,000 videos covering diverse parameter combinations and their resulting dynamics. We show that fine-tuning on this dataset improves physical parameter responsiveness.

  • An investigation of numerical conditioning. We augment a baseline model with a numerical encoder and show that continued training with it further improves parameter responsiveness on held-out interaction types, highlighting numerical conditioning as a promising direction for parameter-controlled world simulation.

Related Work

Physical conditioning for video generation. Video generators use simulated dynamics and explicit controls to specify physical behavior [12, 14, 26, 46, 58], while physics-oriented post-training improves motion realism [19, 20, 61]. PhyCo [31] introduces controllable physical priors; PhyParam [25] pairs simulated rigid-body videos with object-level parameters and evaluates physical-law consistency. PhysResponse focuses on multi-object interactions, linking controlled physical inputs to both the motion they produce and the events they trigger.

Physical evaluation of video models. Benchmarks assess physical plausibility through human or vision-language judgments [1, 2, 15, 16, 21, 27, 56, 63], or measure motion against real and simulated references [17, 30, 42, 49]. PhysInOne [64] measures spectral motion agreement, WorldBench [45] examines numerical physical properties, and Principia [40] tests inter-object relations. Fall-time analyses [41] and PhysWeep [18] recover physical quantities from generated motion, with PhysWeep studying sweeps across five single-object systems. Counterfactual benchmarks test responses to changed conditions in reasoning [3, 4, 10, 43, 57] and generation [5, 8, 22]: PhysicSENS [28] varies stated physical properties, and Ge-oFlip [24] tests outcome flips under geometric interventions. PhysResponse brings parameter recovery and transition-based event evaluation together in multi-object scenes, testing whether generated motion matches the specified conditions and produces the corresponding interactions.

Analysis by synthesis. Physical inference matches simulations to observed motion [51, 59], learns representations from video [52], or uses differentiable simulation [23]; related work probes dynamic properties in video foundation models [60]. We use simulation matching to recover effective gravity, friction and initial speed, conditioning on known geometry, camera, initial state and non-target settings. This provides a quantitative account of the dynamics behind generated interactions.

The PhysResponse Dataset

Interaction Coverage and Physical Conditions

PhysResponse links editable physical conditions to multi-object motion and interaction outcomes (Figure 4; Table 1). Its 20 everyday interaction types (scenarios T01–T20) span four mechanisms: inclined-surface motion, collision-mediated transfer, changes of support, and multistage cascades. Gravity affects acceleration and fall timing; friction affects sliding and motion transfer through contact; initial speed determines whether and when objects reach one another.

Twenty everyday interaction types

Figure 4. Twenty everyday interaction types. Arrows: motion; letters: parameters evaluated; * held out of training.

WorkVideosSetparam.MultiAnyMulti Ctrl.FlipsTrain
Physics-IQ [30]396R✗✓✗✗✗✗
PhyGenBench [27]160P✗(✓)✗✗✗✗
VideoPhy-2 [2]3.9KP✗✓✗✗✗✗
WorldBench [45]748R+Sμ\mu, η\eta†✓✓✗✗✗
PhysInOne [64]2MSμ\mu, ee, ρ\rho, . . .✓✗✗✗✓
PhyParam [25]130KSgg, μ\mu, mm, FF, ee✓✗✗(✓)✓
What-If World [8]638P✗✓✗✗✓✗
RigidBench [17]5.1KS✗✓✗✗✗✓
PhysWeep [18]proc.Sgg, μ\mu, ee, kk, . . .✗✓✗✗✗
GeoFlip [24]964S✗✓✗✗✓(✓)§
PhysResponse72KSgg, μ\mu, vv✓✓✓✓✓

Table 1. Datasets and evaluation capabilities. Recovery: parameters recovered from generated videos; Ctrl.: one parameter varied; flips: opposite reference outcomes. R/S: real/simulated; P: prompts; proc.: on demand; (✓): partial; †by material; §geometry only. ee, ρ\rho, η\eta, kk, mm, FF: restitution, density, viscosity, stiffness, mass, force.

The dataset contains 72,000 videos from 24,000 simulation runs, each rendered from three synchronized viewpoints. Each type contributes 3,600 videos: 2,640 from Genesis [13] and 960 from Isaac Sim [33]. All views of a run share the same data split.

Construction

Scenes specify object configurations, initial states, contact settings, and prescribed controls (Figure 3). We edit gravity across five levels from 1.62 to 24 m/s2^2, with scene-specific friction and initial-speed settings. Four backgrounds and three viewpoints provide appearance variation. Available simulation states and contact records support parameter measurements and reference event outcomes.

Figure 3. Constructing PhysResponse. Scenes are assembled from household assets and four backgrounds, edited one physical condition at a time, simulated in Genesis and Isaac and rendered from three views; panel 2 brackets two reference transitions, in the initial speed of T17 and the channel width of T05. Depth and masks are derived from RGB.

Controlled interventions vary one target condition while fixing non-target conditions. Dependent quantities change accordingly, such as angular speed when editing a rolling object’s linear speed. Reference simulations locate and refine parameter intervals across which event outcomes flip.

Configurations resampled across multiple inputs populate the training pool, providing examples of how physical conditions shape motion and interactions.

Core100 selects five cases per interaction type for evaluation. Figure 3 illustrates training prompts; evaluation uses four English formats: Short states the interaction and physical values; Standard embeds these values in a natural description; Detailed adds the initial spatial arrangement; and Concat appends values to a fixed template. Some prompts also state expected event outcomes. These resources support both learning physical responses and evaluating their motion-level parameters and event-level consequences.

Parameter-Level Evaluation

We recover effective parameters by replaying the source scene, varying θ∈{g,μ,v}\theta\in\{g,\mu,v\} while fixing context cc: geometry, camera, non-target initial and physical conditions, and prescribed controls. Matching replay S(c,θ)\mathcal{S}(c,\theta) to generated motion gives

θ^(c)=arg⁡min⁡θ∈Θc min⁡Δ∈D L(θ,Δ).(1)\hat{\theta}(c)=\underset{\theta\in\Theta_c}{\arg\min}\ \underset{\Delta\in\mathcal{D}}{\min}\ L(\theta,\Delta). \tag*{(1)}

Here Δ\Delta is a bounded onset offset; θ^(c)\hat{\theta}(c) explains the motion under the fixed context (Figure 6).

Figure 6. Recovering gravity from a generated video. (a) MiniMax-H3, Moon gravity: tracks (white) match the replay at g^=11.8 m/s2\hat{g}=11.8\ \mathrm{m/s^2} (gold), not at the prompted gg (grey). (b) Excess fit loss, square-root axis; shaded: tolerance δ\delta; thin grey: a lower bound and an unidentifiable estimate.

Conditional identifiability. Conditioning is essential because different parameters can explain the same motion. In horizontal sliding,

a=−μg,(2)a=-\mu g, \tag*{(2)}

so deceleration identifies μg\mu g, not its factors. Fixing friction permits a conditional gravity estimate; a sharp minimum alone cannot establish joint identifiability. Gravity requires gravity-driven motion, friction visible sliding contact, and speed an observable onset with known direction. The loss curve distinguishes localized estimates, one-sided bounds and unidentifiable cases.

Observation and simulation matching

SAM 3.1 [9] tracks condition-frame objects. We align backgrounds to compensate camera motion, map frames to physical time, and register projected replay boxes after removing condition-frame offsets. Candidate grids exclude the requested value.

For an observed box b^o\hat{b}_o and a projected box bob_o, the four-edge matching loss is

ℓo(t;θ,Δ)=∑j=14ρ(b^o,j(t)−bo,j(t+Δ;c,θ)do)/(2(σ/do)2).(3)\ell_o(t;\theta,\Delta)=\sum_{j=1}^{4}\rho\left(\frac{\hat{b}_{o,j}(t)-b_{o,j}(t+\Delta;c,\theta)}{d_o}\right)\bigg/\left(2(\sigma/d_o)^2\right). \tag*{(3)}

Here dod_o is the initial box diagonal, ρ\rho the Huber loss and σ\sigma the tracking-noise scale. LL sums visibility-penalized temporal averages over objects with a shared physical explanation, within windows supported by replay repeatability and reference agreement. A common onset offset aligns all objects.

A quadratic fit in log-parameter refines the grid minimum. Reference-calibrated tolerance δ\delta defines the indistinguishable range, yielding a point estimate, bound or unidentifiable result according to the loss basin.

Aggregation and kinematics

Aggregation. For each model and parameter, we rank results within scenes containing at least three results. Stronger parameter constraints rank first; if residual breaks ties. Parameter error does not affect ranking. We retain the top 10% (at least one result), pool them with equal total weight per scene, and report the weighted median of eiθ=∣θ^i/θi−1∣e_i^\theta= \lvert\hat{\theta}_i/\theta_i - 1 \rvert for θi≠0\theta_i \ne0. One-sided bounds enter at their grid edge, so their errors are lower bounds. Unmeasurable retained slots count as 100% error, preventing score gains from excluding difficult samples.

Kinematics. Displacement is the generated displacement projected onto the reference direction, divided by reference length: 1 indicates correct motion and 0 no motion. We average over eligible objects. Causal order is the fraction of reference object pairs whose onset order is preserved; nonmoving pairs fail this test.

Event-Level Evaluation

Parameter estimates describe generated dynamics; event evaluation tests their interaction consequences. Reference simulations define outcome transitions against which we evaluate individual conditions and paired interventions (Figure 5).

Figure 5. PhysResponse evaluation overview. Core100 conditions and prompts produce videos for two complementary measurements. Object tracks support simulator-based parameter recovery; reference transitions and visual event observations support NearAcc, FarAcc and PairAcc. Measurability screening identifies the evidence available to each branch, with human validation of the observations.

Object-Centric Event Observation

An event ee comprises participating objects, an observable binary predicate fef_e and a window WW, with reference outcome ye(c,θ;W)∈{0,1}y_e(c,\theta;W) \in\{0,1\}. Generated outcomes are evaluated against this reference. SAM 3.1 [9] supplies object tracks and local crops; Qwen3.8-27B [36] examines these alongside full-scene frames.

Reference-Defined Critical Transitions

Small changes in physical conditions can initiate contact or redirect a cascade; only the faster book in Figure 2 tips the can onto the shelf. Testing both conditions probes this response.

We fix cc, ee and WW and vary θ\theta along a predefined intervention axis in reference simulation. Opposite reference outcomes at two verified settings define a local transition interval:

[l,u]:ye(c,l;W)≠ye(c,u;W),l<u.(4)[l,u]:\quad y_e(c,l;W) \ne y_e(c,u;W),\qquad l<u. \tag*{(4)}

The width u−lu-l gives localization resolution. Intervention axes include physical parameters, geometry and control conditions (Supp. J).

Quantifying Event Responsiveness

Single-condition accuracy. For setting θ\theta and the intervention’s normalization span SS, distance to the reference interval is

dmin⁡(θ)=max⁡(l−θ,θ−u,0)S.(5)d_{\min}(\theta)=\frac{\max(l-\theta,\theta-u,0)}{S}. \tag*{(5)}

Near uses dmin⁡≤0.20d_{\min}\leq0.20, including ties and distance intervals crossing 20%; Far uses dmin⁡>0.20d_{\min}>0.20. For each model and prompt, let aia_i average decidable correctness at reference event–window point ii, NG,yN_{G,y} contain such points with outcome yy in group GG, and YGY_G contain their represented outcomes. For G∈{Near,Far}G\in\{\mathrm{Near},\mathrm{Far}\},

AccG=1∣YG∣∑y∈YG1∣NG,y∣∑i∈NG,yai.(6)\mathrm{Acc}_G=\frac{1}{|Y_G|}\sum_{y\in Y_G}\frac{1}{|N_{G,y}|}\sum_{i\in N_{G,y}}a_i. \tag*{(6)}

The tables report this outcome-balanced score when both reference outcomes have decidable observations.

Accuracy on outcome flips. PairAcc requires both generated endpoints to match their references. We use 51 video pairs across 18 scenes and 245 event–window registrations, matching scene, event and window; generated endpoints share model, prompt and seed. Let dpd_p and cpc_p be the fractions of pair pp’s targets jointly decidable and jointly correct. Each video pair has unit weight.

Dw=∑pdp,Cw=∑pcp,D_w=\sum_p d_p,\qquad C_w=\sum_p c_p,
PairAcc=CwDw,Coverage=Dw51,VPSR=Cw51.(7)\mathrm{PairAcc}=\frac{C_w}{D_w},\qquad\mathrm{Coverage}=\frac{D_w}{51},\qquad\mathrm{VPSR}=\frac{C_w}{51}. \tag*{(7)}

PairAcc measures accuracy among decidable targets; Coverage and VPSR measure decidability and verified success over all 51 pairs.

Experiments

Setup and metric validation

We evaluate ten models [11, 29, 32, 34, 38, 39, 53, 55, 61] on Core100 using four prompt formats and seed 0: 4,000 generated videos. Videos retain native timing. Parameter and motion scores pool formats; event tables report Standard and Detailed.

Measures. Alongside parameter and event metrics (§ 4.2–§ 5.3), we report video-averaged PSNR, SSIM [50], displacement L2 error relative to image height, and moving-region S-IoU [30]. Meas. is the proportion of videos judged measurable by blind human annotators.

Metric validation. Processing GT videos with the same pipeline yields gravity, friction and speed errors of 3.6%, 1.4% and 1.1%; displacement is 0.91 and causal-order accuracy 99%. These establish a reference for interpreting generated-video errors. Fitting baselines and perturbation controls appear in Supp. Tables 31–33.

Against blind human annotations of all 4,000 videos, automatic measurability agreement is 74%. Of 6,070 binary human-labeled events in human-measurable videos, 68% receive automatic binary judgments, with 79% agreement (Supp. Table 26).

Results Analysis

Parameter-level evaluation

Even the lowest parameter errors exceed GT errors by over an order of magnitude (Table 2), consistently across retained shares (Supp. Fig. 13). Recovered values also barely follow the prompts: within scenes, the rank correlation between prompted and recovered gravity, net of a grid-only null, is −0.13 to +0.16 across the ten models, against +0.77 for GT (Supp. G). MiniMax-H3 leads on gravity and friction, yet under a Moon-gravity prompt its motion matches g^=11.8 m/s2\hat{g}=11.8\,\mathrm{m/s^2} (Figure 6). PhysRVG and VideoGPA post-train Wan2.2-TI2V-5B and PhysAlign post-trains Wan2.2-I2V-A14B, yet their gains over these bases are uneven: Phys-RVG lowers friction error but raises gravity and speed errors, VideoGPA lowers gravity and friction errors but raises speed error, and PhysAlign mainly lowers speed error.

Table 2. No model follows the prompted physics. All four prompt formats pooled (per format: supplement); errors aggregated as in § 4.2. Condition frame: first frame repeated, not ranked. Bold/underlined: best/second.

ModelVisual and motion consistencyKinematicsEffective parameter errorHuman
L2 err. ↓S-IoU ↑PSNR ↑SSIM ↑Displ. →1Order ↑gg ↓μ\mu ↓vv ↓Meas. ↑
Condition frame repeated12.59%0.00037.80.9770.000%––––
CogVideoX-1.513.59%0.05722.90.8440.038%114.5%166.0%93.0%23%
Cosmos-Predict213.60%0.07120.80.8000.068%89.2%76.8%81.9%40%
Cosmos3-Nano17.40%0.15329.70.9350.6623%81.8%77.0%53.5%45%
HunyuanVideo-1.514.90%0.20027.40.9270.1026%86.5%77.0%63.9%58%
MiniMax-H313.93%0.29027.70.8530.8054%73.5%68.7%76.8%96%
PhysAlign19.29%0.217<u>30.1</u>0.9400.2927%93.5%<u>74.4%</u><u>63.1%</u><u>87%</u>
PhysRVG12.94%0.04930.30.9260.011%140.3%77.7%93.4%83%
VideoGPA16.10%0.06422.00.8060.119%86.5%101.1%91.1%29%
Wan2.2-I2V-A14B17.16%0.09921.00.7020.63<u>28%</u>88.9%74.9%78.4%72%
Wan2.2-TI2V-5B16.10%0.05123.20.8190.0613%94.9%130.3%85.8%35%

Table 2.

Visual similarity can reward weak motion: repeating the condition frame beats every model in PSNR, SSIM and L2 error. Among models, PhysRVG leads in PSNR and L2 error yet barely moves, and has the largest gravity error. Causal order is also poorly preserved: nine models score 1–28% and MiniMax-H3 54%, versus 99% for GT.

Event-level evaluation

Near/Far accuracy varies with model and prompt (Table 3). The 10%, 15% and 20% cutoff analysis reveals different distance responses: tightening the cutoff lowers NearAcc for Wan2.2-TI2V-5B Detailed but raises it for Cosmos3-Nano Detailed (Supp. Table 14).

Table 3. Critical-boundary evaluation (%). Standard and Detailed prompts; metrics: § 5.3; counts and sensitivity: Supp. A.

ModelStandard (S)Detailed (D)
NearAcc ↑FarAcc ↑PairAcc ↑Coverage ↑VPSR ↑NearAcc ↑FarAcc ↑PairAcc ↑Coverage ↑VPSR ↑
CogVideoX-1.545.750.00.067.60.043.554.04.882.43.9
Cosmos-Predict251.237.27.776.55.960.046.1<u>20.9</u><u>84.3</u><u>17.6</u>
Cosmos3-Nano58.350.612.363.77.8<u>62.2</u>45.414.879.411.8
HunyuanVideo-1.547.749.413.757.27.860.860.612.586.310.8
MiniMax-H367.052.9<u>23.1</u>83.3<u>19.3</u>69.6<u>60.1</u>18.782.415.4
PhysAlign<u>60.0</u><u>60.7</u>20.282.416.745.145.711.982.49.8
PhysRVG50.052.30.064.10.050.050.00.066.00.0
VideoGPA54.252.97.254.23.953.256.815.563.19.8
Wan2.2-I2V-A14B48.362.830.774.522.959.355.442.182.434.6
Wan2.2-TI2V-5B35.425.00.036.30.040.955.68.370.65.9

Table 3.

Success on individual conditions does not ensure correct cross-boundary responses. Despite the highest Near-Acc, 67.0% and 69.6%, MiniMax-H3 attains PairAcc of only 23.1% and 18.7%. Wan2.2-I2V-A14B instead leads in PairAcc at 30.7% and 42.1%, and in VPSR at 22.9% and 34.6%. Even the strongest model scores below 50% PairAcc, while PhysRVG scores 0% under both prompt formats.

Despite outcome descriptions in Standard and Detailed prompts, paired success remains limited. Under Concat

prompts without descriptions of the registered outcomes, PairAcc ranges from 0 to 12.0% across the ten models on the same 51 pairs, highlighting sensitivity to prompt formulation (Supp. I).

Complementary physical diagnoses

Joint parameter and event evaluation reveals complementary failure modes (Supp. Table 39). Among 2,385 gravity records evaluated at both levels, 48 have small parameter errors but incorrect event outcomes, while 929 have large parameter errors yet correct outcomes. Both patterns also occur for friction and initial speed. In Figure 7, Cosmos-Predict2 reproduces contact despite wrong speed, while Cosmos3-Nano recovers gravity but misses the domino chain.

Figure 7. Complementarity of parameter- and event-level diagnostics. Top: Cosmos-Predict2 matches both contact outcomes, yet both speed fits are rejected; v∗v^* is the lowest-loss candidate, and its percentage a deviation, not an accepted error. Bottom: Cosmos3-Nano recovers gravity within 9.6% but misses the complete domino chain.

Fine-tuning on PhysResponse

We fine-tune Wan2.2-TI2V-5B on 16 types, excluding benchmark cases and source-sharing videos, and evaluate 80 ID cases and 20 cases from four held-out types. PR-S/PR-D continue training Standard/Detailed text-only LoRA versions with a numerical encoder that adds sinusoidal embeddings of prompted values to their text tokens (Figure 8, Supp. E). Both variants lower all three parameter errors and improve motion agreement (Table 4). Held-out gravity error falls from 140.3% to 39.8–68.6%, and speed error from 93.1% to 13.0–16.4% (Supp. Table 36). Relative to their text-only versions, PR-S/PR-D further lower gravity error on the 100 cases, from 71.0%/85.3% to 69.1%, and all three held-out errors (Supp. Tables 34 and 36). PR-S improves all ten event metrics; PR-D lowers three, including Detailed PairAcc (Table 5). Parameter gains need not improve events. On real WorldBench videos [45], both variants also lower gravity and ramp-friction errors (Supp. Table 37). On 100 PhysInOne training-split scenes [64], they outperform the base model and PhysInOne LoRA/SFT in PMF, PSNR, SSIM and LPIPS; SFT retains the best FVD (Table 6).

ModelL2 err. ↓\downarrowS-IoU ↑\uparrowPSNR ↑\uparrowSSIM ↑\uparrowDispl. →1\to1Order ↑\uparrowg↓g \downarrowμ↓\mu\downarrowv↓v \downarrowMeas. ↑\uparrow
Wan2.2-TI2V-5B16.10%0.05123.20.8190.0613%94.9%130.3%85.8%35%
PR-S13.84%0.30532.10.949<u>0.59</u>20%69.1%70.6%45.6%80%
PR-D<u>14.69%</u><u>0.292</u><u>32.0</u><u>0.948</u>0.7225%69.1%<u>87.6%</u><u>67.3%</u><u>78%</u>

Table 4. Fine-tuning on PhysResponse lowers all three parameter errors. Wan2.2-TI2V-5B on the 100 cases; bold: best, underlined: second. Text-only versions: Supp. Table 34; event results: Table 5.

Model\multicolumn{5}{c}{Standard (S)}\multicolumn{5}{c}{Detailed (D)}
NearAcc↑\uparrowFarAcc↑\uparrowPairAcc↑\uparrowCoverage↑\uparrowVPSR↑\uparrowNearAcc↑\uparrowFarAcc↑\uparrowPairAcc↑\uparrowCoverage↑\uparrowVPSR↑\uparrow
Wan2.2-TI2V-5B35.425.00.036.30.040.955.6<u>8.3</u>70.6<u>5.9</u>
PR-S42.861.015.096.114.4<u>50.4</u>60.410.296.19.8
PR-D33.0<u>58.0</u><u>14.3</u>96.1<u>13.7</u>54.7<u>60.2</u>4.896.14.6

Table 5. Event response after fine-tuning (%). Standard and Detailed prompts; baseline and metrics as in Table 3. Text-only results: Supp. Table 17. Bold/underlined: best/second.

Figure 8. Matched fine-tuning example. T17_C03, Standard prompt: GT, Wan2.2-TI2V-5B, PR-S and PR-D. Right insets show fitted gravity and contact outcomes; the Wan2.2-TI2V-5B result is marked as excluded.

ModelPMF ↑\uparrowPSNR ↑\uparrowSSIM ↑\uparrowLPIPS ↓\downarrowFVD ↓\downarrow
Wan2.2-TI2V-5B2.05812.50.4040.5031334.3
PhysInOne LoRA†2.83515.80.6220.337893.4
PhysInOne SFT†2.88015.70.6850.305772.3
PR-S<u>3.144</u>17.8<u>0.738</u><u>0.239</u>794.7
PR-D3.14817.80.7400.238<u>786.2</u>

Table 6. Fine-tuning on PhysResponse transfers to PhysInOne. 100 training-split scenes, seed 0; †fine-tuned by the PhysInOne authors on this split. Bold/underlined: best/second.

Discussion and Limitations

Effective parameters depend on context cc, so compensating quantities such as μg\mu g need conditional estimates (Eq. (2)). Real-video transfer is tested only on WorldBench, and held-out types add no new objects or parameter ranges. Event scores rest on few scene clusters and partly on visual proxies. Encoder gains are not isolated from continued training, and benefits for embodied policies remain untested.

Conclusion

We present PhysResponse, a controlled multi-object video dataset and benchmark for evaluating physical response through parameter recovery and critical-transition events. Tests on ten models show that correct events can coexist with incorrect dynamics, while visual metrics often miss these failures. Fine-tuning reduces parameter errors on held-out interactions and transfers to external benchmarks. For video models to serve as world simulators, they must capture how changes in physical conditions alter future motion and interactions. PhysResponse provides training data and evaluation tools for this capability, a foundation for predicting action consequences in embodied planning.

References

  1. [1]Hritik Bansal, Zongyu Lin, Tianyi Xie, Zeshun Zong, Michal Yarom, Yonatan Bitton, Chenfanfu Jiang, Yizhou Sun, Kai-Wei Chang, and Aditya Grover. VideoPhy: Evaluating physical commonsense for video generation. In Int.Conf.Learn.Represent., 2025. 3arxiv.org/abs/2406.03520
  2. [2]Hritik Bansal, Clark Peng, Yonatan Bitton, Roman Goldenberg, Aditya Grover, and Kai-Wei Chang. VideoPhy-2: A challenging action-centric physical commonsense evaluation in video generation. In Int.Conf.Learn.Represent., 2026. 3, 4arxiv.org/abs/2503.06800
  3. [3]Fabien Baradel, Natalia Neverova, Julien Mille, Greg Mori, and Christian Wolf. CoPhy: Counterfactual learning of physical dynamics. In Int.Conf.Learn.Represent., 2020. 3
  4. [4]Daniel M. Bear, Elias Wang, Damian Mrowca, Felix J. Binder, Hsiao-Yu Fish Tung, R. T. Pramod, Cameron Holdaway, Sirui Tao, Kevin Smith, Fan-Yun Sun, Li Fei-Fei, Nancy Kanwisher, Joshua B. Tenenbaum, Daniel L. K. Yamins, and Judith E. Fan. Physion: Evaluating physical prediction from vision in humans and machines. In Adv.NeuralInf.Process.Syst. Datasets and Benchmarks Track, 2021. 3arxiv.org/abs/2106.08261
  5. [5]León Begristain, Olaf Dünkel, and Adam Kortylewski. CRONOS: Benchmarking counterfactual physical consistency in video models. arXiv preprint arXiv:2605.23699, 2026. 3
  6. [8]Kunlin Cai, Rui Song, Jinghuai Zhang, Kaiyuan Zhang, Pranav Bodapati, Alicia Yu, Fnu Suya, Mohammad Rostami, Jiaqi Ma, and Yuan Tian. What-If World: A causal benchmark for general world models in embodied scenarios. arXiv preprint arXiv:2605.27589, 2026. 3, 4arxiv.org/abs/2605.27589
  7. [9]Nicolas Carion, Laura Gustafson, Yuan-Ting Hu, Shoubhik Debnath, Ronghang Hu, et al. SAM 3: Segment anything with concepts. In Int.Conf.Learn.Represent., 2026. 4, 5arxiv.org/abs/2511.16719
  8. [10]Zhenfang Chen, Kexin Yi, Yunzhu Li, Mingyu Ding, Antonio Torralba, Joshua B. Tenenbaum, and Chuang Gan. ComPhy: Compositional physical reasoning of objects and events from videos. In Int.Conf.Learn.Represent., 2022. 3arxiv.org/abs/2205.01089
  9. [11]Hongyang Du, Junjie Ye, Xiaoyan Cong, Runhao Li, Jingcheng Ni, Aman Agarwal, Zeqi Zhou, Zekun Li, Randall Balestriero, and Yue Wang. VideoGEPA: Distilling geometry priors for 3D-consistent video generation. In Int.Conf.Mach.Learn., 2026. 2, 6
  10. [12]Lin Geng Foo, Mark He Huang, Alexandros Lattas, Stylianos Moschoglou, Thabo Beeler, and Christian Theobalt. Physical simulator in-the-loop video generation. In IEEEConf.Comput.Vis.PatternRecog., 2026. 3
  11. [13]Genesis Authors. Genesis: A generative and universal physics engine for robotics and beyond. https://github.com/Genesis-Embodied-AI/genesis-world, 2024. Accessed: 2026-10-27. 4
  12. [14]Nate Gillman, Charles Herrmann, Michael Freeman, Daksh Aggarwal, Evan Luo, Deqing Sun, and Chen Sun. Force prompting: Video generation models can learn and generalize physics-based control signals. In Adv.NeuralInf.Process.Syst., 2025. 3
  13. [15]Jing Gu, Xian Liu, Yu Zeng, Ashwin Nagarajan, Fangrui Zhu, Daniel Hong, Yue Fan, Qianqi Yan, Kaiwen Zhou, Ming-Yu Liu, and Xin Eric Wang. PhyWorldBench: A comprehensive evaluation of physical realism in text-to-video models. In Int.Conf.Learn.Represent., 2026. 3arxiv.org/abs/2507.13428
  14. [16]Xuyang Guo, Jiayan Huo, Zhenmei Shi, Zhao Song, Jiahao Zhang, and Jiale Zhao. T2VPhysBench: A first-principles benchmark for physical consistency in text-to-video generation. arXiv preprint arXiv:2505.00337, 2025. 3
  15. [17]Swarnim Jain and Shangzhe Wu. RigidBench: Evaluating rigid-body physics in video generation models. arXiv preprint arXiv:2608.15555, 2026. 2, 3, 4arxiv.org/abs/2608.15555
  16. [18]Rasul Khanbayov and Hasan Kurban. PhysWeep: Does a video generator realize the physics you ask for? arXiv preprint arXiv:2609.06207, 2026. 3, 4arxiv.org/abs/2609.06207
  17. [19]Minh-Quan Le, Yuanzhi Zhu, Vicky Kalgeiton, and Dimitris Samaras. What about gravity in video generation? Post-training Newton’s laws with verifiable rewards. arXiv preprint arXiv:2512.00425, 2025. 3
  18. [20]Chenyu Li, Oscar Michel, Xichen Pan, Sainan Liu, Mike Roberts, and Saining Xie. PISA experiments: Exploring physics post-training for video diffusion models by watching stuff drop. In Int.Conf.Mach.Learn., 2025. 3arxiv.org/abs/2503.09595
  19. [21]Dacheng Li, Yunhao Fang, Yukang Chen, Shuo Yang, Shiyi Cao, Justin Wong, Michael Luo, Xiaolong Wang, Hongxu Yin, Joseph E. Gonzalez, Ion Stoica, Song Han, and Yao Lu. WorldModelBench: Judging video generation models as world models. In Adv.NeuralInform.Process.Syst. Datasets and Benchmarks Track, 2025. 3
  20. [22]Jialiang Li, Zhengning Xu, Zongyang Ma, and Shaobo Cui. A plausible future is not a causal future: Can video models generate how the world unfolds? arXiv preprint arXiv:2610.15562, 2026. 3
  21. [23]Xuan Li, Yi-Ling Qiao, Peter Yichen Chen, Krishna Murthy Jatavallabhula, Ming Lin, Chenfanfu Jiang, and Chuang Gan. PAC-NeRF: Physics augmented continuum neural radiance fields for geometry-agnostic system identification. In Int.Conf.Learn.Represent., 2023. 3arxiv.org/abs/2303.05512
  22. [24]Yuqi Li, Junming Liu, Bowen Hu, Ruize Niu, Huiran Duan, Ming-Hsuan Yang, Yingli Tian, and Hao Wu. GeoFlip preview: Benchmarking geometry-sensitive physics in video generation. arXiv preprint arXiv:2610.42002, 2026. 3, 4arxiv.org/abs/2610.42002
  23. [25]Yanxun Li, Hao Wen, Bingze Song, Jiashu Zhu, Aiming Hao, Chubin Chen, Jintao Chen, Jiaohong Wu, Xiangxiang Chu, and Miao Wang. Learning explicit physical parameter control and benchmarking for video generation. arXiv preprint arXiv:2607.18924, 2026. 2, 3, 4arxiv.org/abs/2607.18924
  24. [26]Shaowei Liu, Zhongzheng Ren, Saurabh Gupta, and Shenlong Wang. PhysGen: Rigid-body physics-grounded image-to-video generation. In Eur.Conf.Comput.Vis., 2024. 3arxiv.org/abs/2409.18964
  25. [27]Fanqing Meng, Jiaqi Liao, Xinyu Tan, Quanfeng Lu, Wenqi Shao, Kaipeng Zhang, Yu Cheng, Dianqi Li, and Ping Luo. Towards world simulator: Crafting physical commonsense-based benchmark for video generation. In Int.Conf.Mach.Learn., 2025. 3, 4arxiv.org/abs/2410.05363
  26. [28]Isaiah Milkley, Somar Sagar, Aditya Taparia, Xinyuan Liu, Jiqing Wen, and Ransalu Senanayake. PhysicsLENS: Diagnosing physical property blindness in video generation models. arXiv preprint arXiv:2610.0162, 2026. 3arxiv.org/abs/2610.01162
  27. [29]MiniMax. MiniMax-H3. https://huggingface.co/MiniMaxAI/MiniMax-H3, 2026. Accessed: 2026-10-27. 1, 6
  28. [30]Saman Motamed, Laura Culp, Kevin Swersky, Priyank Jaini, and Robert Geirhos. Do generative video models understand physical principles? In IEEEWinterConf.Appl.Comput.Vis., 2026. 2, 3, 4, 6arxiv.org/abs/2501.09038
  29. [31]Sriram Narayanan, Ziyu Jiang, Srinivasa Narasimhan, and Manmohan Chandraker. PhyCo: Learning controllable physical priors for generative motion. In IEEEConf.Comput.Vis.PatternRecog., 2026. 2, 3arxiv.org/abs/2604.28169
  30. [32]NVIDIA. Cosmos-Predict2. https://github.com/nvidia-cosmos/cosmos-predict2, 2025. Accessed: 2026-10-27. 2, 6
  31. [33]NVIDIA. Isaac Sim. https://github.com/isaac-sim/IsaacSim, 2026. Accessed: 2026-10-27. 4
  32. [34]NVIDIA, Aditi, Niket Agarwal, Arslan Ali, Jon Allen, et al. Cosmos 3: Omnimodal world models for physical AI. arXiv preprint arXiv:2606.02800, 2026. 2, 6
  33. [36]Qwen Team. Qwen3-8-27B. https://huggingface.co/Qwen/Qwen3.8-27B, 2026. Accessed: 2026-11-12. 5
  34. [38]Team Wan, Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, et al. Wan: Open and advanced large-scale video generative models. arXiv preprint arXiv:2503.20314, 2025. 1, 6arxiv.org/abs/2503.20314
  35. [39]Tencent Hunyuan Foundation Model Team. HunyuanVideo 1.5 technical report. arXiv preprint arXiv:2511.18870, 2025. 1, 6
  36. [40]Varun Varma Thozhiyeoor, Shivam Tripathi, Venkatesh Babu Radhakrishnan, and Anand Bhattad. Principia: Relational physics tests for video models. arXiv preprint arXiv:2609.04200, 2026. 3
  37. [41]Varun Varma Thozhiyeoor, Shivam Tripathi, Venkatesh Babu Radhakrishnan, and Anand Bhattad. Objects in generated videos are slower than they appear: Models suffer sub-Earth gravity and don’t know Galileo’s principle...for now. In IEEEConf.Comput.Vis.PatternRecog. Findings, 2026. 3
  38. [42]Antonios Tragoudaras, Chenyu Zhang, Daniil Cherniavskii, Antonios Vokizis, Thijmen Nijdam, Derck W. E. Prinzhor n, Mark Bodrackska, Nicu Sebe, Andrii Zadaianchuk, and Efstratios Gavves. Evaluating Newtonian mechanics in video generative models with real physical systems. In Int.Conf.Mach.Learn., 2026. 2, 3arxiv.org/abs/2504.02918
  39. [43]Hsiao-Yu Tung, Mingyu Ding, Zhenfang Chen, Daniel Bear, Chuang Gan, Joshua B. Tenenbaum, Daniel L. K. Yamins, Judith E. Fan, and Kevin A. Smith. Physyon++: Evaluating physical scene understanding that requires online inference of different physical properties. In Adv.NeuralInform.Process.Syst. Datasets and Benchmarks Track, 2023. 3arxiv.org/abs/2306.15668
  40. [45]Rishi Upadhyay, Howard Zhang, Jim Solomon, Ayush Agrawal, Yunhao Bai, Alex Wong, Celso M. de Melo, and Achuta Kadambi. WorldBench: Benchmarking physical understanding of world models by isolating physics concepts. arXiv preprint arXiv:2601.21282, 2026. 3, 4, 9arxiv.org/abs/2601.21282
  41. [46]Chen Wang, Chuhac Chen, Yiming Huang, Zhiyang Dou, Yuan Liu, Jiatao Gu, and Lingjie Liu. PhysCtrl: Generative physics for controllable and physics-grounded video generation. In Adv.NeuralInform.Process.Syst., 2025. 3arxiv.org/abs/2509.20358
  42. [49]Shuai Wang, Yaxin Feng, Xuekun Jiang, Shihan Tian, Ningyu Yan, Xing Shen, Chaoyang Lyu, Hui Wang, Yunsong Zhou, Hanqing Wang, Jiangmiao Pang, Yang Xiang, Xing Gao, Chunhua Shen, and Weinan Zhang. GAUGE: A measurement-grounded benchmark for physical fidelity in simulation engines and video world models. arXiv preprint arXiv:2608.05948, 2026. 3arxiv.org/abs/2608.05948
  43. [50]Zhou Wang, Alan C. Bovik, Hamid R. Sheikh, and Eero P. Simoncelli. Image quality assessment: From error visibility to structural similarity. IEEE Trans. Image Process., 13(4): 600–612, 2004. 6pubmed.ncbi.nlm.nih.gov/15376593
  44. [51]Jiajun Wu, Ilker Yildirim, Joseph J. Lim, Bill Freeman, and Josh Tenenbaum. Galileo: Perceiving physical object properties by integrating a physics engine with deep learning. In Adv.NeuralInform.Process.Syst., 2015. 3
  45. [52]Jiajun Wu, Joseph Lim, Hongyi Zhang, Joshua Tenenbaum, and William Freeman. Physics 101: Learning physical object properties from unlabeled videos. In Brit.Mach.Vis.Conf., 2016. 3DOI
  46. [53]Zhexiao Xiong, Yizhi Song, Liu He, Wei Xiong, Yu Yuan, Feng Qiao, and Nathan Jacobs. PhysAlign: Physics-coherent image-to-video generation through feature and 3D representation alignment. arXiv preprint arXiv:2603.13770, 2026. 2, 6arxiv.org/abs/2603.13770
  47. [55]Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiang Yuan, Yuanming Yang, Wenyi Hong, Xiaohan Zhang, Guanyu Feng, Da Yin, Yuxuan Zhang, Weihan Wang, Yean Cheng, Bin Xu, Xiaotao Gu, Yuxiao Dong, and Jie Tang. CogVideoX: Text-to-video diffusion models with an expert transformer. In Int.Conf.Learn.Represent., 2025. 1, 6arxiv.org/abs/2408.06072
  48. [56]Runmao Yao, Kairui Hu, Yukang Cao, Ruisi Wang, Shulin Tian, Ziang Cao, Weichen Fan, Ziqi Huang, Yuhao Dong, Hao Li, Zhaoxi Chen, Zhongang Cai, Lei Yang, and Ziwei Liu. Apple-π: Benchmarking thinking with video towards law-grounded physical intelligence. arXiv preprint arXiv:2607.16401, 2026. 3arxiv.org/abs/2607.16401
  49. [57]Kexin Yi, Chuang Gan, Yunzhu Li, Pushmeet Kohli, Jiajun Wu, Antonio Torralba, and Joshua B. Tenenbaum. CLEVRER: CoLlision Events for Video Representation and reasoning. In Int.Conf.Learn.Represent., 2020. 3arxiv.org/abs/1910.01442
  50. [58]Yu Yuan, Xijun Wang, Tharindu Wickremasinghe, Zeeshan Nadir, Bole Ma, and Stanley H. Chan. NewtonGen: Physics-consistent and controllable text-to-video generation via neural Newtonian dynamics. In Int.Conf.Learn.Represent., 2026. 3arxiv.org/abs/2509.21309
  51. [59]Conghan Yue, Yuanjie Chen, Yue Han, Ya Gao, Yunyan Xiao, WeiYao Zhang, and Zhineng Chen. VideoPhysEdit: Physical counterfactual video editing via rigid-body physical scene reconstruction. arXiv preprint arXiv:2609.35134, 2026. 3arxiv.org/abs/2609.35134
  52. [60]Guanqi Zhang, Xianzhen Ma, Weidi Xie, and Andrew Zisserman. Inferring dynamic physical properties from video foundation models. arXiv preprint arXiv:2510.02311, 2025. 3
  53. [61]Qiyuan Zhang, Biao Gong, Shuai Tan, Zheng Zhang, Xing Zhu, Yujun Shen, Yuyuan Li, Kelu Yao, Chunhua Shen, and Changqing Zou. PhysRVG: Physics-aware unified reinforcement learning for video generative models. In Eur.Conf.Comput.Vis., 2026. 2, 3, 6arxiv.org/abs/2601.11087
  54. [63]Dian Zheng, Ziqi Huang, Hongbo Liu, Kai Zou, Yinan He, Fan Zhang, Lulu Gu, Yuanhan Zhang, Jingwen He, Wei-Shi Zheng, Yu Qiao, and Ziwei Liu. VBench-2.0: Advancing video generation benchmark suite for intrinsic faithfulness. arXiv preprint arXiv:2503.21755, 2025. 3arxiv.org/abs/2503.21755
  55. [64]Siyuan Zhou, Hejun Wang, Chuanzheng Chen, Jinxi Li, Dongsheng Wang, et al. PhysInOne: Visual physics learning and reasoning in one suite. In IEEEConf.Comput.Vis.PatternRecog., 2026. 2, 3, 4, 9arxiv.org/abs/2604.09415

Paper details

vlnc:W_t6anpxuc
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 11

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents