betaFeedback

WorksComputer Science

IMPACT: Image-Conditioned Motion with Physics-Aware Collision and Trajectory Guidance for Video Generation

Abstract

Visually realistic video generation does not guarantee physically consistent motion. Generated objects may exhibit implausible collision responses or rebounds without energy dissipation, while even physically valid motion predictions may not be faithfully realized during synthesis. We propose IMPACT, a physics-state-driven image-to-video framework that addresses these challenges by coupling explicit dynamics prediction with complementary sparse–dense physical guidance. A learnably calibrated Physics State Generator unifies continuous free-flight dynamics with discrete collision responses, linking impact timing, post-impact velocity, rebound attenuation, and eventual settling within a coherent physical-state sequence. To translate this evolution into effective generative constraints, IMPACT derives sparse trajectories and dense region-level motion priors from the same physical rollout. Sparse trajectories constrain global object displacement, while dense priors guide diffusion sampling to encourage coherent object-region motion and suppress unwanted background changes. Their shared physical basis coordinates global and local motion guidance to reduce prediction–generation mismatch. Experiments on 100 controlled sphere–plane collision scenes spanning different material combinations and visual environments demonstrate improved physical plausibility over representative general-purpose, physics-grounded, and motion-controllable baselines. The project repository is available at https://weiweiwiewei.github.io/impact

Introduction

Generating physically consistent videos is essential for visual generative models to capture how the world evolves, with applications in embodied intelligence, robotics, and interactive simulation. Despite substantial advances in visual quality, generated videos still exhibit implausible contact responses, rebounds without energy dissipation, or abrupt motion termination. These failures reveal a limitation beyond visual realism: individually plausible actions do not necessarily form a physically coherent interaction.

Existing approaches incorporate physical knowledge into video generation through language reasoning, physics-based simulation, or explicit motion control. Language-based methods, such as Physics-Lang [1], describe physical interactions and their consequences, but such descriptions do not explicitly determine impact timing, collision velocities, or energy dissipation. Physics-grounded approaches, including PhysGen [2] and MotionCraft [3], introduce simulated dynamics to provide more concrete motion references, while motion-controllable methods guide synthesis through trajectories, optical flow, or spatial correspondences. Despite these advances, physically coherent motion specification and its faithful visual realization remain distinct requirements. A motion reference may describe a valid physical process without fully constraining how that process unfolds in the generated video. This distinction exposes a fundamental gap between physical-state prediction and generative motion realization.

Bridging this gap involves two challenges. The first is inconsistent motion evolution, where individually plausible actions fail to form a physically coherent sequence. As illustrated in Figure 1(a), a sphere may rebound above its initial release height or continue bouncing without sufficient energy loss. The underlying difficulty lies in maintaining state-dependent consistency across successive physical events. Free-flight dynamics determine the incoming collision state, the collision response determines the outgoing velocity, and the remaining energy governs subsequent rebounds and settling. Without explicitly connecting these transitions, a generated sequence may depict recognizable physical events while violating the causal relationships among them. The second challenge is prediction-generation mismatch. Even when a physically valid motion prediction is available, its visual realization remains underdetermined. A center trajectory specifies the global displacement of an object but does not fully constrain how its entire region should move, how its boundary should interact with the supporting surface, or which background regions should remain stationary. Consequently, matching object-center positions does not necessarily guarantee consistent region-level motion, stable contact, or background preservation, as shown in Figure 1(b).

Illustration of two challenges in physically consistent video generation

Figure 1. Illustration of two challenges in physically consistent video generation. (a) Inconsistent motion evolution: a sphere rebounds above its initial release height despite being released from rest, indicating an implausible gain in energy. (b) Prediction-generation mismatch: even when object positions follow a physically valid prediction (cyan outlines), the generated video can exhibit object deformation, unstable contact, and background drift.

To address these challenges, we propose IMPACT, a physics-state-driven image-to-video framework that couples explicit dynamics prediction with sparse-dense physical guidance. IMPACT predicts a connected sequence of physical states and uses that sequence to guide both global object displacement and region-level motion during synthesis. We study sphere-plane interactions involving free fall, successive collisions, and eventual settling to evaluate the physical consistency of predicted motion and its visual realization across different materials and environments.

To address inconsistent motion evolution, IMPACT makes each stage of the interaction depend explicitly on the physical state reached in the preceding stage. The Image-to-Physics (I2P) module establishes the conditions governing this evolution by estimating a compact, motion-relevant physical configuration from the initial image and text prompt. Building on this configuration, the Physics State Generator (PSG) evolves position and velocity through continuous free-flight dynamics until contact triggers a discrete collision response. The updated velocity then governs the next rebound, linking successive motion stages through a common dynamical process. Learnably calibrated restitution controls the collision response, while explicit rebound attenuation and settling model how repeated impacts progressively bring the sphere to rest. This connected physical evolution is projected into sparse image-space trajectories, allowing a pretrained trajectory-control interface to guide global object displacement according to the predicted dynamics.

To reduce prediction-generation mismatch, IMPACT complements these sparse trajectories with dense physical guidance during diffusion sampling. Rather than relying on center positions alone, we derive dense object masks and optical flow from the same PSG rollout to provide region-level motion priors. Through energy-based latent guidance, these priors encourage motion across the generated object region to follow the physical prediction while suppressing unwanted background changes. Sparse trajectories anchor the overall motion path, whereas dense priors guide how that motion is realized across the object and surrounding scene. Their shared physical origin ensures that both forms of guidance refer to the same predicted interaction, providing complementary constraints on its visual realization.

We evaluate IMPACT on 100 controlled sphere-plane collision scenes with different material combinations and visual environments. Human and vision-language-model evaluations assess physical event completeness, free-fall dynamics, contact realism, rebound attenuation, and overall physical plausibility. The results show improved physical plausibility over representative general-purpose, physics-grounded, and motion-controllable baselines.

Our main contributions are summarized as follows:

  • We propose IMPACT, an image-to-video framework to extend physical consistency beyond motion prediction to two interconnected requirements: consistency in physical-state evolution and faithful visual realization of predicted motion.

  • We develop an event-driven hybrid dynamics model that unifies continuous motion and discrete collision events, explicitly establishing causal dependencies across successive physical events.

  • We translate a unified physical-state evolution into complementary sparse–dense generative constraints, bridging the gap between physical motion prediction and video generation.

The remainder of this paper is organized as follows. Section II reviews related work. Section III presents the proposed method. Section IV provides extensive experiments and analysis. Finally, § 4 concludes this paper.

Physics-Grounded Video Generation

Physics-grounded video generation seeks to bridge the gap between visual realism and physically plausible motion. One line of research explicitly couples physical simulation with visual synthesis. MotionCraft employs simulation-derived optical flow to warp diffusion noise, enabling physically guided motion generation. PhysGen performs image-space rigid-body simulation and uses generative refinement to synthesize realistic videos, while PhysGen3D [4] reconstructs an interactive 3D scene from a single image to simulate object interactions. DreamPhysics [5] estimates material properties to model deformable object dynamics. More recently, PSIVG [6] reconstructs object geometry and 4D scene information from a generated template video, performs physical simulation, and uses the resulting motion cues to guide video synthesis, with additional test-time optimization for texture consistency. These approaches demonstrate the effectiveness of simulation-based motion guidance, while differing in their requirements for scene reconstruction, physical initialization, and integration with generative models.

Another line of research incorporates physical knowledge through learned dynamics, latent representations, or physics-aware optimization. NewtonGen [7] introduces trainable Neural Newtonian Dynamics to predict physical motion and provide dynamical guidance for video synthesis. PhysVideoGenerator [8] incorporates learned latent physics representations into diffusion transformers, while other approaches explore physical priors, language-guided reasoning, and learned state evolution [9, 10, 11, 12, 13]. More recently, PhyCo [14] learns controllable physical responses through property-conditioned fine-tuning and reward optimization, whereas PhysRVC [15] employs physics-aware reinforcement learning to improve rigid-body motion generation. These methods introduce physical knowledge at different stages of the generation process, offering complementary alternatives to explicit simulation.

Despite these advances, rigid-body impacts remain challenging because continuous gravitational motion is interrupted by discrete contact events that alter object velocity and dissipate energy. Accurately modeling repeated rebounds therefore requires coordinated treatment of free-flight dynamics, collision responses, and eventual settling. This motivates an event-driven formulation that explicitly resolves physical state transitions throughout the motion process.

Motion-Controllable Video Generation

Motion-controllable video generation regulates object or camera motion through explicit spatial and temporal conditions. Trajectory-based methods provide intuitive control over object displacement and motion paths. TrackGo [16], Motion Prompting [17], and MotionPro [18] introduce trajectory- or region-level conditions to guide video synthesis. Tora [19] further encodes trajectories into hierarchical spatiotemporal motion representations and integrates them into diffusion transformers, while Wan-Move [20] propagates first-frame features along prescribed motion paths. These approaches demonstrate the effectiveness of trajectory guidance for specifying global motion structures and controlling object movement.

Beyond sparse trajectories, dense motion representations provide finer spatial constraints for video generation. Motion-I2V [21] separates motion-field prediction from video synthesis, allowing explicit motion representations to guide temporal generation. Go-with-the-Flow [22] introduces optical-flow-based noise warping to control motion, while FloVD [23] employs optical-flow representations for controllable video synthesis. MagicMotion [24] further supports multiple levels of trajectory conditions, including masks, bounding boxes, and sparse boxes, enabling motion control at different spatial granularities. Although these representations offer complementary levels of control, their ability to produce physically plausible motion depends on the underlying motion signals. In particular, specifying object positions or displacement fields does not inherently enforce gravitational acceleration, collision responses, or energy dissipation.

Training-Free Guidance for Video Diffusion

Training-free guidance provides a flexible mechanism for controlling pretrained diffusion models without updating their parameters. Diffusion Posterior Sampling [25], FreeDoM [26], and TFG [27] introduce inference-time guidance strategies based on likelihoods, differentiable objectives, or energy functions. By evaluating guidance objectives on denoising predictions and propagating gradients to the sampling variables, these methods steer generation toward desired constraints while preserving the pretrained model. Such approaches provide a general foundation for incorporating external information into diffusion sampling without task-specific retraining.

For video generation, training-free control has been extended to motion manipulation and temporal consistency. Motion-Zero [28] combines position-aware noise priors with spatial and temporal attention constraints to control object trajectories. Related studies investigate motion representations in pretrained video models [29], while MotionFlow [30] performs latent optimization using attention-derived motion cues. DiTFlow [31] extracts motion information from diffusion-transformer attention and employs optimization-based guidance for motion transfer. These approaches demonstrate that motion can be influenced through noise, attention, or latent-space optimization without updating the generative backbone. However, enforcing physical dynamics additionally requires temporally coordinated constraints that reflect the evolution of object states, particularly across discontinuous collision events.

Method

Problem Formulation and Framework Overview

Given an initial image I0∈RH×W×3I_0 \in\mathbb{R}^{H \times W \times3} and a text condition yy, the objective is to generate a video V^={I^t}t=0T−1\hat{V} = \{\hat{I}_t\}_{t=0}^{T-1} that preserves the appearance and scene structure of I0I_0 while exhibiting physically consistent motion. The considered scenario consists of a single rigid sphere released from rest above a stationary rigid plane, observed by a fixed, approximately side-view camera. The target motion comprises gravitational free fall, ground impact, successive rebounds, and eventual settling.

The proposed framework decouples physical motion modeling from video synthesis. As illustrated in Figure 2, an Image-to-Physics (I2P) module first estimates the initial physical state and camera parameters. An event-driven Physics State Generator (PSG) then propagates the state through continuous free flight and discrete collision events, producing a structured physical rollout. Finally, the rollout provides both sparse trajectory conditions and dense region-level constraints for a pretrained video diffusion model. The video backbone and trajectory-control module remain frozen, while only the physical parameter calibration components are trained.

Figure 2. Overview of IMPACT. The Image-to-Physics (I2P) module estimates a compact physical configuration from an initial image and a text prompt. The Physics State Generator (PSG) combines continuous free-flight dynamics with discrete collision events to model free fall, rebound attenuation, and settling. Sparse trajectories and dense region-level motion priors are derived from the same physical rollout and provide complementary guidance for video synthesis. Both the video backbone and the pretrained trajectory-control module remain frozen.

Initial Physical State Estimation

I2P extracts the physical quantities required for sphere–plane dynamics without reconstructing a complete three-dimensional scene. The initial physical state is represented as

S0phys={P0,V0,r,n,c,m},(1)\mathcal{S}^{\mathrm{phys}}_0 = \{\mathbf{P}_0,\mathbf{V}_0,r,\mathbf{n},c,m\}, \tag*{(1)}

where P0,V0∈R3\mathbf{P}_0,\mathbf{V}_0 \in\mathbb{R}^3 denote the initial sphere center and velocity, rr is the sphere radius, (n,c)(\mathbf{n},c) specifies the supporting plane with ∥n∥2=1\lVert\mathbf{n}\rVert_2 = 1, and mm denotes the sphere–ground material condition. The initial velocity is set to V0=0\mathbf{V}_0 = 0. Camera parameters Π0={K,Rcw,tcw}\Pi_0 = \{K,R_{cw},t_{cw}\} are estimated separately for subsequent projection.

As illustrated in Figure 3, I2P combines object localization, depth estimation, and camera calibration to recover the motion-relevant sphere–plane geometry, while the material condition is obtained from the text prompt. The resulting geometric configuration is transformed into a plane-aligned canonical coordinate system, where the supporting plane lies at z=0z = 0 and gravity acts along the negative zz-axis. For a sphere center Pt\mathbf{P}_t, the signed contact gap is

Visualization of the Image-to-Physics (I2P)

Figure 3. Visualization of the Image-to-Physics (I2P). (a) Object localization and segmentation. (b) Monocular depth estimation. (c) Camera calibration and gravity estimation. (d) Sphere–plane geometry recovery from back-projected observations. (e) Material conditioning from the text prompt. (f) Construction of the plane-aligned canonical physical state and camera parameters.

δt=n⊤Pt+c−r.(2)\delta_t = \mathbf{n}^{\top}\mathbf{P}_t + c - r. \tag*{(2)}

A positive gap indicates separation, while zero and negative values correspond to contact and penetration, respectively. This quantity provides the geometric condition for subsequent event-driven state evolution.

Event-Driven Physics State Generator

Sphere–plane motion exhibits hybrid dynamics, where continuous gravitational motion is interrupted by discrete collision events. PSG models these two forms of state evolution explicitly, rather than directly regressing future trajectories. Given the initial physical state, it generates a structured rollout

R={Pt,Vt,χt,et}t=0T−1,(3)\mathcal{R} = \{\mathbf{P}_t,\mathbf{V}_t,\chi_t,e_t\}_{t=0}^{T-1}, \tag*{(3)}

where Pt\mathbf{P}_t and Vt\mathbf{V}_t denote the sphere position and velocity, χt∈{0,1}\chi_t \in\{0,1\} indicates whether a collision occurs within the corresponding time interval, and ete_t is the effective restitution coefficient associated with an impact.

Physical Parameter Calibration

1) Physical Parameter Calibration: PSG employs lightweight parameter calibration to account for uncertainty in the physical quantities recovered from a single image. Rather than predicting object trajectories, the calibration module adjusts the parameters governing their evolution. Specifically, the nominal gravitational magnitude g0g_0 is corrected through a bounded function:

Δg=ϵgg0tanh⁡(fϕ(S0phys)),\Delta g = \epsilon_g g_0 \tanh\left(f_{\phi}\left(\mathcal{S}^{\mathrm{phys}}_0\right)\right),
geff=g0+Δg,g=−geffn,(4)g_{\mathrm{eff}} = g_0 + \Delta g,\qquad\mathbf{g} = -g_{\mathrm{eff}}\mathbf{n}, \tag*{(4)}

where fϕf_{\phi} is a learnable calibration network and ϵg\epsilon_g limits the relative correction. The calibrated gravitational magnitude compensates for scale uncertainty in monocular scene estimation without introducing unrestricted changes to the underlying dynamics.

Collision response is parameterized separately using a material- and velocity-dependent restitution coefficient:

em(v)=emin⁡,m+(emax⁡,m−emin⁡,m)(1+vvref)−km,(5)e_m(v) = e_{\min,m} + \left(e_{\max,m} - e_{\min,m}\right) \left(1+\frac{v}{v_{\mathrm{ref}}}\right)^{-k_m}, \tag*{(5)}

where emin⁡,me_{\min,m}, emax⁡,me_{\max,m}, and kmk_m control the restitution behavior of material condition mm, subject to 0≤emin⁡,m≤emax⁡,m≤10 \le e_{\min,m} \le e_{\max,m} \le1 and km≥0k_m \ge0. These parameterizations allow physical properties to be calibrated while retaining explicit state evolution.

Event-Driven State Evolution and Collision Response

2) Event-Driven State Evolution and Collision Response: Between collision events, the sphere follows constant-acceleration motion. Given the state (Pt,Vt)(\mathbf{P}_t,\mathbf{V}_t), its continuous evolution over an interval τ\tau is

Pt(τ)=Pt+Vtτ+12gτ2,\mathbf{P}_t(\tau) = \mathbf{P}_t + \mathbf{V}_t\tau+ \frac{1}{2}\mathbf{g}\tau^2,
Vt(τ)=Vt+gτ.(6)\mathbf{V}_t(\tau) = \mathbf{V}_t + \mathbf{g}\tau. \tag*{(6)}

Rather than applying collision updates only at discrete frame boundaries, PSG resolves the first valid contact time within each simulation interval Δt\Delta t:

τ∗=min⁡{τ∈(0,Δt]∣δt(τ)=0},(7)\tau^{*} = \min\{\tau\in(0,\Delta t] \mid\delta_t(\tau)=0\}, \tag*{(7)}

subject to the sphere approaching the supporting plane. The corresponding pre-impact velocity is V−=Vt(τ∗)\mathbf{V}^{-}=\mathbf{V}_t(\tau^{*}).

At contact, the normal component of velocity is updated according to the effective restitution coefficient et=em(−n⊤V−)e_t=e_m(-\mathbf{n}^{\top}\mathbf{V}^{-}):

V+=V−−(1+et)(n⊤V−)n.(8)\mathbf{V}^{+}=\mathbf{V}^{-}-(1+e_t)(\mathbf{n}^{\top}\mathbf{V}^{-})\mathbf{n}. \tag*{(8)}

The remaining portion of the current time interval is then propagated from the post-impact state. By explicitly locating impact events and resolving the corresponding velocity transitions, PSG maintains the temporal ordering of free flight and collision while reducing contact-time errors associated with frame-boundary updates.

Successive rebounds are obtained by repeatedly applying the continuous evolution and collision response. When the predicted rebound height becomes sufficiently small, a short settling transition brings the sphere to a nonpenetrating rest state, avoiding unresolved micro-bounces and abrupt termination. Numerical contact correction and settling thresholds are detailed in the supplementary material.

Physical-State Supervision

The learnable parameters of PSG are optimized using two complementary physical-state objectives. The position loss constrains the complete state trajectory, whereas the impact loss specifically supervises the discontinuous velocity changes induced by collisions. The training objective is

LPSG=Lpos+λimpactLimpact,(9)\mathcal{L}_{\mathrm{PSG}}=\mathcal{L}_{\mathrm{pos}}+\lambda_{\mathrm{impact}}\mathcal{L}_{\mathrm{impact}}, \tag*{(9)}

where λimpact\lambda_{\mathrm{impact}} is the hyperparameter and is set to 0.2 in the experiments.

Given predicted sphere-center positions P^b,t\hat{\mathbf{P}}_{b,t} and reference positions Pb,t\mathbf{P}_{b,t}, the position loss is defined as

Lpos=13BT∑b=1B∑t=0T−1∑d=13ρ(P^b,t,d−Pb,t,d),(10)\mathcal{L}_{\mathrm{pos}}=\frac{1}{3BT}\sum_{b=1}^{B}\sum_{t=0}^{T-1}\sum_{d=1}^{3}\rho\left(\hat{P}_{b,t,d}-P_{b,t,d}\right), \tag*{(10)}

where BB is the batch size, TT is the sequence length, and ρ(⋅)\rho(\cdot) denotes the Smooth-L1L_1 function. This term provides trajectory-level supervision across free-flight, rebound, and settling phases.

To constrain collision response, the impact loss measures the discrepancy between predicted and reference post-impact normal velocities:

Limpact=1∣I∣∑(b,t)∈Iρ(nb⊤V^b,t+−nb⊤Vb,t+),(11)\mathcal{L}_{\mathrm{impact}}=\frac{1}{|\mathcal{I}|}\sum_{(b,t)\in\mathcal{I}}\rho\left(\mathbf{n}_b^{\top}\hat{\mathbf{V}}^{+}_{b,t}-\mathbf{n}_b^{\top}\mathbf{V}^{+}_{b,t}\right), \tag*{(11)}

where I\mathcal{I} denotes the set of annotated impact states. The two objectives jointly constrain continuous motion and discrete collision response, while the resulting trajectories remain determined by explicit physical dynamics rather than direct neural prediction.

Physics-Coherent Video Generation

The physical rollout specifies the target motion, but its consistency is not automatically preserved during video generation. As illustrated in Figure 4, we employ two complementary guidance mechanisms within a frozen video diffusion backbone. Sparse trajectory conditioning transfers the predicted global motion and event timing through pretrained Motion Track IC-LoRA, while dense physical priors impose region-level constraints through sampling-time latent guidance. Both forms of guidance originate from the same physical rollout, connecting global motion control with local motion consistency and background stability.

Figure 4. Complementary Sparse-Dense Physical Guidance. The physics-derived sparse trajectory CtrajC_{\mathrm{traj}} conditions the frozen LTX-Video backbone through pretrained Motion Track IC-LoRA. Dense physical priors Pdense\mathcal{P}_{\mathrm{dense}} define object-motion and background-stability constraints on the predicted clean latent. Their gradients update the noisy latent during selected denoising steps without modifying the pretrained model parameters.

Physics-Derived Motion Conditioning

The three-dimensional sphere centers are projected into the image plane as pt=π(Pt;Π0)\mathbf{p}_t=\pi(\mathbf{P}_t;\Pi_0), with visibility indicators qt∈{0,1}q_t\in\{0,1\}. The projected trajectory is converted into the conditioning format of the pretrained Motion Track IC-LoRA:

Ctraj=Htraj({pt,qt}t=0T−1).(12)C_{\mathrm{traj}}=\mathcal{H}_{\mathrm{traj}}\left(\{\mathbf{p}_t,q_t\}_{t=0}^{T-1}\right). \tag*{(12)}

This sparse condition transfers the physically determined global motion and event timing to the video generator without retraining the trajectory control module.

To provide region-level constraints, the same trajectory is used to construct target masks MtM_t and a cumulative displacement field referenced to the initial frame:

Ft(x)=M0(x)(pt−p0),(13)\mathbf{F}_t(\mathbf{x})=M_0(\mathbf{x})(\mathbf{p}_t-\mathbf{p}_0), \tag*{(13)}

where M0M_0 is the initial object mask. Under the fixed-camera and approximately constant-depth setting, this field describes the dominant translational motion of the target. Reliable static background regions are defined by

Bt=1−Dilate⁡(M0∨Mt),(14)B_t = 1 - \operatorname{Dilate}(M_0 \lor M_t), \tag*{(14)}

excluding the initial and current object regions and their uncertain boundaries.

The resulting dense prior Pdense={Mt,Ft,Bt,qt}t=0T−1\mathcal{P}_{\mathrm{dense}} = \{M_t, \mathbf{F}_t, B_t, q_t\}_{t=0}^{T-1} is aligned with the spatiotemporal resolution of the video VAE:

Pdensez=Az(Pdense)={Mℓz,Fℓz,Bℓz,qℓz}ℓ=0Tz−1.(15)\mathcal{P}_{\mathrm{dense}}^z = \mathcal{A}_z(\mathcal{P}_{\mathrm{dense}}) = \{M_\ell^z, \mathbf{F}_\ell^z, B_\ell^z, q_\ell^z\}_{\ell=0}^{T_z-1}. \tag*{(15)}

Here Az\mathcal{A}_z denotes spatial downsampling and temporal alignment, and TzT_z is the latent sequence length. Unlike sparse trajectory conditions, these dense priors are not directly injected as network inputs; instead, they define constraints for the subsequent denoising process.

Sampling-Time Physics-Guided Diffusion

As illustrated in the lower part of Figure 4, dense physical priors complement sparse trajectory conditioning by imposing differentiable constraints on object-region motion and background stability during diffusion sampling. Therefore, dense physical priors are incorporated through differentiable latent-space constraints during diffusion sampling.

Let zsvz_s^v denote the noisy video latent at denoising step ss, with noise level σs\sigma_s. Conditioned on the initial image, text, and physics-derived trajectory, the frozen denoiser produces an estimate of the clean latent:

z^0(s)=Dθ(zsv,σs;I0,y,Ctraj).(16)\hat{z}_0^{(s)} = \mathcal{D}_{\theta}\left(z_s^v, \sigma_s; I_0, y, C_{\mathrm{traj}}\right). \tag*{(16)}

The initial target representation is transported to its physically predicted location using the cumulative displacement field:

z~0,ℓ(s)=W(z^0,0(s),F~ℓ),(17)\tilde{z}_{0,\ell}^{(s)} = \mathcal{W}\left(\hat{z}_{0,0}^{(s)}, \tilde{\mathbf{F}}_\ell\right), \tag*{(17)}

where W\mathcal{W} denotes differentiable spatial warping.

The object-motion consistency energy is defined as

Eobj(s)=∑ℓ=1Tz−1qℓz∥Mℓz⊙(z^0,ℓ(s)−z~0,ℓ(s))∥1∑ℓ=1Tz−1qℓz∥Mℓz∥1+ε,(18)E_{\mathrm{obj}}^{(s)} = \frac{\sum_{\ell=1}^{T_z-1} q_\ell^z \left\|M_\ell^z \odot\left(\hat{z}_{0,\ell}^{(s)}-\tilde{z}_{0,\ell}^{(s)}\right)\right\|_1} {\sum_{\ell=1}^{T_z-1} q_\ell^z \left\|M_\ell^z\right\|_1+\varepsilon}, \tag*{(18)}

where ⊙\odot denotes elementwise multiplication. This term constrains the generated target region to remain consistent with the physics-derived displacement, rather than relying solely on point-level trajectory alignment.

A complementary background-stability energy penalizes changes in regions expected to remain static:

Ebg(s)=∑ℓ=1Tz−1qℓz∥Bℓz⊙(z^0,ℓ(s)−z^0,0(s))∥1∑ℓ=1Tz−1qℓz∥Bℓz∥1+ε,(19)E_{\mathrm{bg}}^{(s)} = \frac{\sum_{\ell=1}^{T_z-1} q_\ell^z \left\|B_\ell^z \odot\left(\hat{z}_{0,\ell}^{(s)}-\hat{z}_{0,0}^{(s)}\right)\right\|_1} {\sum_{\ell=1}^{T_z-1} q_\ell^z \left\|B_\ell^z\right\|_1+\varepsilon}, \tag*{(19)}

where ε>0\varepsilon> 0 provides numerical stability.

Following differentiable sampling-time guidance [25, 32], the two energies are combined as

Ephy(s)=λobj(s)Eobj(s)+λbg(s)Ebg(s),(20)E_{\mathrm{phy}}^{(s)} = \lambda_{\mathrm{obj}}^{(s)} E_{\mathrm{obj}}^{(s)} + \lambda_{\mathrm{bg}}^{(s)} E_{\mathrm{bg}}^{(s)}, \tag*{(20)}

and the current noisy latent is updated by

z~sv=zsv−η∇zsvEphy(s).(21)\tilde{z}_s^v = z_s^v - \eta\nabla_{z_s^v} E_{\mathrm{phy}}^{(s)}. \tag*{(21)}

Although the energy is evaluated on the predicted clean latent, its gradient is propagated to the current noisy latent through the frozen denoising computation. The sampler subsequently continues from z~sv\tilde{z}_s^v without updating the video backbone or trajectory-control parameters.

Physics guidance is applied during an intermediate portion of the denoising process, with scheduled guidance strength and energy weights to limit interference with visual synthesis. The physical rollout and corresponding priors are computed once before sampling, without repeated physical simulation or video-level motion estimation inside the denoising loop. In this manner, the sparse condition specifies the global motion, while the dense latent constraints regulate region-level consistency using the same physical rollout.

Experiments

Experimental Setup

Benchmarks. Two datasets are used to evaluate video generation and physical-state prediction. Specifically, we construct a rigid-body collision benchmark containing 100 initial images and corresponding text prompts. The benchmark covers six sphere–ground material pairs (rubber–wood, rubber–grass, stone–grass, stone–cement, steel–cement, and steel–grass) across diverse scene backgrounds. For each material–scene setting, we systematically vary the initial release height, camera-relative depth, and sphere size, together with object appearance and illumination conditions. This design provides diverse physical initial conditions and visual configurations, enabling a comprehensive evaluation of physical motion generation across varying geometric and environmental settings. For physical-state evaluation, we construct a Kubric-simulated dataset containing 18,900 rigid-body collision sequences with reference trajectories and collision events. The dataset comprises 17,010 training sequences and 1,890 validation sequences, with the latter used for physical-state assessment.

Implementation Details. All experiments are conducted on a computing server equipped with four NVIDIA A100 GPUs. Our method generates 81-frame videos at 25 fps with a resolution of 1024×512 under a fixed-camera setting. The pretrained LTX-Video backbone and trajectory conditioning module remain frozen, with optimization restricted to the lightweight physical parameter calibration components of PSG.

Comparison Methods. We consider three categories of competing methods: general-purpose image-to-video generators (CogVideoX-I2V [33], HunyuanVideo-I2V [34], Wan-I2V [35], and LTX-Video [36]), physics-grounded methods (PhysGen [2] and MotionCraft [3]), and motion-controllable methods (MotionPro [18] and WanMove [20]).

Table I summarizes the inference configurations and additional input requirements of these methods. All methods are evaluated using identical initial images and text prompts, with consistent generation settings where supported; otherwise, we follow the officially recommended configurations. Unlike methods requiring externally specified physical parameters or motion controls, our framework automatically derives physical priors from the initial image and text prompt without additional user-provided inputs. Each method generates one video per input without best-of-NN selection.

MethodResolutionFramesFPSAdditional Input
CogVideoX-I2V1024×5121024 \times5128125None
HunyuanVideo-I2V1344×6721344 \times6728125None
Wan-I2V1024×5121024 \times5128125None
LTX-Video1024×5121024 \times5128125None
PhysGen512×512512 \times5128125Physical parameters
MotionCraft512×512512 \times5128125Simulated motion
MotionPro512×512512 \times512168Trajectory + Mask
WanMove880×432880 \times4328125Trajectory
IMPACT (Ours)1024×5121024 \times5128125None (automatic)

Table I. Inference configurations and additional input requirements of the compared methods.

Evaluation Metrics. We evaluate IMPACT from three complementary perspectives: human evaluation, vision-language-model (VLM) evaluation, and objective physical evaluation. Human and VLM assessments measure the perceived physical plausibility of generated videos, while objective physical metrics quantify the accuracy of predicted state evolution against simulation references.

Human and VLM Evaluation. Both evaluations adopt five criteria: Physical Event Completeness (PEC), Free-Fall Dynamics Realism (FDR), Contact and Collision Realism (CCR), Rebound and Energy Dissipation Consistency (REC), and Overall Physical Plausibility (OPP). Each criterion is rated on a five-point scale, with higher scores indicating greater physical plausibility. Ten graduate students independently assess anonymized videos, and the reported scores are averaged across videos and raters. For VLM evaluation, 16 uniformly sampled frames from each video are assessed alongside the corresponding text and material conditions using the same criteria. We additionally report the Complete Failure Rate (CFR), defined as the proportion of evaluation units receiving the minimum score across all five criteria. Human CFR is computed over video–rater pairs, whereas VLM CFR is computed over videos.

Objective Physical Evaluation. To complement perceptual assessments, we employ five objective metrics to quantify deviations from simulation references in continuous state evolution and discrete collision dynamics.

Continuous-State Accuracy. Position Error (PE) and Velocity Error (VE) measure the average Euclidean distances between predicted and reference positions and velocities, respectively:

PE=1N∑i=1N1Ti∑t=1Ti∥p^i,t−pi,t∗∥2,\mathrm{PE} = \frac{1}{N}\sum_{i=1}^{N}\frac{1}{T_i}\sum_{t=1}^{T_i}\left\lVert\hat{\mathbf{p}}_{i,t}-\mathbf{p}^{*}_{i,t}\right\rVert_2,
VE=1N∑i=1N1Ti∑t=1Ti∥v^i,t−vi,t∗∥2,(22)\mathrm{VE} = \frac{1}{N}\sum_{i=1}^{N}\frac{1}{T_i}\sum_{t=1}^{T_i}\left\lVert\hat{\mathbf{v}}_{i,t}-\mathbf{v}^{*}_{i,t}\right\rVert_2, \tag*{(22)}

where NN denotes the number of evaluation sequences and TiT_i is the number of evaluated time steps in sequence ii. The superscripts ⋅^\hat{\cdot} and ∗* indicate predicted and reference quantities, respectively.

Collision-Event Accuracy. To evaluate discrete collision dynamics, predicted impacts are matched one-to-one with reference impacts within a predefined temporal tolerance. Impact-detection F1 measures collision-event detection accuracy:

F1=2TP2TP+FP+FN,(23)\mathrm{F1} = \frac{2\mathrm{TP}}{2\mathrm{TP}+\mathrm{FP}+\mathrm{FN}}, \tag*{(23)}

where TP, FP, and FN denote correctly matched impacts, unmatched predicted impacts, and missed reference impacts, respectively.

For matched events, Impact-Time Error (TE) and Rebound-Apex Error (AE) measure the mean absolute deviations in collision timing and maximum rebound height, respectively:

TE=1∣M∣∑(i,j,k)∈M∣τ^i,j−τi,k∗∣,\mathrm{TE} = \frac{1}{|\mathcal{M}|}\sum_{(i,j,k)\in\mathcal{M}}\left|\hat{\tau}_{i,j}-\tau^{*}_{i,k}\right|,
AE=1∣Ma∣∑(i,j,k)∈Ma∣h^i,j−hi,k∗∣,(24)\mathrm{AE} = \frac{1}{|\mathcal{M}_a|}\sum_{(i,j,k)\in\mathcal{M}_a}\left|\hat{h}_{i,j}-h^{*}_{i,k}\right|, \tag*{(24)}

where M\mathcal{M} denotes the set of matched impact pairs and Ma⊆M\mathcal{M}_a \subseteq\mathcal{M} contains those with observable rebound apexes. Here, τ\tau denotes the impact timestamp, and hh represents the maximum post-impact height of the sphere’s lowest point above the ground plane. Since TE and AE are computed only for matched events, they are interpreted jointly with F1 to account for missed and spurious collisions.

Performance Comparison

Overall Quantitative Comparison

1) Overall Quantitative Comparison: We evaluate our method against three categories of baselines: general-purpose video generators (CogVideoX-I2V, HunyuanVideo-I2V, Wan-I2V, and LTX-Video), physics-grounded methods (PhysGen and MotionCraft), and motion-controllable methods (MotionPro and WanMove).

As shown in TABLE II, our method consistently outperforms all competing approaches across the evaluation criteria while achieving the lowest complete-failure rate. Notably, the discrepancy between failure rates and individual criterion scores indicates that avoiding complete generation failures does not necessarily ensure physical consistency. These results highlight the limitations of existing approaches in maintaining coherent physical dynamics throughout video generation.

To further examine human-perceived quality, we conduct pairwise comparisons based on Overall Preference (OPP) ratings. The winning fractions in Figure 5 show that our method is consistently preferred over competing approaches. This preference-level evidence complements the aggregate evaluation scores and further demonstrates the perceptual advantages of physics-guided generation.

Winning fraction heatmap

Figure 5. Pairwise human rating win-rate heatmap based on the Overall Preference (OPP) metric. Each cell reports the percentage of non-tied comparisons in which the row method receives a higher rating than the column method. Ratings are paired by test video and evaluator, with ties excluded from the calculation. Warmer colors indicate higher winning fractions.

Comparison with General-Purpose Video Generators

2) Comparison with General-Purpose Video Generators: General-purpose video generators rely primarily on learned visual and temporal correlations, without explicitly enforcing physical state transitions. Figure 6 presents representative results for outdoor sphere–grass and indoor sphere–floor interactions.

Qualitative comparison of evaluated free-fall and bouncing sequences

Figure 6. Qualitative comparison of the evaluated free-fall and bouncing sequences. Representative frames illustrate free fall, contact, rebound attenuation, and settling.

CogVideoX-I2V exhibits visual artifacts and object disappearance, disrupting appearance consistency and motion continuity. Wan-I2V shows implausible floating and fails to settle, while HunyuanVideo-I2V produces persistent oscillations with insufficient rebound attenuation. LTX-Video exhibits motion stagnation, interrupting the progression of the bouncing process.

MethodHuman EvaluationVLM Evaluation
PEC↑FDR↑CCR↑REC↑OPP↑CFR (%)↓PEC↑FDR↑CCR↑REC↑OPP↑CFR (%)↓
General-Purpose Video Generators
CogVideoX-I2V1.60 ± 0.391.58 ± 0.511.47 ± 0.361.28 ± 0.231.36 ± 0.2537.42.491.792.271.271.9214.0
HunyuanVideo-I2V2.54 ± 1.042.18 ± 1.051.99 ± 0.791.73 ± 0.531.85 ± 0.6716.22.882.002.711.572.1610.0
Wan-I2V2.19 ± 0.581.96 ± 0.631.85 ± 0.501.70 ± 0.471.75 ± 0.4730.43.332.693.432.112.900.0
LTX-Video2.52 ± 0.841.95 ± 0.672.11 ± 0.741.83 ± 0.581.95 ± 0.5910.53.322.733.432.122.870.0
Physics-Grounded Methods
PhysGen3.55 ± 0.583.58 ± 0.712.51 ± 0.573.39 ± 0.893.06 ± 0.670.43.452.983.582.353.050.0
MotionCraft1.33 ± 0.351.26 ± 0.201.26 ± 0.301.11 ± 0.101.26 ± 0.3255.51.961.481.371.071.5414.0
Motion-Controllable Methods
MotionPro1.67 ± 0.401.72 ± 0.661.38 ± 0.381.28 ± 0.241.34 ± 0.2935.22.651.852.251.351.7516.0
WanMove3.38 ± 0.863.07 ± 1.012.88 ± 0.882.73 ± 0.802.81 ± 0.836.23.102.422.951.872.453.0
IMPACT (Ours)3.87 ± 0.613.69 ± 0.343.72 ± 0.553.47 ± 0.643.61 ± 0.490.04.253.694.163.403.580.0

Table 2. Quantitative comparison with general-purpose, physics-grounded, and motion-controllable video generation methods under both human and VLM evaluations. Human evaluation scores are reported as the mean ± standard deviation of evaluator-level mean ratings on a 1-5 scale.

In contrast, our method preserves a coherent sequence of ground impact, progressively attenuated rebounds, and eventual rest, while maintaining stable object appearance. These observations demonstrate the importance of explicitly modeling physical events rather than relying solely on learned motion patterns.

Comparison with Physics-Grounded Methods

We compare our method with PhysGen and MotionCraft to investigate whether physics-derived motion can be faithfully translated into generated video content.

PhysGen employs vision-language-based physical property estimation and user-specified motion conditions to construct image-space simulation and motion references. However, in our evaluated examples, it exhibits inaccurate contact timing and misaligned contact locations. Its generated motion often resembles the translation of a rigid two-dimensional cutout, with limited natural appearance variation and imperfect scene integration.

MotionCraft incorporates simulation-derived optical flow through latent noise warping. Nevertheless, it frequently fails to preserve the complete progression from ground impact to damped rebounds and eventual rest, occasionally introducing object duplication and visual artifacts.

As illustrated in Figure 7, our method produces more coherent sphere–ground interactions while preserving natural object appearance and scene integration. Unlike PhysGen, our framework automatically constructs motion-relevant physical states from a single image and text prompt without requiring user-specified initial forces or velocities for the considered free-fall task. Moreover, our framework incorporates physics-derived constraints directly into video diffusion rather than relying on pre-warped image sequences. This comparison highlights the importance of coupling explicit physical simulation with generation-time guidance.

Qualitative comparison with physics-grounded methods at matched timestamps

Figure 7. Qualitative comparison with physics-grounded methods at matched timestamps. PhysGen exhibits delayed ground contact and cutout-like motion, while MotionCraft suffers from object duplication and scene artifacts. In contrast, our method produces coherent ground contact and rebound dynamics while maintaining visual consistency.

MethodPE↓VE↓F1↑TE↓AE↓
Analytic Physics0.52491.43530.3720.03330.7746
PSG (Full)0.26581.16850.5300.02610.1751

Table 3. Physical-state prediction accuracy on the Kubric validation set under identical I2P initialization.

Comparison with Motion-Controllable Methods

We compare our method with MotionPro and WanMove to examine whether physically consistent motion inputs are sufficient to ensure coherent sphere–ground interactions. MotionPro employs region-wise trajectories and motion masks, while WanMove introduces latent trajectory guidance. For a controlled comparison, we convert the motion priors generated by our PSG into the respective control representations required by both methods. Thus, all methods receive motion constraints derived from the same physical state evolution.

Despite these physically grounded inputs, MotionPro often produces limited object displacement and fails to establish a complete physical event sequence. The sphere may disappear near or after ground contact, disrupting appearance continuity and rebound behavior. WanMove produces more noticeable motion but frequently exhibits nearly uniform falling speed, unphysical floating, and persistent movement without eventual settling.

As illustrated in Figure 8, our method achieves more complete physical event progression while maintaining stable object appearance. These results suggest that physically consistent trajectory inputs alone cannot guarantee faithful physical motion execution. By combining sparse trajectory conditioning with dense physics guidance, our framework more effectively translates the PSG-derived physical states into temporally coherent video dynamics.

Qualitative comparison with motion-controllable methods at matched timestamps

Figure 8. Qualitative comparison with motion-controllable methods at matched timestamps. MotionPro exhibits ground penetration and object disappearance, while WanMove suffers from visual artifacts and missing rebound behavior. In contrast, our method produces coherent ground contact, damped rebounds, and eventual rest.

Physical State Prediction Accuracy

Evaluation Protocol. To assess the physical modeling capability of PSG independently of visual estimation and video synthesis, we conduct a controlled evaluation on a Kubric-based validation set with simulation-derived ground truth. For each sequence ii, the simulator provides reference sphere-center positions and velocities {(pi,t∗,vi,t∗)}t=1Ti\{(p^{*}_{i,t}, v^{*}_{i,t})\}_{t=1}^{T_i}. Given the simulation-defined initial conditions, PSG predicts the corresponding state trajectory {(p^i,t,v^i,t)}t=1Ti\{(\hat{p}_{i,t}, \hat{v}_{i,t})\}_{t=1}^{T_i}. All trajectories are evaluated in a common physical coordinate system at identical time steps, enabling direct assessment of physical-state evolution.

To examine the effectiveness of PSG’s dynamics modeling, we compare it with an analytical physics baseline that describes free fall under constant gravitational acceleration and collisions using fixed material-dependent restitution coefficients. In contrast, PSG incorporates learned gravity calibration and impact-velocity-dependent restitution to model physical state transitions. Both methods share identical simulation-defined initial states, material configurations, ground geometry, and prediction horizons, with reference future states reserved exclusively for evaluation. By eliminating variations in initialization and scene configuration, this protocol isolates the effects of dynamics modeling and enables a controlled comparison of their ability to reproduce free-fall, impact, and rebound dynamics.

Quantitative Results. As shown in TABLE III, PSG outperforms the analytic baseline across all five metrics, demonstrating closer agreement with the Kubric simulation references. For continuous state prediction, PSG reduces position and velocity errors by 49.4% and 18.6%, respectively, indicating more accurate motion evolution. The improvements extend to discrete collision dynamics: impact-detection F1 increases from 0.372 to 0.530, while impact-time error decreases by 21.6%. Notably, PSG reduces rebound-apex error

by 77.4%, demonstrating substantially improved prediction of post-impact rebound amplitudes. These results indicate that PSG more faithfully reproduces both continuous motion and collision-induced state transitions than the fixed-restitution analytic model, providing more accurate physical state sequences for subsequent video generation.

VariantPE↓VE↓F1↑TE↓AE↓
w/o Impact Loss0.26771.60370.4770.06260.1769
Full (Oracle)0.24221.21310.5370.05290.1573
Full (I2P)0.26581.16850.5300.02610.1751

Table 4. IMPACT-SUPERVISION ABLATION AND INITIAL-STATE COMPARISON ON THE KUBRIC VALIDATION SET.

Ablation Studies

Impact Supervision and State Initialization. We further investigate the contribution of impact supervision and the influence of initial-state estimation on PSG. For the supervision ablation, the full model and the variant without Limpact\mathcal{L}_{\mathrm{impact}} share identical oracle initial states, training data, network architectures, and optimization settings, differing only in the training objective. We additionally evaluate the same full PSG using I2P-estimated initial states to examine its sensitivity to initialization.

As shown in TABLE IV, removing impact supervision degrades performance across all evaluated metrics under otherwise identical conditions. This demonstrates that impact supervision complements trajectory-level position constraints by improving the prediction of collision-induced state transitions and subsequent rebound dynamics.

When oracle initialization is replaced with I2P estimates, the full PSG maintains broadly comparable state-prediction and impact-detection performance, although individual metrics respond differently. This indicates that PSG can operate with image-estimated physical states without requiring simulator-provided initialization. Since impact-time errors are evaluated only for matched events, their interpretation should remain coupled with impact-detection F1.

Motion-Prior Construction and Sparse–Dense Guidance.

To investigate the influence of motion-prior construction, we compare several trajectory variants and guidance settings, as summarized in Table V. PSG-KF Linear and PSG-KF Smoothstep share the same PSG-derived event keyframes but employ different inter-event interpolation strategies. Naive Physics simulates free fall and repeated collisions using constant gravity and fixed material-dependent restitution coefficients. We further compare PSG + Sparse, which uses the complete PSG trajectory without dense guidance, with Ours (Full), which additionally incorporates dense physics guidance during diffusion sampling.

MethodHuman EvaluationVLM Evaluation
PEC↑FDR↑CCR↑REC↑OPP↑CFR (%)↓PEC↑FDR↑CCR↑REC↑OPP↑CFR (%)↓
Trajectory Variants
PSG-KF Linear3.46 ± 0.613.05 ± 0.403.22 ± 0.402.79 ± 0.192.95 ± 0.230.34.183.294.023.123.250.0
PSG-KF Smoothstep3.40 ± 0.583.06 ± 0.413.26 ± 0.412.74 ± 0.442.94 ± 0.350.34.243.624.133.343.510.0
Naive Physics3.33 ± 0.762.91 ± 0.533.28 ± 0.532.62 ± 0.492.86 ± 0.380.34.223.474.103.273.420.0
Sparse–Dense Guidance
LTX-Video2.52 ± 0.841.95 ± 0.672.11 ± 0.741.83 ± 0.581.95 ± 0.5910.53.322.733.432.122.870.0
PSG + Sparse2.73 ± 0.752.34 ± 0.532.55 ± 0.592.22 ± 0.422.38 ± 0.444.83.252.543.172.342.510.0
IMPACT (Ours)3.87 ± 0.613.69 ± 0.343.72 ± 0.553.47 ± 0.643.61 ± 0.490.04.253.694.163.403.580.0

Table V. Ablation of motion-prior construction and sparse–dense physics guidance under human and VLM evaluations. Human evaluation scores are reported as the mean ± standard deviation of evaluator-level mean ratings on a 1-5 scale.

The results reveal that perceptually smooth motion does not necessarily imply physically accurate state transitions, highlighting the importance of event-driven physical modeling. Moreover, sparse trajectory conditioning alone cannot fully enforce physical consistency during video synthesis, motivating the integration of dense generation-time guidance.

As illustrated in Figure 9, LTX-Video produces ground collisions but exhibits spatial deviations and physically implausible motion. Although PSG + Sparse largely follows the predicted trajectory, the generated sphere undergoes local enlargement, revealing a prediction–generation mismatch between physically valid motion predictions and their visual realization. In contrast, Ours (Full) better maintains object scale and coherent contact behavior throughout the motion sequence. These observations demonstrate that sparse trajectory control primarily constrains global displacement, whereas dense physics guidance is essential for improving local physical consistency during video generation.

Ablation study: indoor ball motion

Figure 9. Qualitative comparison of LTX-Video, PSG + Sparse, and Ours (Full) at matched timestamps. LTX-Video exhibits spatial deviations and physically inconsistent motion despite producing ground collisions. PSG + Sparse follows the predicted trajectory but suffers from local object enlargement, revealing a prediction–generation mismatch. In contrast, our full method preserves more consistent object scale and coherent collision dynamics.

Limitation and Discussion

The current framework is developed and evaluated under a controlled sphere-plane interaction setting, where a single rigid sphere is released from rest and undergoes successive collisions with a stationary plane under a fixed camera. More complex interactions, such as collisions between multiple objects, non-spherical geometries, rotational motion, and deformable-body dynamics, involve additional physical states and contact constraints that are not captured by the present formulation. Moreover, the initial physical configuration is estimated from a single image, making the predicted motion sensitive to uncertainties in depth, object geometry, and camera calibration. Extending PSG to more general physical interactions therefore requires richer state representations and more robust physical parameter estimation.

A further limitation concerns the translation of predicted physical states into generated video content. Although sparse-dense guidance improves motion realization, it does not guarantee exact agreement between the generated video and the underlying physical trajectory. The current dense priors primarily describe translational object motion and background stability, leaving fine-grained contact deformation, rotation, and complex appearance changes insufficiently constrained. In addition, sampling-time latent guidance introduces additional computational overhead and remains dependent on the representational capabilities of the pretrained video generator. These limitations suggest that physically consistent video generation requires not only accurate state prediction but also more expressive and efficient mechanisms for enforcing physical constraints during synthesis. Future work will explore richer physical representations and adaptive generation-time guidance to improve the fidelity and generality of physical motion realization.

Conclusion

In this paper, we presented IMPACT, a physics-state-driven framework for physically consistent image-to-video generation. Our findings highlight that physical consistency depends on both the coherent evolution of physical states and their faithful realization during video synthesis. Technically, IMPACT employs an event-driven Physics State Generator to connect continuous free-flight dynamics with calibrated collision responses, ensuring coherent transitions across successive physical events. It further translates the predicted physical evolution into complementary sparse trajectory and dense region-level constraints to guide motion realization during diffusion sampling. Experimental results demonstrate improved physical plausibility across different material combinations and visual environments. Notably, the ablation results reveal that trajectory guidance alone is insufficient to ensure physically consistent visual motion, highlighting the importance of region-level constraints during synthesis. These findings suggest that physically grounded video generation should move beyond plausible motion prediction toward a tighter integration of physical dynamics and generative control. Extending this principle to more complex physical interactions remains an important direction for future research.

Acknowledgment

This work was supported in part by the National Natural Science Foundation of China under Grant 62576313, the Key Research and Development Program of Zhejiang Province under Grant 2026C01016, and the Key Research and Development Program of Ningbo under Grant 2025Z128.

References

References

  1. [1]L. Lu, X. Ma, W. He, G. Zhan, Y. Zhao, J. Chen, M. Xu, J. Fan, W. Ge, Y. Gu, Y. Liu, B. Li, Z. Dong, V. Prisacariu, M.-Y. Liu, S. Han, and H. Cai, “Physys-Lang: Self-evolving language as a physical representation for video world model,” arXiv:2609.40358, 2026. 1arxiv.org/abs/2609.40358
  2. [2]S. Liu, Z. Ren, S. Gupta, and S. Wang, “PhysGen: Rigid-body physics-grounded image-to-video generation,” in European Conference on Computer Vision, 2024, pp. 360–378. 1, 7arxiv.org/abs/2409.18964
  3. [3]A. Montanaro, L. Savant Aira, E. Aiello, D. Valsesia, and E. Magli, “MotionCraft: Physics-based zero-shot video generation,” Advances in Neural Information Processing Systems, pp. 123 155–123 181, 2024. 1, 7
  4. [4]B. Chen, H. Jiang, S. Liu, S. Gupta, Y. Li, H. Zhao, and S. Wang, “PhysGen3D: Crafting a miniature interactive world from a single image,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2025, pp. 6178–6189. 2arxiv.org/abs/2503.20746
  5. [5]T. Huang, H. Zhang, Y. Zeng, Z. Zhang, H. Li, W. Zuo, and R. W. H. Lau, “DreamPhysics: Learning physics-based 3D dynamics with video diffusion priors,” in Proceedings of the AAAI Conference on Artificial Intelligence, 2025, pp. 3733–3741. 2arxiv.org/abs/2406.01476
  6. [6]L. G. Foo, M. H. Huang, A. Lattas, S. Moschoglou, T. Beeler, and C. Theobalt, “Physical simulator in-the-loop video generation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2026, pp. 4301–4311. 2arxiv.org/abs/2603.06408
  7. [7]Y. Yuan, X. Wang, T. Wickremasinghe, Z. Nadir, B. Ma, and S. H. Chan, “NewtonGen: Physics-consistent and controllable text-to-video generation via neural Newtonian dynamics,” in International Conference on Learning Representations, 2026. 2DOI
  8. [8]S. N. K. Satish, D. Jaiswal, H. Chen, and A. Bakshi, “PhysVideoGenerator: Towards physically aware video generation via latent physics guidance,” arXiv:2601.03665, 2026. 2arxiv.org/abs/2601.03665
  9. [9]Q. Cao, X. Li, D. Wang, C. Ma, Y. Chen, and X. Yang, “Latent knowledge-guided video diffusion for scientific phenomena generation from a single initial frame,” in Proceedings of the AAAI Conference on Artificial Intelligence, 2026, pp. 2625–2633. 2
  10. [10]X. Yang, B. Li, Y. Zhang, Z. Yin, L. Bai, Ma, Z. Wang, J. Cai, T.-T. Wong, H. Lu, and X. Jia, “VLIPP: Towards physically plausible video generation with vision and language informed physical prior,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2025, pp. 12 360–12 370. 2arxiv.org/abs/2503.23368
  11. [11]K. Zhang, C. Xiao, J. Xu, Y. Mei, and V. M. Patel, “Think before you diffuse: Infusing physical rules into video diffusion,” arXiv:2505.21653, 2025. 2arxiv.org/abs/2505.21653
  12. [12]C. Wang, C. Chen, Y. Huang, Z. Dou, Y. Liu, J. Gu, and L. Liu, “PhysCtrl: Generative physics for controllable and physics-grounded video generation,” Advances in Neural Information Processing Systems, pp. 186 107–186 132, 2025. 2arxiv.org/abs/2509.20358
  13. [13]Y. Feng, J. Wang, C. Xu, Y. Qian, H. Wang, W. Hou, Y. Liu, B. Sun, Y. Liu, and S. Wang, “NEWTON: Agentic planning for physically grounded video generation,” arXiv:2605.18396, 2026. 2arxiv.org/abs/2605.18396
  14. [14]S. Narayanan, Z. Jiang, S. Narasimhan, and M. Chandraker, “PhyCo: Learning controllable physical priors for generative motion,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2026, pp. 41 892–41 902. 2arxiv.org/abs/2604.28169
  15. [15]Q. Zhang, B. Gong, S. Tan, Z. Zhang, Y. Shen, X. Zhu, Y. Li, K. Yao, C. Shen, and C. Zou, “PhysRVG: Physics-aware unified reinforcement learning for video generative models,” in European Conference on Computer Vision, 2026, pp. 149–166. 2arxiv.org/abs/2601.11087
  16. [16]H. Zhou, C. Wang, R. Nie, J. Liu, D. Yu, Q. Yu, and C. Wang, “TrackGo: A flexible and efficient method for controllable video generation,” in Proceedings of the AAAI Conference on Artificial Intelligence, 2025, pp. 10743–10751. 3arxiv.org/abs/2408.11475
  17. [17]D. Geng, C. Herrmann, J. Hur, F. Cole, S. Zhang, T. Pfaff, T. Lopez-Guevara, Y. Aytar, M. Rubinstein, C. Sun, O. Wang, A. Owens, and D. Sun, “Motion prompting: Controlling video generation with motion trajectories,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2025, pp. 1–12. 3arxiv.org/abs/2412.02700
  18. [18]Z. Zhang, F. Long, Z. Qiu, Y. Pan, W. Liu, T. Yao, and T. Mei, “MotionPro: A precise motion controller for image-to-video generation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2025, pp. 27 957–27 967. 3, 7arxiv.org/abs/2505.20287
  19. [19]Z. Zhang, J. Liao, M. Li, Z. Dai, B. Qiu, S. Zhu, L. Qin, and W. Wang, “Tora: Trajectory-oriented diffusion transformer for video generation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2025, pp. 2063–2073. 3arxiv.org/abs/2407.21705
  20. [20]R. Chu, Y. He, Z. Chen, S. Zhang, X. Xu, B. Xia, D. Wang, H. Yi, X. Liu, H. Zhao, Y. Liu, Y. Zhang, and Y. Yang, “Wan-Move: Motion-controllable video generation via latent trajectory guidance,” Advances in Neural Information Processing Systems, pp. 448–476, 2025. 3, 7arxiv.org/abs/2512.08765
  21. [21]X. Shi, Z. Huang, F.-Y. Wang, W. Bian, D. Li, Y. Zhang, M. Zhang, K. C. Cheung, S. See, H. Qin, J. Dai, and H. Li, “Motion-I2V: Consistent and controllable image-to-video generation with explicit motion modeling,” in ACM SIGGRAPH Conference Papers, 2024, pp. 1–11. 3arxiv.org/abs/2401.15977
  22. [22]R. Burgert, Y. Xu, W. Xian, O. Pilarski, P. Clausen, M. He, L. Ma, Y. Deng, L. Li, M. Mousavi, M. Ryoo, P. Debevec, and N. Yu, “Go-with-the-Flow: Motion-controllable video diffusion models using real-time warped noise,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2025, pp. 13–23. 3arxiv.org/abs/2501.08331
  23. [23]W. Jin, Q. Dai, C. Luo, S.-H. Baek, and S. Cho, “FloVD: Optical flow meets video diffusion model for enhanced camera-controlled video synthesis,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2025, pp. 2040–2049. 3arxiv.org/abs/2502.08244
  24. [24]Q. Li, Z. Xing, R. Wang, H. Zhang, Q. Dai, and Z. Wu, “MagicMotion: Controllable video generation with dense-to-sparse trajectory guidance,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2025, pp. 12 112–12 123. 3arxiv.org/abs/2503.16421
  25. [25]H. Chung, J. Kim, M. T. McCann, M. L. Klasky, and J. C. Ye, “Diffusion posterior sampling for general noisy inverse problems,” in International Conference on Learning Representations, 2023. 3, 6arxiv.org/abs/2209.14687
  26. [26]J. Yu, Y. Wang, C. Zhao, B. Ghamen, and J. Zhang, “FreeDoM: Training-free energy-guided conditional diffusion model,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 23 174–23 184. 3arxiv.org/abs/2303.09833
  27. [27]H. Ye, H. Lin, J. Han, M. Xu, S. Liu, Y. Liang, J. Ma, J. Zou, and S. Ermon, “TFG: Unified training-free guidance for diffusion models,” Advances in Neural Information Processing Systems, pp. 22 370–22 417, 2024. 3DOI
  28. [28]C. Chen, J. Shu, G. He, C. Wang, and Y. Li, “Motion-Zero: A zero-shot trajectory control framework of moving object for diffusion-based video generation,” in Proceedings of the AAAI Conference on Artificial Intelligence, 2025, pp. 2016–2024. 3
  29. [29]Z. Xiao, Y. Zhou, S. Yang, and X. Pan, “Video diffusion models are training-free motion interpreter and controller,” Advances in Neural Information Processing Systems, pp. 76 115–76 138, 2024. 3arxiv.org/abs/2405.14864
  30. [30]T. H. S. Meral, H. Yesiltepe, C. Dunlop, and P. Yanardag, “Motion-Flow: Attention-driven motion transfer in video diffusion models,” in Proceedings of the AAAI Conference on Artificial Intelligence, 2026, pp. 8043–8051. 3
  31. [31]A. Pondaven, A. Siarohin, S. Tulyakov, P. Torr, and F. Pizzati, “Video motion transfer with diffusion transformers,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2025, pp. 22 911–22 921. 3arxiv.org/abs/2412.07776
  32. [32]A. Bansal, H.-M. Chu, A. Schwarzschild, R. Sengupta, M. Goldblum, J. Geiping, and T. Goldstein, “Universal guidance for diffusion models,” in International Conference on Learning Representations, 2024. 6arxiv.org/abs/2302.07121
  33. [33]Z. Yang, J. Teng, W. Zheng, M. Ding, S. Huang, J. Xu, Y. Yang, W. Hong, X. Zhang, G. Feng, D. Yin, Y. Zhang, W. Wang, Y. Cheng, X. Bin, X. Gu, Y. Dong, and J. Tang, “CogVideoX: Text-to-video diffusion models with an expert transformer,” in International Conference on Learning Representations, 2025. 7arxiv.org/abs/2408.06072
  34. [34]W. Kong, Q. Tian, Z. Zhang, R. Min, Z. Dai, J. Zhou, J. Xiong, X. Li, B. Wu, J. Zhang et al., “HunyuanVideo: A systematic framework for large video generative models,” arXiv:2412.03603, 2024. 7
  35. [35]Team Wan et al., “Wan: Open and advanced large-scale video generative models,” arXiv:2503.20314, 2025. 7arxiv.org/abs/2503.20314
  36. [36]Y. HaCohen, N. Chiprut, B. Brazowski, D. Shalem, D. Moshe, E. Richardson, E. Levin, G. Shiran, N. Zabari, O. Gordon, P. Panet, S. Weissbuch, V. Kulikov, Y. Bitterman, Z. Melumian, and O. Bibi, “LTX-Video: Realtime video latent diffusion,” arXiv:2501.00103, 2025. 7arxiv.org/abs/2501.00103

Paper details

vlnc:W_3aypr5jg
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents