WorksComputer Science
See the Lives of Stars in a Handful of Light
Abstract
Galaxies evolve over hundreds of millions of years, and no telescope has ever watched one move. Cosmological simulations make this evolution visible as a sequence of multi-view images, inviting the question of whether a galaxy can be recovered in four dimensions from its images. This setting defeats existing dynamic scene methods, since millions of stars span only a few pixels each, faint populations blend into diffuse light, and free deformations reproduce the images without obeying gravity. We present Cosmos4D, which rests on two components. Gravitational 4D Gaussian Splatting represents each star as a self-luminous Gaussian whose motion is integrated through a gravitational potential and a low-dimensional force field shared by all stars. Luminosity-stratified dynamics give bright stars individual orbits supervised across image scales and carry faint populations with a velocity field estimated from optical flow. We further build a 4D dataset of eight galaxies from four cosmological simulations, on which Cosmos4D outperforms dynamic Gaussian baselines in both photometric quality and physical accuracy. With these advancements, we glimpse eight hundred million years of a galaxy's life in ninety-six frames.
Figure 1. Cosmos4D recovers a galaxy in four dimensions from a handful of multi-view images. Across eight hundred million years, 4DGS [55] (top) blurs the stars into a smooth glow, whereas Cosmos4D (bottom) preserves the star-level texture of the spiral arms.
Introduction
To see a World in a Grain of Sand, Hold Infinity in the palm of your hand. — William Blake, Auguries of Innocence
A human life lasts less than a century, while the Sun, moving at about , needs roughly 200 million years to orbit the Milky Way once [2]. A disk star in Andromeda, 2.5 million light-years away [25], moves at a similar speed [7], yet its image shifts by only a seventh of a Hubble pixel per century, which only years of space astrometry or radio interferometry can measure [4, 49]. We know that stars sweep along their orbits [1], that new stars ignite and older populations redden and fade, yet every image of a galaxy is a single snapshot of this evolution.
Simulations can supply the frames that observations lack. Cosmological simulations such as TNG50 [40, 44] follow the stars of a galaxy through cosmic time, allowing it to be rendered from any viewpoint at any moment of its history. The evolution of a galaxy thus becomes a sequence of multi-view images, and a natural question arises. Can a galaxy be recovered in four dimensions from its images alone? The question also poses a demanding test for dynamic reconstruction, because the scene obeys known physical laws while its motion must still be inferred from light.
Such a scene differs in several ways from those that dynamic radiance fields [32, 45, 55] were designed for. A galaxy is optically thin and self-luminous, so its hundred thousand point sources add together instead of occluding one another, leaving no surfaces, silhouettes, or depth order to anchor correspondence. Its appearance also changes for reasons unrelated to motion, as stars are born during the sequence and stellar populations brighten, redden, and fade with age. Most importantly, the sequence is long compared with the dynamics. Ninety-six frames span 0.8 Gyr, several rotations of the inner disk, whereas the three-dimensional positions of stars are known at only a single instant. A free-form deformation field [41, 45, 55] can then fit every frame with motion that no galaxy could follow. Per-star trajectories [32] drift away, since a star spanning only a few pixels receives no image gradient once it moves off its target.
The problem becomes tractable once we recall that stars do not move independently. They fall together through a shared gravitational potential [16, 36, 38], which is shaped by the very distribution of mass observed at the known instant [1]. Their motion is therefore governed by an equation of motion rather than by an arbitrary function of space and time, and the trajectories it permits form a far smaller family than the deformations a network can express. Gravity should nevertheless guide the reconstruction without demanding more precision than images can offer. Over hundreds of millions of years the identity of an individual star cannot be recovered from images, and faint populations leave almost no trace of their own motion.
Guided by these observations, we present Cosmos4D, which represents an evolving galaxy as a set of self-luminous Gaussians [20] driven by gravity. Each star becomes a Gaussian rendered by additive splatting, with its changing emission separated from its motion. Its position follows a differentiable orbit integrator [8] through a potential inferred from the known instant, corrected by a low-dimensional force field shared by all stars. Because images constrain stars unequally, bright stars receive individual orbital states, while faint stars, which stand for unresolved populations, are carried by a shared velocity field estimated from multi-view optical flow. Because a bright star renders as a point source only a few pixels wide, we also supervise it across an image pyramid, which keeps guiding it toward its target after it drifts beyond its own footprint. To evaluate this setting, we build a dataset of eight evolving galaxies from four cosmological simulations [40, 44] and measure both the photometric quality of the renderings and the physical accuracy of the recovered motion.
To summarize, our contributions are as follows.
We propose Cosmos4D, a Gravitational 4D Gaussian Splatting framework that models each star as a self-luminous Gaussian whose motion is integrated through a gravitational potential and a low-dimensional force field shared by all stars.
We introduce luminosity-stratified dynamics, which give bright stars individual orbits supervised across an image pyramid and carry faint populations with a velocity field estimated from images.
We build a multi-view dataset of eight evolving galaxies from four cosmological simulations, on which Cosmos4D outperforms dynamic Gaussian baselines in both rendering fidelity and physical accuracy.
Related Work
Dynamic Scene Representations
Building on NeRF [35], canonical deformation [41, 45] and higher-dimensional embeddings [42] accommodate non-rigid motion and topology changes. Factorized fields [6, 13] improve spatiotemporal efficiency, while 3D Gaussian Splatting [20] enables fast explicit rendering. Its dynamic extensions encode temporal correspondence through persistent particles [32], shared deformation fields [55, 58], or parametric spacetime primitives [28], trading motion flexibility against compactness. This progression replaces independent frame reconstruction with temporal parameter sharing, enabling coherent interpolation while limiting the cost of representing long sequences.
Sparse controls and motion scaffolds [18, 23] constrain neighboring trajectories, and recent motion bases [52, 61, 64] further structure reconstruction across time and space. These advances improve temporal coherence, but their geometric and kinematic priors do not specify stellar accelerations or population luminosity evolution. In optically thin stellar systems, changing emission and unequal motion evidence across populations require additional structure. Cosmos4D couples evolving emission with gravitational orbits for bright stars and a shared flow for faint stars.
Physics-Informed Reconstruction
Neural ODEs [8] learn continuous dynamics, while Hamiltonian models [15], equation residuals [47], and differentiable simulators [17] incorporate physical structure. Coupled with rendering, these ideas support fluid reconstruction [9, 10], deformable system identification [27, 46], and spring-mass estimation [19, 63]. Shared simulation–rendering primitives [57] additionally connect reconstructed Gaussian assets to material-based animation. In inverse formulations, physical structure constrains hidden quantities that images alone leave ambiguous, including geometry and motion.
Complementary approaches infer simulation parameters from video-generation priors [31, 60] or multimodal material cues [62], while recent physical Gaussian predic tion [34] and learned force fields [26] extend this direction toward efficient dynamics synthesis. These methods primarily address material response and object interactions. Our task instead reconstructs stellar motion from images and a single positional anchor, requiring gravitational acceleration coupled to evolving emission. We allocate orbital states according to population luminosity and supervise point-like stars across image scales so that long trajectories remain optimizable.
Method
Cosmos4D reconstructs an evolving galaxy from multi-view images, representing each star as a self-luminous Gaussian that moves along a gravitational orbit. We first introduce Gravitational 4D Gaussian Splatting (§ 3.1), then Luminosity-Stratified Dynamics (§ 3.2), which assigns orbital freedom according to stellar brightness, and finally the training objective (Sec. 3.3). Figure 2 gives an overview.
Figure 2. Overview of Cosmos4D. (a) Anchor stars are split by luminosity into young bright stars and old faint stars , each a self-luminous Gaussian with evolving emission. (b) Each bright star follows its own orbit, integrated backward under the fixed potential and a shared learned force . (c) Faint stars circle their rings at the speed measured from optical flow, refined by a shared spin and an offset . (d) Stars add their light rather than occlude, and average pooling lets even drifted stars still overlap their observed targets.
Problem Formulation. Given calibrated images from cameras at snapshot times , we aim to reconstruct the evolving appearance of a galaxy together with physically plausible orbits for its stars. Since a real galaxy can be seen at just one moment of its history, we assume that its stars are known only at the latest snapshot . This snapshot, which we call the anchor, plays the role of the present day. It provides the position , age, metallicity, and luminosity of each star , as well as the total stellar mass . Velocities, individual masses, and all earlier configurations remain unknown. The whole scene evolves in cosmic time (Gyr). We also write for the normalized time, which runs from 0 at the first snapshot to 1 at the anchor.
Gravitational 4D Gaussian Splatting
Optically Thin Rendering. We represent each sampled star by one isotropic Gaussian with a time-dependent center , a learned radius, an opacity , and a view-independent RGB emission . Isotropy keeps each star a point source and prevents elongated splats from absorbing motion errors. The number of Gaussians is fixed, with no densification or pruning.
We treat the galaxy as optically thin and free of dust, so stars add their light rather than occlude one another. The linear RGB image rendered for camera is thus a plain sum,
where is a pixel location, , and are the projected mean and covariance, and the birth mask hides each star until it forms. A custom differentiable rasterizer evaluates this sum and passes image gradients to both appearance and positions.
Stellar surface brightness drops by several orders of magnitude from the galactic center to the outer disk [12]. We therefore pass the linear image of Eq. (1) through the arcsinh tone mapping commonly used in astronomical imaging [33], which stays linear for faint light and compresses the bright cores. This gives the rendered image , which is optimized to match the observation . Appendix C details the tone mapping.
Evolving Emission. Stars change in brightness and color as they age. Young stars are blue and fade within tens of millions of years, while old stars are red and change slowly. To separate this evolution from stellar motion, we write the emission of each star as a product of three terms,
where is element-wise multiplication over the RGB channels and is the normalized frame time. The first term is the learned RGB emission of star at the anchor, initialized from its age, metallicity, and luminosity. The second term is a fixed per-channel gain that scales the anchor emission back in time. We compute from a simple analytic stellar population model, in which a star fades as a power law of its age and reddens as it grows older. Young stars thus appear brighter and bluer in early frames. The third term is a learned correction for what the analytic model misses, where is a per-star RGB curve over the normalized time. The bounds this correction, so image changes caused by stellar motion are left to the dynamics.
Gravitational Dynamics. Stars in a galaxy move under the combined gravity of stars, gas, and dark matter [1]. Instead of learning a free displacement per star and frame, we let each star follow an orbit whose acceleration depends on where the star is,
where is the acceleration of star , and positions are expressed in a frame centered on the galaxy and aligned with its disk. The acceleration combines the pull of a fixed gravitational potential with a learned residual force . The potential acts as an energy landscape whose downhill direction points toward the attracting mass. We build from the anchor stars and as a standard analytic model of a central bulge, a flat disk, and a dark matter halo [16, 36, 38]. This smooth model is axisymmetric, so it cannot capture bars, spiral arms, or deviations from the assumed mass distribution [48]. We therefore learn the residual force to account for such unmodeled forces,
where each is a predefined smooth force field and is the learned time-varying coefficient of that field. All stars share these coefficients, so they follow one common force rather than per-star offsets. To obtain the trajectory of a star, we start from its orbital state at the anchor, namely the position and velocity , and integrate the acceleration of Eq. (3) twice backward in time with a differentiable leapfrog scheme [1]. The anchor position is known, and the velocity is learned from the images. Image losses pass gradients through every integration step to the orbital states and the force coefficients. Appendix D details the potential, force fields, and integrator.
Luminosity-Stratified Dynamics
Representing every star of a galaxy as a Gaussian is computationally prohibitive, so we model the galaxy with an importance sample of its stars, drawn with probability proportional to luminosity. At the anchor, we stratify the sampled stars by luminosity into a bright set and a faint set . Every bright star is kept and modeled individually, whereas each faint star stands in for several similar neighbors. As shown in Figure 3, each bright star appears as a distinct spot, while faint stars blend into a smooth glow. We therefore assign each bright star its own gravitational orbit, while faint stars share one flow field estimated from the images. Both groups keep the evolving emission of Eq. (2).
Figure 3. Luminosity stratification. Bright stars are distinct points along the spiral arms, while faint stars blend into a glow.
Bright-Star Orbits. Stars in follow the gravitational dynamics of Eq. (3). Their velocities are initialized to the circular speed of the anchor potential at their anchor radius , and are then refined during training. Each bright star is labeled as a disk or halo star from its height above the disk plane. Disk stars start on circular orbits around the disk axis, matching the rotation of a disk galaxy, while halo stars start on circular orbits in randomly oriented planes.

Figure 4. Qualitative comparison on test views, with the boxed region magnified at the bottom-right of each tile. Baselines blur the stellar disk or break it into streaks, whereas Cosmos4D recovers star-level texture, young blue stars, and tidal features seen in GT.
Faint-Star Flow. Although individual faint stars cannot be tracked, their combined glow shows how the disk rotates. We therefore fit a shared velocity field for to the optical flow of the training views. Image patterns such as spiral arms can travel at a speed different from that of their stars, so we keep only the rotation speed at each radius . Each faint star circles the disk axis at its anchor radius and height . In disk-centered cylindrical coordinates , we integrate
backward in time from the anchor, which gives the flow position . The radial and vertical profiles of the faint population thus stay exactly as observed at the anchor. We refine this pure rotation with two bounded corrections,
where rotates the whole faint population about the disk axis by a shared angle , and each star is further moved by a small offset. This offset comes from a deformation network with parameters , which follows 4DGS [55] and combines a HexPlane encoding [6] with a small MLP. The network takes the anchor position of a star and the normalized time as input and predicts a bounded 3D offset, so a faint star moves only a limited distance away from its flow position. The factor makes the offset vanish at the anchor, where the stars are observed. Details of the flow estimation are given in Appendix § 12.
Optimization
Multi-Scale Supervision. Each bright star projects to a compact footprint of only a few pixels. When its position error exceeds this extent, the rendered star no longer overlaps its observed counterpart, and the per-pixel loss provides no gradient toward the correct position. We therefore com@page 5/18 starts_mid=0 ends_mid=1

Figure 4 (PDF p. 5). Qualitative comparison on test views, with the boxed region magnified at the bottom-right of each tile. Baselines blur the stellar disk or break it into streaks, whereas Cosmos4D recovers star-level texture, young blue stars, and tidal features seen in GT.
Faint-Star Flow. Although individual faint stars cannot be tracked, their combined glow shows how the disk rotates. We therefore fit a shared velocity field for to the optical flow of the training views. Image patterns such as spiral arms can travel at a speed different from that of their stars, so we keep only the rotation speed at each radius . Each faint star circles the disk axis at its anchor radius and height . In disk-centered cylindrical coordinates , we integrate
backward in time from the anchor, which gives the flow position . The radial and vertical profiles of the faint population thus stay exactly as observed at the anchor. We refine this pure rotation with two bounded corrections,
where rotates the whole faint population about the disk axis by a shared angle , and each star is further moved by a small offset. This offset comes from a deformation network with parameters , which follows 4DGS [55] and combines a HexPlane encoding [6] with a small MLP. The network takes the anchor position of a star and the normalized time as input and predicts a bounded 3D offset, so a faint star moves only a limited distance away from its flow position. The factor makes the offset vanish at the anchor, where the stars are observed. Details of the flow estimation are given in Appendix § 12.
Optimization
Multi-Scale Supervision. Each bright star projects to a compact footprint of only a few pixels. When its position error exceeds this extent, the rendered star no longer overlaps its observed counterpart, and the per-pixel loss provides no gradient toward the correct position. We therefore compare the rendered image and the observation over an image pyramid,
where average-pools an image to pixels, and covers the full image resolution and three coarse levels. The distance averages the RGB error over pixels with a pixel weight that grows with the observed brightness , so the galaxy is not outweighed by the dark sky around it. Pooling enlarges the footprint of each star relative to the pixel grid, which widens the basin of convergence so that misaligned stars still receive informative gradients. The full-resolution level then refines the positions of these stars.
Overall Loss. We jointly optimize the appearance of all stars, the orbital states of bright stars, the shared force coefficients, and the faint-star corrections by minimizing
where is the multi-scale image loss of Eq. (9), and weights the structural loss [54], which compares local patterns such as spiral arms and star-forming clumps. The regularizer
acts on the force coefficients of Eq. (4) and the luminosity correction of Eq. (2). The squared norm keeps each correction small, and the squared difference between neighboring time samples keeps it smooth, with weights and . Details are given in Appendix § 13.
Experiments
Implementation Details. Training has two stages. An anchor-only stage of 300 iterations fits color, opacity, and splat radii with every star held at its anchor position, after which the radii are frozen. Joint optimization of dynamics and appearance then runs for 32,000 Adam iterations with batch size one, with a 400-iteration linear warmup followed by cosine decay. We set . Further settings are given in Appendix § 13.
Dataset. We build eight evolving galaxies from the sub-boxes of the TNG50 [40, 44], TNG100 and TNG300 [39, 43, 50], and Illustris [14, 51] cosmological simulations, using the last 96 snapshots of each sequence. Stellar particles are importance-sampled by luminosity with identities fixed across frames and rendered as optically thin emitters into 14 cameras on a sphere at resolution. Each scene has 11 training and 3 test views, detailed in Appendix § 8.
Baselines & Metrics. We compare Cosmos4D with 4DGS [55], FreeGave [24], MoDec-GS [22], 4C4D [65], and RetimeGS [53] on all eight scenes and evaluate novel-view rendering with PSNR, SSIM [54], and LPIPS [59]. Each metric is computed on the full image and within the galaxy foreground mask, which keeps pixels whose ground-truth intensity exceeds 0.02. Because dark sky dominates the frame, the masked scores better reflect the quality of the galaxy itself. PSNR and SSIM are known to favor blurred over sharp predictions [3], a bias amplified for stars a few pixels wide, since a slightly misplaced star is penalized at both its true and its predicted location. LPIPS compares deep features and better reflects resolved stars. Best results are shaded as first, second, and third.
Photometric Evaluation
Table 1 summarizes rendering quality. Cosmos4D attains the best average LPIPS in both the masked and full-image settings and the best masked PSNR and SSIM, and the margin is widest in LPIPS, where our masked average of 0.392 is 39% lower than the 0.640 of 4C4D and at least 51% lower than the deformation-based methods. Cosmos4D ranks first in LPIPS on seven of eight scenes, except the dense G02, where splats spanning several pixels blur the core, and leads masked PSNR on G01, G02, and C001. On C010, C005, whose faint particles follow the shared flow instead of orbits, C014, and the sparse TNG300 scenes, baselines reach higher PSNR through smooth predictions that per-pixel metrics favor, whereas Cosmos4D resolves the stars.
| Methods | Metrics | G01 Mask | G01 Full | G02 Mask | G02 Full | C001 Mask | C001 Full | C010 Mask | C010 Full | C005 Mask | C005 Full | C014 Mask | C014 Full | T300C010 Mask | T300C010 Full | T300C021 Mask | T300C021 Full | Avg. Mask | Avg. Full |
| 4DGS [55] | PSNR↑ | 19.10 | 25.15 | 20.27 | 25.42 | 20.41 | 26.00 | 24.42 | 29.45 | 23.59 | 29.77 | 19.91 | 25.47 | 17.15 | 24.48 | 15.03 | 23.10 | 19.99 | 26.11 |
| SSIM↑ | 0.302 | 0.670 | 0.348 | 0.632 | 0.286 | 0.603 | 0.391 | 0.610 | 0.458 | 0.654 | 0.322 | 0.627 | 0.149 | 0.554 | 0.110 | 0.643 | 0.296 | 0.624 | |
| LPIPS↓ | 0.896 | 0.576 | 0.857 | 0.561 | 0.870 | 0.594 | 0.763 | 0.590 | 0.671 | 0.489 | 0.812 | 0.575 | 0.792 | 0.679 | 0.805 | 0.690 | 0.808 | 0.594 | |
| FreeGave [24] | PSNR↑ | 19.99 | 26.00 | 20.75 | 25.88 | 19.77 | 25.21 | 24.29 | 29.40 | 23.91 | 30.14 | 19.86 | 25.45 | 17.08 | 24.38 | 15.62 | 24.80 | 20.16 | 26.41 |
| SSIM↑ | 0.320 | 0.685 | 0.362 | 0.660 | 0.283 | 0.622 | 0.405 | 0.643 | 0.493 | 0.693 | 0.349 | 0.659 | 0.168 | 0.556 | 0.115 | 0.645 | 0.312 | 0.645 | |
| LPIPS↓ | 0.926 | 0.586 | 0.866 | 0.567 | 0.886 | 0.600 | 0.755 | 0.586 | 0.659 | 0.484 | 0.820 | 0.578 | 0.818 | 0.686 | 0.820 | 0.695 | 0.819 | 0.598 | |
| MoDec-GS [22] | PSNR↑ | 20.01 | 26.03 | 21.10 | 26.24 | 19.83 | 25.41 | 22.86 | 28.13 | 23.92 | 30.20 | 20.26 | 25.85 | 17.16 | 24.49 | 15.23 | 24.43 | 20.05 | 26.35 |
| SSIM↑ | 0.324 | 0.686 | 0.380 | 0.668 | 0.282 | 0.610 | 0.400 | 0.632 | 0.481 | 0.675 | 0.366 | 0.666 | 0.169 | 0.560 | 0.104 | 0.643 | 0.313 | 0.643 | |
| LPIPS↓ | 0.887 | 0.572 | 0.844 | 0.557 | 0.868 | 0.593 | 0.761 | 0.588 | 0.660 | 0.485 | 0.797 | 0.570 | 0.804 | 0.683 | 0.807 | 0.687 | 0.804 | 0.592 | |
| 4C4D [65] | PSNR↑ | 19.70 | 25.74 | 21.87 | 27.26 | 20.24 | 25.75 | 23.46 | 28.82 | 23.58 | 28.75 | 20.29 | 25.85 | 17.17 | 24.46 | 15.78 | 24.96 | 20.26 | 26.45 |
| SSIM↑ | 0.391 | 0.705 | 0.402 | 0.679 | 0.308 | 0.611 | 0.408 | 0.633 | 0.480 | 0.678 | 0.386 | 0.662 | 0.175 | 0.557 | 0.120 | 0.644 | 0.334 | 0.646 | |
| LPIPS↓ | 0.486 | 0.445 | 0.602 | 0.474 | 0.652 | 0.509 | 0.696 | 0.563 | 0.641 | 0.477 | 0.593 | 0.488 | 0.677 | 0.648 | 0.769 | 0.682 | 0.640 | 0.536 | |
| RetimeGS [53] | PSNR↑ | 20.12 | 26.14 | 20.89 | 26.02 | 20.27 | 25.86 | 23.82 | 29.09 | 24.04 | 29.34 | 20.29 | 25.89 | 16.50 | 23.91 | 14.62 | 23.85 | 20.07 | 26.26 |
| SSIM↑ | 0.382 | 0.714 | 0.405 | 0.694 | 0.343 | 0.638 | 0.429 | 0.647 | 0.498 | 0.691 | 0.389 | 0.661 | 0.130 | 0.549 | 0.091 | 0.638 | 0.333 | 0.654 | |
| LPIPS↓ | 0.656 | 0.482 | 0.618 | 0.455 | 0.669 | 0.515 | 0.597 | 0.527 | 0.538 | 0.442 | 0.613 | 0.500 | 0.729 | 0.666 | 0.740 | 0.681 | 0.645 | 0.534 | |
| Ours | PSNR↑ | 20.44 | 26.45 | 22.40 | 27.45 | 20.84 | 26.38 | 23.29 | 28.55 | 23.53 | 28.83 | 20.20 | 25.52 | 17.01 | 23.74 | 15.38 | 23.54 | 20.39 | 26.31 |
| SSIM↑ | 0.387 | 0.712 | 0.409 | 0.683 | 0.377 | 0.663 | 0.404 | 0.644 | 0.416 | 0.655 | 0.400 | 0.673 | 0.183 | 0.561 | 0.142 | 0.629 | 0.340 | 0.635 | |
| LPIPS↓ | 0.363 | 0.330 | 0.705 | 0.487 | 0.476 | 0.377 | 0.439 | 0.418 | 0.230 | 0.262 | 0.279 | 0.276 | 0.330 | 0.387 | 0.316 | 0.381 | 0.392 | 0.365 |
Table 1. Quantitative comparison of novel-view rendering on all eight scenes. For each scene, Mask reports metrics within the galaxy foreground mask and Full reports metrics on the whole image. All metrics are averaged over three test views and 96 frames at resolution. Darker cells mark the best, second best, and third best results.
Fig. 4 illustrates why LPIPS is more informative. 4DGS, FreeGave, and MoDec-GS render each galaxy as a smooth glow without resolved stars, while 4C4D and RetimeGS stretch their Gaussians into streaks along the motion, as in the outskirts of C001 and C005. Cosmos4D keeps stars compact across the disk, preserves the young blue population of C014, and reproduces the tidal feature in T300C010 that the other methods miss or smear.
Physical Analysis
Protocol. Even LPIPS assesses each view independently, and plausible images may arise from primitives that fade
and reappear rather than orbit. We therefore export the 3D Gaussian centers and luminosity weights of each method at all 96 frames with its own rendering code, express them in a disk frame fixed by the anchor snapshot, and compare them with the ground-truth particles, which are used only for evaluation. Velocities come from central differences, and all errors use the same luminosity weighting.
Rotation Curves. Table 2 reports the relative error of the luminosity-weighted rotation curve over 12 radial bins. 4DGS, FreeGave, and MoDec-GS reach 0.85 to 1.00, so their Gaussians barely orbit and the apparent rotation comes from changing color and opacity. Cosmos4D lowers the average error to 0.357, 43% below the best baseline, and ranks first on six scenes, although our TNG300 disks spin 1.5 to 2.3 × too fast and C005 and G02 too slowly.
| Scene | 4DGS | FreeGave | MoDec-GS | 4C4D | RetimeGS | Ours |
| G01 | 0.984 | 0.921 | 0.960 | 0.550 | 0.468 | 0.121 |
| G02 | 0.987 | 0.962 | 0.969 | 0.784 | 0.512 | 0.522 |
| C001 | 0.990 | 0.958 | 0.969 | 0.704 | 0.471 | 0.016 |
| C010 | 0.996 | 0.979 | 0.986 | 0.965 | 0.849 | 0.096 |
| C005 | 0.995 | 0.982 | 0.973 | 0.910 | 0.689 | 0.677 |
| C014 | 0.984 | 0.854 | 0.933 | 0.723 | 0.402 | 0.267 |
| T300C010 | 0.975 | 0.876 | 0.925 | 0.706 | 0.926 | 0.776 |
| T300C021 | 0.940 | 0.919 | 0.959 | 0.569 | 0.729 | 0.383 |
| Avg. | 0.981 | 0.931 | 0.959 | 0.739 | 0.631 | 0.357 |
Table 2. Relative error of the rotation curve ↓, measured on the 3D Gaussian centers exported from each model.
Angular Momentum Conservation. In a nearly axisymmetric potential, the angular momentum of each star is approximately conserved. For every primitive tracked over at least ten frames we measure the relative drift of , take the luminosity-weighted median, and compare it in dex with the drift of the ground-truth particles. As shown in Table 3, Cosmos4D is closest to real stars on seven scenes, averaging 0.389 dex, 71% below the 1.350 dex of the best baseline. Baseline Gaussians follow paths that violate it, and RetimeGS primitives live for only about four frames.
| Scene | 4DGS | FreeGave | MoDec-GS | 4C4D | RetimeGS | Ours |
| G01 | 2.454 | 1.809 | 1.996 | 1.682 | – | 0.192 |
| G02 | 2.561 | 2.002 | 1.874 | 1.848 | – | 0.501 |
| C001 | 2.473 | 1.665 | 2.104 | 1.643 | – | 0.241 |
| C010 | 2.043 | 0.656 | 1.763 | 2.271 | – | 0.000 |
| C005 | 2.235 | 1.559 | 1.938 | 1.979 | – | 0.461 |
| C014 | 2.383 | 1.340 | 1.745 | 1.456 | – | 0.718 |
| T300C010 | 2.238 | 0.965 | 1.435 | 2.049 | – | 0.047 |
| T300C021 | 2.359 | 0.803 | 1.511 | 1.774 | – | 0.951 |
| Avg. | 2.343 | 1.350 | 1.796 | 1.838 | – | 0.389 |
Table 3. Angular momentum conservation error ↓ in dex. RetimeGS primitives stay active for only about four frames, so their along a trajectory is undefined and is thus reported as a dash.
Half-Light Radius. The half-light radius needs only rendered images, so all methods are compared on equal terms. We deproject every frame of an unseen near face-on view onto the disk plane and find the radius enclosing half of the light over 24 rings. As shown in Table 4, Cosmos4D reduces the average relative error by 55%, from 0.264 for the best baseline to 0.120, and is best on six scenes. Because the images are tone-mapped, is an apparent radius.
| Scene | 4DGS | FreeGave | MoDec-GS | 4C4D | RetimeGS | Ours |
| G01 | 0.060 | 0.058 | 0.057 | 0.042 | 0.031 | 0.009 |
| G02 | 0.131 | 0.130 | 0.082 | 0.051 | 0.062 | 0.019 |
| C001 | 0.288 | 0.280 | 0.323 | 0.307 | 0.237 | 0.163 |
| C010 | 0.250 | 0.180 | 0.233 | 0.232 | 0.218 | 0.194 |
| C005 | 0.106 | 0.083 | 0.125 | 0.272 | 0.109 | 0.122 |
| C014 | 0.272 | 0.247 | 0.280 | 0.214 | 0.170 | 0.129 |
| T300C010 | 0.505 | 0.506 | 0.514 | 0.464 | 0.545 | 0.125 |
| T300C021 | 0.724 | 0.730 | 0.721 | 0.705 | 0.739 | 0.199 |
| Avg. | 0.292 | 0.277 | 0.292 | 0.286 | 0.264 | 0.120 |
Table 4. Relative error of the half-light radius ↓, measured on deprojected renderings from the unseen near face-on view.
Ablation
We ablate the main components on G01, with both bright and faint particles, and on the sparse T300C010, whose particles are all bright. Table 5 reports image and physical metrics, and Appendix § 14 gives more detailed results.
| Setting | G01 Mask PSNR↑ | G01 Mask SSIM↑ | G01 Mask LPIPS↓ | G01 Full PSNR↑ | G01 Full SSIM↑ | G01 Full LPIPS↓ | G01 Physics RMSE↓ | G01 Physics ↓ | T300C010 Mask PSNR↑ | T300C010 Mask SSIM↑ | T300C010 Mask LPIPS↓ | T300C010 Full PSNR↑ | T300C010 Full SSIM↑ | T300C010 Full LPIPS↓ | T300C010 Physics RMSE↓ | T300C010 Physics ↓ |
| w/o additive splatting | 19.19 | 0.326 | 0.391 | 25.16 | 0.701 | 0.322 | 8.61 | 0.185 | 16.66 | 0.168 | 0.356 | 23.51 | 0.558 | 0.419 | 24.57 | 0.740 |
| w/o evolving emission | 20.32 | 0.368 | 0.405 | 26.33 | 0.705 | 0.350 | 8.58 | 0.125 | 17.02 | 0.178 | 0.337 | 23.78 | 0.560 | 0.404 | 24.58 | 0.751 |
| w/o gravitational orbits | 19.46 | 0.288 | 0.395 | 25.47 | 0.673 | 0.353 | 8.88 | 0.154 | 17.18 | 0.158 | 0.400 | 23.65 | 0.553 | 0.465 | 23.71 | 0.506 |
| w/o anchor potential | 20.00 | 0.338 | 0.379 | 26.02 | 0.692 | 0.342 | 8.92 | 0.232 | 17.24 | 0.175 | 0.359 | 23.64 | 0.553 | 0.429 | 24.00 | 0.286 |
| w/o multi-scale | 20.12 | 0.372 | 0.366 | 26.15 | 0.708 | 0.336 | 8.60 | 0.122 | 16.64 | 0.165 | 0.427 | 23.59 | 0.564 | 0.499 | 24.55 | 0.783 |
| Full model | 20.44 | 0.387 | 0.363 | 26.45 | 0.712 | 0.330 | 8.60 | 0.121 | 17.01 | 0.183 | 0.330 | 23.74 | 0.561 | 0.387 | 24.58 | 0.776 |
Table 5. Ablation on the test views. Mask and Full are metrics inside the galaxy foreground mask and on the whole image. RMSE (ckpc/h) is the position error of the modeled stars against ground-truth particles, and is the relative rotation-curve error used in Table 2.
Optically Thin Rendering and Evolving Emission. Replacing additive splatting with alpha compositing causes the largest drop on G01, 1.25 dB in masked PSNR, and raises masked LPIPS by 8% on G01, which confirms that an optically thin galaxy should not be rendered with occlusion. When each star keeps its anchor color over time, SSIM falls on both scenes and LPIPS rises by 12% on G01. Figure 5 shows that the young blue stars then fade to grey, so evolving emission is needed to follow stellar aging.

Figure 5. Ablations rendered on test views of G01 (top) and T300C010 (bottom). Without additive splatting, front stars dim those behind them. Without evolving emission, the young blue stars fade to grey. Without gravitational orbits or the anchor potential, bright stars drift off the disk and blur the cluster. The insets reveal these details in a young star cluster on the outer arm of G01 and in the core of T300C010.
Luminosity-Stratified Dynamics. Carrying all particles with the shared flow field instead of giving bright stars individual orbits lowers masked SSIM by 26% on G01 and 14% on T300C010 and raises LPIPS on both, and on G01 the rotation-curve error grows from 0.121 to 0.154. Keeping only the full-resolution level of the pyramid costs 0.32 and 0.37 dB, and on T300C010 masked LPIPS grows by 29%, as misplaced stars in this sparse scene lose gradients.
Gravitational Dynamics. Removing the anchor potential and keeping only the learned force lowers masked SSIM by 13% on G01 and 4% on T300C010 and raises LPIPS on both, and on G01 the trajectory and rotation-curve errors grow from 8.60 to 8.92 ckpc/h and from 0.121 to 0.232. On T300C010, whose fitted disk spins too fast, both orbit ablations lower , yet Figure 5 shows their bright stars scattering away from the disk.
Conclusion
We present Cosmos4D, which reconstructs an evolving galaxy in four dimensions from multi-view images. Stars become self-luminous Gaussians that fall through a gravitational potential, which confines their motion to physically plausible orbits. Luminosity-stratified dynamics trace bright stars along individual orbits and carry faint ones with a shared flow. On eight simulated galaxies, Cosmos4D renders the sharpest stellar texture among dynamic Gaussian methods and best recovers rotation, angular momentum, and galaxy size. Future work could refine the potential jointly with the images and move toward real observations.
Astrophysical Background
This section summarizes the astrophysics behind our setting for readers outside astronomy, and Binney and Tremaine [1] give a complete treatment. Figure 7 illustrates the main quantities on G01.
Figure 7. Background quantities on G01. (a) Circular speed of the anchor potential fitted in Sec. D and of its bulge, disk, and dark matter halo components, which add in quadrature. (b) Luminosity against age at the anchor for 1,500 sampled stars each of the bright set and the faint set . (c) Median emission gain of Sec. C over lookback time for 400 young stars born during the sequence and 400 old stars, normalized at the anchor.
Units and Scales. Galactic distances are measured in kiloparsecs, and 1 kpc is about 3,260 light-years. Cosmological simulations store positions in comoving units of ckpc/, where is the dimensionless Hubble constant. At the present epoch, which ends every sequence, 1 ckpc/ equals kpc, or about 1.5 kpc for the TNG value . Times are given in Myr and Gyr, and lookback time counts backward from the anchor. Stellar speeds are quoted in km s, and 1 km s is about 1.02 kpc per Gyr. A star moving at 200 km s thus travels about 160 kpc during the 0.8 Gyr spanned by G01, far more than the size of the disk.
Anatomy of a Disk Galaxy. A disk galaxy such as G01 has three main stellar components. The thin rotating disk holds most young stars and the spiral arms, the central bulge is a compact spheroid of mostly old stars, and a sparse stellar halo surrounds both. Figure 6 shows these components in two nearby spirals imaged by Hubble, together with the cylindrical coordinates used in this paper. The M74 image is credited to NASA, ESA, and The Hubble Heritage (STScI/AURA)-ESA/Hubble Collaboration, and the NGC 891 image to ESA/Hubble & NASA with acknowledgement to Nick Rose, both released under CC BY 4.0. All three sit inside a much larger halo of dark matter, which emits no light yet dominates the mass beyond the inner few kiloparsecs. Figure 7a shows these contributions through the circular speed , the speed of a circular orbit at radius in the disk plane. The dark matter halo keeps nearly flat at large radii, a hallmark of disk galaxies. Spiral arms are patterns through which stars pass rather than fixed groups of stars, so following an arm in the images does not follow its current stars.

Figure 6. Real disk galaxies imaged by Hubble. (a) The face-on spiral M74 with radius and azimuth in the disk plane. (b) The edge-on spiral NGC 891 with height above the disk.
Collisionless Orbits. A star is about a hundred-millionth the size of the distance to its nearest neighbors, so stars almost never collide or deflect one another. Each star instead
moves in the smooth gravitational potential of the whole galaxy, which follows from the mass density through the Poisson equation with gravitational constant . Disk stars travel on nearly circular orbits with small radial and vertical oscillations. These oscillations do not repeat with the orbital period, so an orbit traces a rosette rather than a closed curve. Inner stars complete an orbit faster than outer ones, so the disk rotates differentially and any pattern drawn on it winds up over time. Near the Sun an orbit takes about 200 Myr [2]. In G01, a circular orbit takes about 165 Myr at kpc and 480 Myr at kpc, so the sequence covers about five inner orbits but fewer than two outer ones.
Conserved Quantities. In a potential that is symmetric around the disk axis and constant in time, a star keeps its energy and its angular momentum about the axis, , where is its azimuthal velocity. Bars and spiral arms break this symmetry slightly and exert small torques, so drifts slowly in real galaxies. For this reason Table 3 compares the drift of each method with that of the ground-truth particles rather than with zero. The rotation curve , the mean azimuthal velocity of stars at radius , traces the mass enclosed within and is the most common kinematic summary of a disk galaxy.
Stellar Populations. A star particle in a cosmological simulation is not a single star. It stands for a population of stars born together, with a mass of about solar masses in TNG50 and up to about in TNG300, and it carries a formation time and a metallicity, the mass fraction of elements heavier than helium. Massive stars are hot, blue, and extremely luminous but live only a few to a few tens of Myr. A young population is therefore bright and blue, and it fades and reddens as its massive members die. Metal-rich populations are also redder at a given age. Figure 7b shows the fading on G01, where young particles form most of the bright set and old particles most of the faint set . Figure 7c shows the resulting emission gain, which falls by more than an order of magnitude between birth and the anchor for young stars and stays nearly constant for old ones. New particles form throughout the sequence, so part of the light at the anchor is absent from the earliest frames.
Optically Thin Light. Because stars cover a negligible fraction of the sky even in a dense disk, light from distant stars reaches the camera past nearer ones. Without dust, an image is the sum of the light of all stars along each line of sight. Dust in real galaxies absorbs and reddens part of this light, and our renderings omit it as described in Sec. B. This additive image formation motivates rendering each star by additive splatting in § 3.1.
Dataset
Simulation Snapshots. Each scene is a galaxy taken from a cosmological hydrodynamical simulation, which evolves dark matter, gas, and stars under gravity and baryonic physics and stores the full particle state at discrete times called snapshots. We use the subbox outputs of TNG50-1 [40, 44], TNG100-1, TNG300-1, and Illustris-1, which save a fixed subvolume at a much higher cadence than the full box and therefore resolve galactic motion between frames. For every scene we keep the last 96 subbox snapshots, so the final snapshot serves as the anchor. Each snapshot is recentered on the galaxy when the data are generated. Table 6 lists the scenes and their particle counts.
| Scene | Source | |||
| G01 | TNG50-1 Subbox0 | 110,013 | 56,572 | 53,441 |
| G02 | TNG50-1 Subbox0 | 110,453 | 45,217 | 65,236 |
| C001 | TNG100-1 Subbox1 | 75,123 | 75,123 | 0 |
| C010 | TNG100-1 Subbox0 | 77,379 | 77,379 | 0 |
| C005 | Illustris-1 Subbox0 | 110,265 | 3,159 | 107,106 |
| C014 | Illustris-1 Subbox0 | 90,324 | 90,324 | 0 |
| T300C010 | TNG300-1 Subbox1 | 18,795 | 18,795 | 0 |
| T300C021 | TNG300-1 Subbox1 | 9,332 | 9,332 | 0 |
Table 6. Scenes used in our experiments. is the number of sampled stellar particles, split into bright and faint sets.
Stellar Sampling. Candidates are the stars already formed within a scene-scaled radius of 115 ckpc/. Each star enters the sample with probability , where is chosen so that , through a single Bernoulli draw. The same identities are used in every frame and each star is rendered with luminosity , which keeps the linear flux unbiased and avoids the frame-to-frame flicker of per-snapshot resampling. Stars with form the bright set of § 3.2, and sampled stars with form the faint set , each standing in for about stars. If every formed star is sampled, the scene has no faint stars.
Rendering. Fourteen cameras are placed on a Fibonacci sphere, with radius and field of view scaled by the galactic half-light radius. Star colors follow an analytic model of age and metallicity, scaled by the sampled luminosity. Each star is splatted as a point source, radiance is accumulated per band without dust attenuation, and the image is slightly blurred by a Gaussian kernel. Exposure is set per view at the 99.9th percentile with temporal smoothing, and the final image maps each normalized intensity to . Cameras 1, 7, and 13 are held out for testing. Figure 8 shows the camera layout. We exclude the FIRE-2 galaxies m12m and m12f because their snapshot spacing is non-uniform, which makes frame-indexed baselines and our time-integrated dynamics incomparable.
Figure 8. Camera layout on G01, drawn with the disk normal pointing up. Fourteen cameras lie on a Fibonacci sphere around the galaxy. Gold circles are training views, blue squares are test views, and each short tick points toward the galaxy center.
Appearance Model
Camera Response. The camera response is with , where is an exposure scale and the 99.9th percentile is computed per rendered image under a stop-gradient.
Evolving Emission. The anchor emission is initialized from the age, metallicity, and importance-weighted luminosity of each star. The emission gain in Eq. (2) is , normalized by its value at the anchor and linearly interpolated between frames, where is the age of star at time , is its metallicity, and is the same age and metallicity color used to initialize . The luminosity correction in Eq. (2) is linearly interpolated from 13 uniform temporal knots, so its factor stays between and . Both and this factor equal one at the anchor, where reduces to .
Gravitational Dynamics
Anchor Potential. The disk frame is centered at the median anchor position, with the disk normal given by the smallest principal axis of particles within 20 kpc. The potential combines a Hernquist bulge, a Miyamoto–Nagai disk, and an NFW halo, the three main mass components of a disk galaxy. Disk scale length, scale height, and bulge fraction are estimated from anchor positions only. Dark matter emits no light yet dominates the mass of the outer galaxy, so the halo is derived from . Its mass follows the stellar-to-halo mass relation of Moster et al. [37], and its concentration follows Dutton and Maccio [11]. The potential contains no bar or spiral terms, so non-axisymmetric structure is left to the residual force field . Bright disk stars start on circular orbits around the disk axis, with the circular speed of at their anchor radius. Disk and halo stars are separated by a two-component exponential mixture over fitted within each radial bin, and stars whose disk posterior is below 0.5 are labeled as halo. Halo stars start on circular orbits in randomly oriented planes. In thick-disk scenes (C014, T300C010, and T300C021), all bright stars are treated as disk stars.
Residual Force Basis. Some fields in Eq. (4) are symmetric around the disk axis and adjust the overall pull, while others have two arms () and represent bars and spirals. Each non-uniform field is with . The 36 axisymmetric terms () use 12 log-spaced radii in kpc and heights kpc. The 16 terms use the middle 8 radii with . Three uniform accelerations complete the basis and absorb the independent recentering of the benchmark snapshots, giving 55 fields in total. The sum in Eq. (4) is scaled by . Coefficients are zero-initialized and defined on 13 uniform temporal knots, which gives 715 shared coefficients.
Leapfrog Integrator. We use a kick–drift–kick leapfrog scheme with substeps of at most 1 Myr, run in double precision. The scheme alternates small velocity and position updates and conserves energy well over many orbital periods, which makes it a standard choice for stellar dynamics. With step , negative when integrating backward from the anchor, one step updates velocity and position as
with from Eq. (3). Within each frame interval, the residual force is evaluated at the starting position and interpolated in time, while gravity is evaluated throughout the integration. Gradients follow this discrete map exactly. Hermite interpolation of frame positions and velocities yields motion between frames. Gradients of each frame-interval map are computed in forward mode inside a CUDA kernel and applied as vector–Jacobian products, with gradient checkpointing per interval. On our scenes, the anchor position error is kpc/, and finite-difference checks of directional gradients agree to a relative error of .
Blended Orbital Segments
The main text describes each bright orbit as one backward integration from the anchor (see § 3.1). This section describes how we keep these orbits optimizable over long sequences in practice. A single backward integration from the anchor ties early frames to accumulated state errors. Short independent integrations loosen this coupling, but switching abruptly between them produces visible jumps in position and velocity. We instead optimize overlapping segments and render one continuous trajectory. As illustrated in Figure 9a, each bright orbit is split into 16-frame segments that overlap by 8 frames. Each segment is integrated backward from a learned endpoint state, and only the last one reaches the anchor. Within an overlap the trajectory passes smoothly from one segment to the next rather than jumping between them (Figure 9b and Figure 9c).
Figure 9. Blended orbital segments. (a) Overlapping segments integrated backward from learned endpoints. (b) Blending two segments within an overlap versus a hard switch. (c) Quintic blend weight and the step weight of a hard switch.
Local Orbit Integration. Each bright-particle segment is integrated backward from endpoint , starting from
where bars denote analytic seeds and corresponds to 40 km/s in integration units. Endpoint offsets are learned from images, with the anchor position held fixed. For the 96-frame sequence, segments span 16 frame intervals with an overlap of 8, giving endpoints 15, 23, , 95 and starts . All segments share the same force field and differentiable integrator.
Partition-of-Unity Blending. Within the overlap of adjacent segments and , let denote the normalized overlap time. We blend the two trajectories as
The velocity is the exact derivative of this blend, including . Because the weight has vanishing first and second derivatives at both ends, the assembly introduces no jumps in position or velocity. With the Hermite interpolants, each bright trajectory is , and the anchor, covered only by its own segment, keeps its position exactly.
Continuity, however, does not by itself ensure dynamical consistency. Let and . If both segments satisfy Eq. (3), then the blended trajectory obeys
up to integration and interpolation error. Non-Newtonian forces thus arise only where segments disagree and vanish once they agree. They also weaken as the overlap length grows, since and . Classical multiple shooting amounts to a step weight, which concentrates the same disagreement into an impulse at each node. We therefore treat the result as a physically guided approximation rather than an exact Newtonian solution of the dynamics.
Integrating each bright orbit as a single segment from the anchor degrades rendering on both scenes and raises the trajectory error from 8.60 to 11.10 ckpc/ on G01 and from 24.58 to 26.53 ckpc/ on T300C010 (Table 7), which supports the segmented formulation. Hard switching between segments changes every metric only marginally, so the blending mainly provides continuous motion rather than better image fits.
| Setting | G01 PSNR | G01 SSIM | G01 LPIPS | G01 RMSE | T300C010 PSNR | T300C010 SSIM | T300C010 LPIPS | T300C010 RMSE | ||||||
| Full model | 20.44 | 0.387 | 0.363 | 8.60 | 17.01 | 0.183 | 0.330 | 24.58 | ||||||
| Single segment | 19.88 | 0.319 | 0.366 | 11.10 | 16.93 | 0.157 | 0.364 | 26.53 | ||||||
| Hard switching | 20.43 | 0.385 | 0.364 | 8.60 | 16.97 | 0.180 | 0.335 | 24.60 | ||||||
| Setting | G01 | T300C010 | ||||||||||||
| Mask | Full | Mask | Full | |||||||||||
| PSNR↑ | SSIM↑ | LPIPS↓ | PSNR↑ | SSIM↑ | LPIPS↓ | RMSE↓ | PSNR↑ | SSIM↑ | LPIPS↓ | PSNR↑ | SSIM↑ | LPIPS↓ | RMSE↓ | |
| Full model | 20.44 | 0.387 | 0.363 | 26.45 | 0.712 | 0.330 | 8.60 | 17.01 | 0.183 | 0.330 | 23.74 | 0.561 | 0.387 | 24.58 |
| w/o additive splatting | 19.19 | 0.326 | 0.391 | 25.16 | 0.701 | 0.322 | 8.61 | 16.66 | 0.168 | 0.356 | 23.51 | 0.558 | 0.419 | 24.57 |
| w/o evolving emission | 20.32 | 0.368 | 0.405 | 26.33 | 0.705 | 0.350 | 8.58 | 17.02 | 0.178 | 0.337 | 23.78 | 0.560 | 0.404 | 24.58 |
| w/o residual force | 20.44 | 0.386 | 0.362 | 26.45 | 0.711 | 0.330 | 8.63 | 16.97 | 0.182 | 0.331 | 23.72 | 0.561 | 0.385 | 24.57 |
| w/o gravitational orbits | 19.46 | 0.288 | 0.395 | 25.47 | 0.673 | 0.353 | 8.88 | 17.18 | 0.158 | 0.400 | 23.65 | 0.553 | 0.465 | 23.71 |
| w/o anchor potential | 20.00 | 0.338 | 0.379 | 26.02 | 0.692 | 0.342 | 8.92 | 17.24 | 0.175 | 0.359 | 23.64 | 0.553 | 0.429 | 24.00 |
| w/o multi-scale | 20.12 | 0.372 | 0.366 | 26.15 | 0.708 | 0.336 | 8.60 | 16.64 | 0.165 | 0.427 | 23.59 | 0.564 | 0.499 | 24.55 |
Table 7. Segmented integration on G01 and T300C010. Single segment integrates each bright orbit from the anchor in one piece, and hard switching replaces the blending weight with a step. Metrics are masked, and RMSE is the 3D position error of the modeled stars in ckpc/, used only for diagnosis.
Faint-Star Flow
Flow Estimation. Optical flow is computed with DIS [21] on grayscale training views in both directions. At sampled disk-plane locations, camera projection Jacobians map each three-dimensional basis velocity to a pixel displacement, and the coefficients are recovered by robust least
squares weighted by visibility, brightness, and forward–backward consistency. The disk-plane sampling grid has 32 log-spaced radii and 96 azimuths. The velocity basis has 99 coefficients per frame, comprising 9 rotational, 72 radial/tangential , 9 vertical, and 9 radial components. We solve ridge-regularized weighted least squares with four reweighting iterations and smooth the coefficients temporally with a Gaussian of standard deviation 2 frames.
Bounded Corrections. The deformation network in Eq. (6) outputs , where is the raw MLP output and the componentwise bound is ckpc/ by default and 40 ckpc/ on C005. The shared phase correction is , with linearly interpolated from 13 uniform temporal knots, so that rad and vanishes at the anchor. The neural residual uses a multi-resolution HexPlane with a zero-initialized output head, and its learning rates are rescaled by the scene normalization factor so that they match the effective rates of 4DGS [55].
Implementation Details
Gaussians and Rasterizer. Each Gaussian keeps a fixed one-to-one correspondence with a sampled stellar particle, and Gaussian centers have no free parameters and are always produced by the dynamics module. Rotations are frozen and only the DC color component is used. Splat radii are with ckpc/, and are learned only during the anchor-only stage. Opacities are initialized to 0.08. The additive rasterizer skips contributions below and applies no output blur.
Losses. The pixel weight in Eq. (9) is , where the constant keeps a unit weight on the dark sky, and at the coarse levels the pixel weights are average-pooled together with the images. In Eq. (11) we use with and with . The phase curve of the faint-star rotation is regularized in the same form, with and .
Optimization. The anchor-only stage uses images at the anchor time only and keeps positions static. In the joint stage, every base learning rate is multiplied by with clip at iteration , which gives a 400-iteration linear warmup followed by cosine decay to a 10% floor at iteration 24,000. Base learning rates are for color, opacity, and force coefficients, 0.025 for shooting position offsets, and 0.01 for shooting velocity offsets and luminosity corrections. Dynamics, velocity, and neural-field gradients are clipped to norm 0.1. Image resolution grows from to the full , reached at iteration 801. The anchor-to-past curriculum samples frames uniformly from the last frames ending at the anchor, so all 96 frames are covered from iteration 761. We report iteration 32,000 without test-view checkpoint selection. Training takes 6–20 hours per scene on one GPU, depending on the particle count.
Hyperparameter Selection. Hyperparameters and the few scene-dependent settings are selected on a held-out training camera, trained for 8,000 iterations, using a rule fixed in advance. That camera is excluded from flow estimation during validation. Test cameras are never used for selection. The scene-dependent settings are the exposure scale , whether a separate halo population is modeled for seeding, and the faint residual bound , which is 5 ckpc/ except 40 ckpc/ for C005.
Additional Ablations
Tab. 8 lists every ablation on the two ablation scenes, with masked and full-image metrics and the trajectory error of the modeled stars. Ground-truth particles enter only this diagnostic and never training or selection. Freezing the residual force changes PSNR by less than 0.1 dB and SSIM and LPIPS by less than 0.01. Each variant is trained with a single seed, so differences of this size should not be read as meaningful. Figure 10 follows the trajectory error of each variant over the sequence. The error is zero at the anchor and grows back in time for every variant, and removing the anchor potential raises the radial error over most of the sequence on both scenes. Figure 11 plots the disk-plane tracks of sample bright stars for every variant.
Figure 10. Trajectory error of all modeled stars against ground-truth particles over lookback time for each ablation variant on G01 (top) and T300C010 (bottom). Columns show the 3D position error and its radial and vertical components in ckpc/.
Figure 11. Disk-plane tracks of four bright stars for each ablation variant on G01 (top) and T300C010 (bottom), with the full model first and the ablations in the order of Table 5. Each row covers one third of the sequence, counted back in lookback time from the anchor. Dashed gray lines are ground-truth particle tracks, and solid lines are the modeled tracks in the color of the column title. Filled dots mark the start of each interval and open circles its end, in black or gray for the ground truth and in the variant color for the model. The cross marks the galaxy center. Ground-truth particles are used for evaluation only.
Limitations
Single-View Observations. A telescope observes each galaxy from a single direction, and galactic rotation is far too slow to change the image over a human lifetime. A real observation therefore provides one view at one instant, whereas Cosmos4D, like other dynamic scene reconstruction methods, requires multiple calibrated views over time. For this reason our experiments use simulated galaxies rendered into 14 cameras. Applying the method to real data requires priors that supply the missing views and epochs. Generative models that synthesize multi-view videos from sparse inputs [56] are one option, and our gravitational dynamics could then constrain the generated views toward consistent orbital motion.
Evaluation. Ground-truth three-dimensional positions and velocities of individual stars are unavailable for real galaxies, so both the image and the physical metrics are computed on simulated scenes. The evaluation is thus bounded by how closely TNG and Illustris galaxies, with their finite particle resolution and subgrid physics, resemble real ones. Real observations offer only indirect checks, such as spectroscopic line-of-sight velocities, which constrain a single velocity component and do not yield full trajectories.
Dust. Our renderer treats stars as optically thin emitters. In real galaxies, interstellar dust absorbs and scatters starlight and dims and reddens regions of active star formation [5]. Dust lanes violate the additive emission model and may be explained as missing stars or altered luminosities. Real data would require wavelength-dependent attenuation along each ray.
Noise and Resolution. Our simulated images are noise-free, and stars span a few pixels each. Real deep-sky images contain sensor noise from photon statistics, dark current, and readout [29], and the limited angular resolution of telescopes blends many stars into each pixel [30]. Faint stars, which carry much of the flow in our model, are affected most. Physics-based denoising [29] and flux-conserving super-resolution [30] could serve as preprocessing, and modeling such noise and degradation inside the rendering loss is a natural extension.
References
- [1]James Binney and Scott Tremaine. Galactic Dynamics. Princeton University Press, second edition, 2008. 1, 2, 4, 9
- [2]Joss Bland-Hawthorn and Ortwin Gerhard. The galaxy in context: Structural, kinematic, and integrated properties. Annual Review of Astronomy and Astrophysics, 54:529–596, 2016. 1, 9
- [3]Yochai Blau and Tomer Michaeli. The perception-distortion tradeoff. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 6228–6237, 2018. 7DOI
- [4]Andreas Brunthaler, Mark J. Reid, Heino Falcke, Lincoln J. Greenhill, and Christian Henkel. The geometric distance and proper motion of the Triangulum Galaxy (M33). Science, 307(5714):1440–1443, 2005. 1DOI
- [5]Daniela Calzetti, Lee Armus, Ralph C. Bohlin, Anne L. Kinney, Jan Koornneef, and Thaisa Storchi-Bergmann. The dust content and opacity of actively star-forming galaxies. The Astrophysical Journal, 533(2):682–695, 2000. 15DOI
- [6]Ang Cao and Justin Johnson. HexPlane: A fast representation for dynamic scenes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 130–141, 2023. 2, 5arxiv.org/abs/2301.09632
- [7]Laurent Chemin, Claude Carignan, and Tyler Foster. HI kinematics and dynamics of Messier 31. The Astrophysical Journal, 705(2):1395–1415, 2009. 1DOI
- [8]Ricky T. Q. Chen, Yulia Rubanova, Jesse Bettencourt, and David K. Duvenaud. Neural ordinary differential equations. In Advances in Neural Information Processing Systems, 2018. 2arxiv.org/abs/1806.07366
- [9]Mengyu Chu, Lingjie Liu, Quan Zheng, Aleksandra Franz, Hans-Peter Seidel, Christian Theobalt, and Rhaleb Zayer. Physics informed neural fields for smoke reconstruction with sparse data. ACM Transactions on Graphics, 41(4):1–14, 2022. 2DOI
- [10]Feilong Du, Yalan Zhang, Yihang Ji, Xiaokun Wang, Chao Yao, Jiri Kosinka, Steffen Frey, Alexandru Telea, and Xiaojuan Ban. GaussFluids: Reconstructing lagrangian fluid particles from videos via gaussian splatting. In Pacific Graphics Conference Papers, Posters, and Demos, 2025. 2DOI
- [11]Aaron A. Dutton and Andrea V. Macciò. Cold dark matter haloes in the Planck era: evolution of structural parameters for Einasto and NFW profiles. Monthly Notices of the Royal Astronomical Society, 441(4):3359–3374, 2014. 11DOI
- [12]Kenneth C. Freeman. On the disks of spiral and S0 galaxies. The Astrophysical Journal, 160:811–830, 1970. 3
- [13]Sara Fridovich-Keil, Giacomo Meanti, Frederik Rahbæk Warburg, Benjamin Recht, and Angjoo Kanazawa. K-Planes: Explicit radiance fields in space, time, and appearance. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12479–12488, 2023. 2
- [14]Shy Genel, Mark Vogelsberger, Volker Springel, Debora Sijacki, Dylan Nelson, Greg Snyder, Vicente Rodriguez-Gomez, Paul Torrey, and Lars Hernquist. Introducing the Illustris project: the evolution of galaxy populations across cosmic time. Monthly Notices of the Royal Astronomical Society, 445(1):175–200, 2014. 6DOI
- [15]Samuel Greydanus, Misko Dzamba, and Jason Yosinski. Hamiltonian neural networks. In Advances in Neural Information Processing Systems, 2019. 2arxiv.org/abs/1906.01563
- [16]Lars Hernquist. An analytical model for spherical galaxies and bulges. The Astrophysical Journal, 356:359–364, 1990. 2, 4DOI
- [17]Yuanming Hu, Luke Anderson, Tzu-Mao Li, Qi Sun, Nathan Carr, Jonathan Ragan-Kelley, and Frédo Durand. DiffTaichi: Differentiable programming for physical simulation. In International Conference on Learning Representations, 2020. 2arxiv.org/abs/1910.00935
- [18]Yi-Hua Huang, Yang-Tian Sun, Ziyi Yang, Xiaoyang Lyu, Yan-Pei Cao, and Xiaojun Qi. SC-GS: Sparse-controlled gaussian splatting for editable dynamic scenes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4220–4230, 2024. 2arxiv.org/abs/2312.14937
- [19]Hanxiao Jiang, Hao-Yu Hsu, Kaifeng Zhang, Hsin-Ni Yu, Shenlong Wang, and Yunzhu Li. PhysTwin: Physics-informed reconstruction and simulation of deformable objects from videos. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 7219–7230, 2025. 2arxiv.org/abs/2503.17973
- [20]Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis. 3D gaussian splatting for real-time radiance field rendering. ACM Transactions on Graphics, 42(4):1–14, 2023. 2arxiv.org/abs/2308.04079
- [21]Till Kroeger, Radu Timofte, Dengxin Dai, and Luc Van Gool. Fast optical flow using dense inverse search. In Computer Vision – ECCV 2016, pages 471–488, 2016. 12arxiv.org/abs/1603.03590
- [22]Sangwoon Kwak, Joonsoo Kim, Jun Young Jeong, Won-Sik Cheong, Jihyong Oh, and Munchurl Kim. MoDeC-GS: Global-to-local motion decomposition and temporal interval adjustment for compact dynamic 3D gaussian splatting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11338–11348, 2025. 6, 7arxiv.org/abs/2501.03714
- [23]Jiahui Lei, Yijia Weng, Adam W. Harley, Leonidas Guibas, and Kostas Daniilidis. MoSca: Dynamic gaussian fusion from casual videos via 4D motion scaffolds. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6165–6177, 2025. 2arxiv.org/abs/2405.17421
- [24]Jinxi Li, Ziyang Song, Siyuan Zhou, and Bo Yang. Freegave: 3d physics learning from dynamic videos by gaussian velocity. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12433–12443, 2025. 6, 7arxiv.org/abs/2506.07865
- [25]Siyang Li, Adam G. Riess, Michael P. Busch, Stefano Casertano, Lucas M. Macri, and Wenlong Yuan. A sub-2% distance to M31 from photometrically homogeneous near-infrared Cepheid period–luminosity relations measured with the Hubble Space Telescope. The Astrophysical Journal, 920(2):84, 2021. 1DOI
- [26]Shiqian Li, Ruihong Shen, Junfeng Ni, Chang Pan, Chi Zhang, and Yixin Zhu. Learning physics-grounded 4D dynamics with neural gaussian force fields. In International Conference on Learning Representations, 2026. 3arxiv.org/abs/2602.00148
- [27]Xuan Li, Yi-Ling Qiao, Peter Yichen Chen, Krishna Murthy Jatavallabhula, Ming Lin, Chenfanfu Jiang, and Chuang Gan. PAC-NeRF: Physics augmented continuum neural radiance fields for geometry-agnostic system identification. In International Conference on Learning Representations, 2023. 2arxiv.org/abs/2303.05512
- [28]Zhan Li, Zhang Chen, Zhong Li, and Yi Xu. Spacetime gaussian feature splatting for real-time dynamic view synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8508–8520, 2024. 2arxiv.org/abs/2312.16812
- [29]Shuhong Liu, Xining Ge, Ziying Gu, Quanfeng Xu, Lin Gu, Ziteng Cui, Xuangeng Chu, Jun Liu, Dong Li, and Tatsuya Harada. Denoising the deep sky: Physics-based CCD noise formation for astronomical imaging. In Computer Vision – ECCV 2026, pages 550–569, 2026. 15arxiv.org/abs/2601.23276
- [30]Shuhong Liu, Xining Ge, Quanfeng Xu, Ziteng Cui, Liuzhuozheng Li, Gengjia Chang, Jun Liu, Ziying Gu, Dong Li, Xuangeng Chu, Lin Gu, and Tatsuya Harada. FluxFlow: Conservative flow-matching for astronomical image super- resolution. In Advances in Neural Information Processing Systems (NeurIPS), 2026. 15arxiv.org/abs/2605.03749
- [31]Zhuoman Liu, Weicai Ye, Yan Luximon, Pengfei Wan, and Di Zhang. Unleashing the potential of multi-modal foundation models and video diffusion for 4D dynamic physical scene simulation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11016–11025, 2025. 2
- [32]Jonathon Luiten, Georgios Kopanas, Bastian Leibe, and Deva Ramanan. Dynamic 3D gaussians: Tracking by persistent dynamic view synthesis. In 2024 International Conference on 3D Vision (3DV), pages 800–809, 2024. 2
- [33]Robert Lupton, Michael R. Blanton, George Fekete, David W. Hogg, Wil O’Mullane, Alex Szalay, and Nicholas Wherry. Preparing red-green-blue images from CCD data. Publications of the Astronomical Society of the Pacific, 116(816):133–137, 2004. 4DOI
- [34]Chunji Lv, Zequn Chen, Donglin Di, Weinan Zhang, Hao Li, Chen Wei, Yinjie Lei, and Changsheng Li. PhysGM: Large physical gaussian model for feed-forward 4D synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 29855–29865, 2026. 3arxiv.org/abs/2508.13911
- [35]Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren Ng. NeRF: Representing scenes as neural radiance fields for view synthesis. In Computer Vision – ECCV 2020, pages 405–421, 2020. 2arxiv.org/abs/2003.08934
- [36]Masanori Miyamoto and Ryuzaburo Nagai. Three-dimensional models for the distribution of mass in galaxies. Publications of the Astronomical Society of Japan, 27:533–543, 1975. 2, 4DOI
- [37]Benjamin P. Moster, Thorsten Naab, and Simon D. M. White. Galactic star formation and accretion histories from matching galaxies to dark matter haloes. Monthly Notices of the Royal Astronomical Society, 428(4):3121–3138, 2013. 11DOI
- [38]Julio F. Navarro, Carlos S. Frenk, and Simon D. M. White. A universal density profile from hierarchical clustering. The Astrophysical Journal, 490(2):493–508, 1997. 2, 4
- [39]Dylan Nelson, Annalisa Pillepich, Volker Springel, Rainer Weinberger, Lars Hernquist, Rüdiger Pakmor, Shy Genel, Paul Torrey, Mark Vogelsberger, Guinevere Kauffmann, Federico Marinacci, and Jill Naiman. First results from the IllustrisTNG simulations: the galaxy colour bimodality. Monthly Notices of the Royal Astronomical Society, 475(1):624–647, 2018. 6DOI
- [40]Dylan Nelson, Annalisa Pillepich, Volker Springel, Rüdiger Pakmor, Rainer Weinberger, Shy Genel, Paul Torrey, Mark Vogelsberger, Federico Marinacci, and Lars Hernquist. First results from the TNG50 simulation: galactic outflows driven by supernovae and black hole feedback. Monthly Notices of the Royal Astronomical Society, 490(3):3234–3261, 2019. 1, 2, 6, 10DOI
- [41]Keunhong Park, Utkarsh Sinha, Jonathan T. Barron, Sofien Bouaziz, Dan B Goldman, Steven M. Seitz, and Ricardo Martin-Brualla. Nerfies: Deformable neural radiance fields. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 5865–5874, 2021. 2arxiv.org/abs/2011.12948
- [42]Keunhong Park, Utkarsh Sinha, Peter Hedman, Jonathan T. Barron, Sofien Bouaziz, Dan B Goldman, Ricardo Martin-Brualla, and Steven M. Seitz. HyperNeRF: A higher-dimensional representation for topologically varying neural radiance fields. ACM Transactions on Graphics, 40(6):1–12, 2021. 2arxiv.org/abs/2106.13228
- [43]Annalisa Pillepich, Dylan Nelson, Lars Hernquist, Volker Springel, Rüdiger Pakmor, Paul Torrey, Rainer Weinberger, Shy Genel, Jill P. Naiman, Federico Marinacci, and Mark Vogelsberger. First results from the IllustrisTNG simulations: the stellar mass content of groups and clusters of galaxies. Monthly Notices of the Royal Astronomical Society, 475(1):648–675, 2018. 6DOI
- [44]Annalisa Pillepich, Dylan Nelson, Volker Springel, Rüdiger Pakmor, Paul Torrey, Rainer Weinberger, Mark Vogelsberger, Federico Marinacci, Shy Genel, Arjen van der Wel, and Lars Hernquist. First results from the TNG50 simulation: the evolution of stellar and gaseous discs across cosmic time. Monthly Notices of the Royal Astronomical Society, 490(3):3196–3233, 2019. 1, 2, 6, 10DOI
- [45]Albert Pumarola, Enric Corona, Gerard Pons-Moll, and Francesc Moreno-Noguer. D-NeRF: Neural radiance fields for dynamic scenes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10318–10327, 2021. 2arxiv.org/abs/2011.13961
- [46]Yi-Ling Qiao, Alexander Gao, and Ming Lin. Neu-Physics: Editable neural geometry and physics from monocular videos. In Advances in Neural Information Processing Systems, pages 12841–12854, 2022. 2arxiv.org/abs/2210.12352
- [47]Maziar Raissi, Paris Perdikaris, and George E. Karniadakis. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics, 378:686–707, 2019. 2DOI
- [48]J. A. Sellwood. Secular evolution in disk galaxies. Reviews of Modern Physics, 86(1):1–46, 2014. 4DOI
- [49]Sangmo Tony Sohn, Jay Anderson, and Roeland P. van der Marel. The M31 velocity vector. I. Hubble Space Telescope proper-motion measurements. The Astrophysical Journal, 753(1):7, 2012. 1
- [50]Volker Springel, Rüdiger Pakmor, Annalisa Pillepich, Rainer Weinberger, Dylan Nelson, Lars Hernquist, Mark Vogelsberger, Shy Genel, Paul Torrey, Federico Marinacci, and Jill Naiman. First results from the IllustrisTNG simulations: matter and galaxy clustering. Monthly Notices of the Royal Astronomical Society, 475(1):676–698, 2018. 6DOI
- [51]Mark Vogelsberger, Shy Genel, Volker Springel, Paul Torrey, Debora Sijacki, Dandan Xu, Greg Snyder, Dylan Nelson, and Lars Hernquist. Introducing the Illustris project: simulating the coevolution of dark and visible matter in the Universe. Monthly Notices of the Royal Astronomical Society, 444(2):1518–1547, 2014. 6DOI
- [52]Qianqian Wang, Vickie Ye, Hang Gao, Weijia Zeng, Jake Austin, Zhengqi Li, and Angjoo Kanazawa. Shape of motion: 4D reconstruction from a single video. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 9660–9672, 2025. 2arxiv.org/abs/2407.13764
- [53]Xuezhen Wang, Li Ma, Yulin Shen, Zeyu Wang, and Pedro V. Sander. RetimGS: Continuous-time reconstruction of 4D gaussian splatting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7340–7350, 2026. 6, 7arxiv.org/abs/2603.13783
- [54]Zhou Wang, Alan C. Bovik, Hamid R. Sheikh, and Eero P. Simoncelli. Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4):600–612, 2004. 6, 7pubmed.ncbi.nlm.nih.gov/15376593
- [55]Guangjun Wu, Taoran Yi, Jiemin Fang, Lingxi Xie, Xiaopeng Zhang, Wei Wei, Wenyu Liu, Qi Tian, and Xinggang Wang. 4D gaussian splatting for real-time dynamic scene rendering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20310–20320, 2024. 1, 2, 5, 6, 7, 13arxiv.org/abs/2310.08528
- [56]Rundi Wu, Ruiqi Gao, Ben Poole, Alex Trevithick, Changxi Zheng, Jonathan T. Barron, and Aleksander Holynski. CAT4D: Create anything in 4D with multi-view video diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 26057–26068, 2025. 15arxiv.org/abs/2411.18613
- [57]Tianyi Xie, Zeshun Zong, Yuxing Qiu, Xuan Li, Yutao Feng, Yin Yang, and Chenfanfu Jiang. PhysGaussian: Physics-integrated 3D gaussians for generative dynamics. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4389–4398, 2024. 2arxiv.org/abs/2311.12198
- [58]Ziyi Yang, Xinyu Gao, Wen Zhou, Shaohui Jiao, Yuqing Zhang, and Xiangyang Jin. Deformable 3D gaussians for high-fidelity monocular dynamic scene reconstruction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20331–20341, 2024. 2arxiv.org/abs/2309.13101
- [59]Richard Zhang, Phillip Isola, Alexei A. Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 586–595, 2018. 7arxiv.org/abs/1801.03924
- [60]Tianyuan Zhang, Hong-Xing Yu, Rundi Wu, Brandon Y. Feng, Changxi Zheng, Noah Snavely, Jiajun Wu, and William T. Freeman. PhysDreamer: Physics-based interaction with 3D objects via video generation. In Computer Vision – ECCV 2024, pages 388–406, 2024. 2arxiv.org/abs/2404.13026
- [61]Xuankai Zhang, Junjin Xiao, Shangwei Huang, Wei-shi Zheng, and Qing Zhang. Learning explicit continuous motion representation for dynamic gaussian splatting from monocular videos. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 33291–33300, 2026. 2arxiv.org/abs/2603.25058
- [62]Haoyu Zhao, Hao Wang, Xingyue Zhao, Hao Fei, Hongqiu Wang, Chengjiang Long, and Hua Zhou. PhysSplat: Efficient physics simulation for 3D scenes via MLLM-guided gaussian splatting. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 5242–5252, 2025. 2
- [63]Licheng Zhong, Hong-Xing Yu, Jiajun Wu, and Yunzhu Li. Reconstruction and simulation of elastic objects with spring-mass 3D gaussians. In Computer Vision – ECCV 2024, pages 407–423, 2024. 2arxiv.org/abs/2403.09434
- [64]Haoran Zhou and Gim Hee Lee. MotionScale: Reconstructing appearance, geometry, and motion of dynamic scenes with scalable 4D gaussian splatting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11860–11870, 2026. 2arxiv.org/abs/2603.29296
- [65]Junsheng Zhou, Zhifan Yang, Liang Han, Wenyuan Zhang, Kanle Shi, Shenkun Xu, and Yu-Shen Liu. 4C4D: 4 camera 4D gaussian splatting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11829–11839, 2026. 6, 7arxiv.org/abs/2604.04063







