WorksComputer Science
OCLDA-GS: Object-Centric Local Deformation-Aware Gaussian Streaming
Abstract
Streaming 3D Gaussian Splatting (3DGS) allows for online reconstruction of dynamic scenes from multi-view videos for Free Viewpoint Video (FVV). Current time and storage efficient methods rely on selecting a subset of anchor Gaussians and neighborhood interpolation for fast streamable 3DGS. However, these methods can overlook object-specific motion and introduce inconsistencies near object boundaries. We propose OCLDA-GS, an object-centric streaming framework that dynamically reallocates object and motion-aware Gaussian anchors to allocate anchor Gaussian support for objects undergoing large motion while at the same time, restricting the gaussian neighborhood within object groups for more precise object boundaries. Furthermore, it estimates object-wise rigid transformations (e.g. a hand moving) and leverages these estimates to learn per-anchor local deformation correction for small deformations (e.g. wrinkles on a shirt). Then, we conduct experiments on three dynamic multi-view video datasets and demonstrate improvements over the previous baselines alongside improved quality between refinement timesteps. Under matched refinement budgets, our method improves PSNR over Instant Gaussian Stream (IGS) by 0.169 dB on N3DV, 0.202 dB on Meeting Room and +0.199 on Google Immersive, while reducing the mean PSNR drop between refinement timesteps by 9.2% and 13.8% in N3DV and Meeting Room, respectively.
Introduction
3D Gaussian Splatting (3DGS) [13] allows for photorealistic reconstruction from RGB images and camera poses using 3D Gaussians as primitives. 4D Gaussian Splatting (4DGS) or Dynamic Gaussian Splatting adds spatiotemporal deformation in 3DGS. This enables realtime Free Viewpoint Video (FVV) for novel view synthesis [20, 33, 37]. FVV has applications in VR, immersive communication and sports viewing [1, 35]. Current Dynamic Gaussian Splatting methods can be divided into two broad categories: offline and online [8, 27, 33]. Offline methods optimize spatiotemporal deformation by learning the motion features throughout the entire video and emphasizes quality and refinement taking significant time and compute in the process [36, 37]. In the online streaming setting, the scene representation is updated as each set of new multi-view frames arrive and the central concern is to make that update computationally cheap while maintaining image fidelity [8, 27].
Recent time and storage-efficient Gaussian streaming methods rely on selection of subset of Gaussians called anchors, whose motion features are then interpolated to update the scene representation. Methods like IGS [35] and ReconGS [6] work under fixed computational budget controlled by the number of anchors and refinement per step or keyframes making them extremely effective for real-time dynamic scene streaming under limited computation bandwidth. IGS performs farthest point sampling (FPS) and periodic refinement on pre-specified timesteps called keyframes, while ReConGS [6] organizes anchors into an adaptive hierarchy for compact motion optimization and performs per-step refinements.
These methods assume neighboring Gaussians share the 3D deformation trajectories of each other. [11, 26] However, this assumption fails for Gaussians near object boundaries. For example, in the N3DV dataset (Figure 1a), when the chef’s hand moves near a stationary table, deformation via interpolated anchor neighborhood can induce incompatible motion in Gaussians belonging to the stationary table.

Figure 1. Object-centric anchors for streaming reconstruction. (a) Selected N3DV renders with dataset-average PSNR (dB). Below, anchors selected by respective methods are shown, colors show object groups. (b) Quality versus reconstruction time (in seconds per frame) in the N3DV dataset.
Existing anchor selection methods like FPS or hierarchical sampling also may not reflect the motion distribution of the scene accurately. Under a limited anchor budget, these methods can allocate excessive support to stationary regions. Some related object and motion-aware control-point methods address parts of this challenge [4, 9], but maintaining accurate, object-consistent motion with limited refinement remains challenging. Furthermore, motion within an object often comprises a shared rigid component and local non-rigid deformations (e.g., a moving body with wrinkling clothes). Accounting for both components is important for accurate motion prediction under limited refinement budgets.
We introduce Object-Centric Local Deformation-Aware Gaussian Streaming (OCLDA-GS). OCLDA-GS jointly addresses anchor selection, object-aware motion propagation, and local deformation correction. First, OCLDA-GS uses optical flow [28] observed over each completed keyframe interval to redistribute a fixed anchor budget across object groups, increasing support for groups with larger observed motion. Second, it restricts motion propagation to anchors within the same group, preventing cross-group interpolation for supported Gaussians. Finally, it fits a rigid transformation to the predicted anchor correspondences of each group and predicts an additional local deformation correction.
More broadly, OCLDA-GS highlights object-aware motion modeling as a promising direction for improving the quality–efficiency trade-off in Gaussian streaming. Extensive experiments on three datasets demonstrate competitive reconstruction quality and streaming efficiency in both in-domain and cross-domain settings. Temporal analyses against online streaming method like Instant Gaussian Stream (IGS) further show reduced quality degradation between keyframes. In summary, our contributions are:
Flow-guided anchor selection: We dynamically redistribute a fixed anchor budget across object groups based on observed optical flow, providing greater motion support for objects undergoing larger motion.
Object-constrained motion propagation: We restrict anchor-based motion interpolation to within object groups, preventing incompatible motion from propagating across object boundaries.
Rigid-conditioned deformation correction: We leverage object-wise rigid motion estimates to learn per-anchor local deformation corrections, accounting for local non-rigid deformations within objects without additional refinement.
Related Work
Dynamic novel view synthesis. NeRF established neural radiance fields as a representation for novel view synthesis [21]. Dynamic extensions model scene changes through deformation fields, scene flow, and higher-dimensional embeddings [18, 22, 23, 24]. HexPlane and K-Planes improve efficiency through factorized spatiotemporal features [2, 5]. With an explicit Gaussian representation and differentiable rasterization, 3DGS enables high-quality real-time rendering [13]. Dynamic Gaussian methods extend this representation using learned deformation fields and spatiotemporal Gaussian formulations [19, 33, 36, 37]. These advances improve dynamic scene representation and rendering, while online reconstruction additionally requires efficient updates as new observations arrive.
Efficient Streaming. StreamRF and ReRF investigate incremental learning and residual representations for dynamic radiance fields [16, 29], while NeRFPlayer and VideoRF explore decomposed fields and feature streams for compact video representation and playback [25, 30]. For Gaussian-based online reconstruction, 3DGStream trains scene updates on the fly [27], and QUEEN uses sparse, quantized Gaussian attribute residuals [8]. Complementary compression approaches, including 4DGC and GIFStream, target compact representations and efficient video delivery [10, 15]. HiCoM and ReCon-GS use hierarchical motion representations to reduce update and storage costs [6, 7]. IGS instead employs a generalizable anchor-driven motion predictor with periodic Gaussian refinement at keyframes [35]. OCLDA-GS builds on this inference-based setting to improve motion support and Gaussian deformation.
Structured motion and object-aware deformation. Embedded deformation represents motion through blended local transformations [26], and SC-GS uses sparse control points with learned interpolation and local rigidity regularization [11]. Dynamic 3D Gaussians encourage coherent tracking through local motion constraints [20]. MotionGS incorporates optical-flow-based motion guidance, while MoDec-GS uses global-to-local motion decomposition [14, 38]. Closer to object-aware streaming, H3D-DGS combines object-wise motion processing with flow-assisted control points [9], and ComGS uses motion-sensitive keypoints and a motion-aware influence field [4]. Building on these ideas, OCLDA-GS combines motion-aware anchor allocation, explicit group-restricted propagation, and learned rigid-motion-conditioned correction within keyframe-based streaming.
Method
We introduce Object-Centric Local Deformation-Aware Gaussian Streaming (OCLDA-GS). We present a visual overview of our method in Figure 2 and describe it component by component in the following subsections.
Figure 2. Overview of our method. (a) Flow-guided anchor selection within object groups. (b) Object-wise rigid motion estimation from AGM-Net predictions. (c) Learned local deformation corrections propagated to Gaussians within each object. (d) Improved rendering quality compared to Farthest Point Sampling (FPS)-based IGS.
Preliminaries. We follow the paradigm of keyframe to target frame prediction for streaming [35]. This paradigm propagates Gaussian motion from the latest refined keyframe to a target frame using a sparse set of Gaussian anchors. Predicted Gaussian representation is refined periodically at keyframes, thereby removing computation for per-frame refinement. Let denote the Gaussian representation at keyframe , where each Gaussian is parameterized by its 3D center , orientation quaternion , and other geometric and appearance attributes.
Let anchor Gaussians be indexed by . Let map anchor to its corresponding Gaussian index, such that its source position is . Given the target frame , a pretrained optical flow conditioned model predicts motion features for the selected anchors, which are interpolated to obtain per-Gaussian features . A linear motion head decodes the translation and rotation updates:
Here, denotes quaternion multiplication and norm denotes quaternion normalization. The predicted target position of anchor is .
Object Groups. We first obtain object-level segmentation of the Gaussian representation. Let denote the object semantic embedding feature of Gaussian , and let denote the mean semantic embedding of anchors in . Each Gaussian is assigned an object-group label , where denotes the number of identified objects and indicates an unknown group. Each anchor has object-group label . The anchor indices belonging to object are
Anchor Reselection. We address the problem of anchor allocation on static regions by periodic anchor redistribution during keyframe refinement. We redistribute the fixed anchor budget across object groups based on their size and observed motion. For each group , we estimate a normalized motion score from optical flow over the preceding refinement interval, compensating for global image motion and filtering unreliable correspondences. Specifically, let be the 75th percentile of flow magnitudes within group in camera view , after subtracting the component-wise median flow over reliable image pixels. We compute
where is the image diagonal, pixels, , and denotes the 95th percentile. The set includes input views with at least eight distinct reliable, visible projected pixels for the group. Let denote the number of Gaussians in group . We assign each group an allocation weight
The anchor budget is distributed proportionally to these weights, subject to group population limits. Within each group, anchors are selected using 3D farthest-point sampling on the refined Gaussian centers.
Object-level Rigid Motion Estimation. For each object , we estimate a rigid transformation that best approximates its motion predicted by the frozen optical flow model. Let denote the Gaussians which participate in deformation at keyframe . We consider the subset of object anchors . Using the source and predicted anchor positions, we estimate an object-level rotation and translation by solving
This least-squares problem is solved analytically using the Kabsch algorithm [12]. The resulting transformation approximates the dominant rigid motion of each object.
Local Deformation. Given the estimated rigid transformation , we compute the difference between the pretrained optical flow predicted anchor positions at frame and those obtained by applying the fitted rigid transformation to their positions at keyframe . For each anchor ,
The inverse rotation expresses this displacement in the source-frame axes.
We define the centered anchor position as . We condition a shared residual network on the frozen optical flow model features, anchor motion feature, encoded anchor position, rigid-relative displacement, semantic features, and temporal offset:
where denotes positional encoding and is the normalized temporal offset.
The residual network predicts an anchor-level translation correction:
Motion and Correction Propagation. To prevent cross-object motion mixing, we restrict the anchors contributing to in Eq. (1) to those belonging to the same object group as Gaussian . For each Gaussian with a known group label , we define its neighborhood as the eight nearest same-group anchors:
We assign larger weights to anchors closer to the Gaussian and normalize them within :
These weights interpolate the anchor motion features to obtain in Eq. (1), and propagate the learned corrections:
For supported Gaussians , we rotate the correction into world coordinates and add it to the position predicted by :
Training Objective
We train the residual network using a photometric reconstruction loss with magnitude and smoothness regularization:
Let denote the rendered image of the corrected Gaussian representation and the ground-truth image. The photometric loss is
For the eligible anchor set , let . The regularization terms are
where connects neighboring same-object anchors, denotes their spatial-semantic affinity, and is the normalization factor.
Experiments
Configuration and Implementation Details
Datasets. We evaluate on three datasets: N3DV [17], Meeting Room [16], and Google Immersive [1]. We train the residual heads on four N3DV scenes (Coffee Martini, Cook Spinach, Flame Steak, and Flame Salmon) and two Google Immersive scenes (Dog and Truck). Cut Roasted Beef and Sear Steak from N3DV, and Welder and Alexa Meade Face Paint 1 from Google Immersive, are held out for testing. Cross-dataset evaluation uses Discussion and Trimming from Meeting Room.
Baselines. We compare against four online methods: IGS [35], ReConGS [6], 3DGStream [27], and QUEEN [8]. We reproduce these methods using their released implementations and evaluate them on common held-out scenes, cameras, and frames. For context, we also include four offline methods: Real-Time 4DGS [37], 4D-GS [33], STG [19], and Swift4D [34]. Offline rows are shaded gray and serve as contextual references.
Implementation Details. Our implementation uses PyTorch and runs on an NVIDIA A100 GPU with 80 GB memory. is the frozen AGM-Net from IGS [35]. It receives four input views, and we use motion anchors with eight same-group neighbors per Gaussian, as defined in Eq. (9). The residual network contains two hidden layers of width 256 with SiLU activations. We use SAGA [3] to segment the Gaussian splat at timestep 0. Details on SAGA grouping are provided in the supplementary. We optimize Eq. (13) using AdamW with a learning rate of and weight decay of 0.05, setting , , and . During streaming, Gaussian refinement is performed every five frames, using 100 updates for N3DV, 200 for Meeting Room, and 300 for Google Immersive. IGS and Ours share the same initialization and refinement budget for each scene.
Evaluation Metrics. We report PSNR (dB) [31] and structural dissimilarity, DSSIM [32], to measure reconstruction quality. Seconds per frame (s/frm) denotes the average online reconstruction time per target frame, including periodic keyframe refinement and excluding initial Gaussian reconstruction. Higher PSNR and lower DSSIM and s/frm indicate better performance.
Results
In-domain Evaluation. We evaluate on held-out scenes from N3DV and Google Immersive for the in-domain evaluation. As shown in Table 1, our method achieves 33.046 dB on N3DV and 23.798 dB on Google Immersive, improving over IGS by 0.169 dB and 0.199 dB, respectively. Our method obtains the second-highest PSNR among the online methods on both datasets, outperforming IGS, ReConGS, and 3DGStream, while QUEEN achieves the highest PSNR.
| Method | # Params | # Online refinements | N3DV PSNR ↑ | N3DV DSSIM ↓ | N3DV s/frm ↓ | Meeting Room PSNR ↑ | Meeting Room DSSIM ↓ | Meeting Room s/frm ↓ | Google Immersive PSNR ↑ | Google Immersive DSSIM ↓ | Google Immersive s/frm ↓ | Average PSNR ↑ | Average DSSIM ↓ | Average s/frm ↓ |
| Offline methods | ||||||||||||||
| RT 4DGS [37] | 0 | 32.010 | 0.014 | 30.470 | 0.025 | 26.760 | 0.030 | 29.747 | 0.023 | |||||
| 4D-GS [33] | 2.46M | 31.150 | 0.016 | 31.275 | 0.024 | 23.990 | 0.079 | 28.805 | 0.040 | |||||
| STG [19] | 90 | 32.050 | 0.026 | 29.510 | 0.034 | 29.200 | 0.042 | 30.253 | 0.034 | |||||
| Swift4D [34] | 31.92M | 32.230 | 0.014 | 31.306 | 0.022 | 23.482 | 0.060 | 29.006 | 0.032 | |||||
| Online methods | ||||||||||||||
| ReConGS [6] | 0 | 2,994 | 32.253 | 0.021 | 7.45 | 30.425 | 0.028 | 4.44 | 21.890 | 0.103 | 7.95 | 28.189 | 0.051 | 6.61 |
| 3DGStream [27] | 1.99M | 2,994 | 31.740 | 0.022 | 8.27 | 30.686 | 0.027 | 5.54 | 22.381 | 0.079 | 12.62 | 28.269 | 0.043 | 8.81 |
| QUEEN [8] | 345.01M | 2,994 | 33.512 | 0.018 | 5.61 | 29.126 | 0.027 | 2.25 | 25.820 | 0.046 | 15.23 | 29.486 | 0.030 | 7.70 |
| IGS [35] | 21.90M | 594 | 32.877 | 0.020 | 2.00 | 30.088 | 0.028 | 1.96 | 23.599 | 0.081 | 4.55 | 28.855 | 0.043 | 2.84 |
| Ours | 27.27M | 594 | 33.046 | 0.019 | 2.30 | 30.290 | 0.027 | 2.18 | 23.798 | 0.081 | 5.50 | 29.045 | 0.042 | 3.33 |
Table 1. Quantitative comparison on N3DV, Meeting Room, and Google Immersive. Average is the equal-weight mean of the three displayed dataset results. Gray rows are offline methods; the remaining rows are online methods. Darker blue indicates the best, and second-best online results. Offline quality includes published reference results.
Cross-domain Evaluation. We evaluate the N3DV-trained checkpoint on Meeting Room, without further neural-weight training. Results in Table 1 demonstrate competitive transfer on Meeting Room. On Meeting Room, our method achieves 30.290 dB, improving over IGS by 0.202 dB and obtaining lower DSSIM. This shows improved cross-domain transfer of OCLDA-GS.
Reconstruction Efficiency. As discussed in § 3, we do Gaussian optimization only at keyframes and use neural prediction for intervening frames. On N3DV and Meeting Room, reconstruction requires 2.30 and 2.18 s/frm, respectively, compared with 2.00 and 1.96 s/frm for IGS. The added processing cost is approximately 0.30 and 0.22 s/frm, accompanying the quality improvements on these datasets. Across the completed comparisons in Table 1, our method has the second-lowest reported reconstruction time on every dataset. Our method achieves an average PSNR of 29.045 dB with 594 total refinements, outperforming IGS at the same refinement count and achieving the highest avThe average PSNR among online methods using the fewest refinements. QUEEN, the only online method with higher average PSNR (29.486 dB), uses 2,994 refinements, approximately 5.04× as ours.
Qualitative Results
Figure 3 compares our method with online baselines on selected examples from the five datasets. Our reconstructions show closer alignment with the ground truth around moving objects, with reduced ghosting and boundary artifacts, complementing the aggregate comparisons in Table 1.

Figure 3. Qualitative comparison of online methods across three datasets. Each row uses the same frame, viewpoint, and crop for all methods. Orange boxes mark regions enlarged in the insets, highlighting motion alignment and object boundaries.
Figure 4 illustrates anchor selection, group propagation, and local deformation correction on a Meeting Room sequence. Flow over the completed interval reallocates a fixed anchor budget at the current keyframe. Object group-constrained propagation preserves the recovered grouping, while the learned local deformation adjusts the propagated deformation. The final panel shows the reconstruction after standard target-keyframe refinement.

Figure 4. Flow-guided anchor selection and temporal propagation. Flow guides anchor reselection within a fixed budget. Colors denote SAGA groups; white points mark selected anchors. Markers and trails visualize anchor and Gaussian motion. Ours shows the refined target keyframe from an input-camera view. Reselection and refinement occur only at keyframes.
Ablation Study
Component contributions. Table 2 compares flow-guided anchor selection, object-constrained propagation, and learned local deformation correction. Object restricts motion propagation within object groups, Flow prioritizes moving objects during anchor allocation, and LD learns per-anchor corrections. The complete method improves PSNR over IGS by 0.169 dB on N3DV and 0.202 dB on Meeting Room. Flow-containing variants consistently improve Meeting Room reconstruction, while local deformation correction provides the highest N3DV PSNR when combined with FPS. Although component gains are not uniformly additive, the complete method remains within 0.013 dB and 0.002 dB of the highest-scoring configurations on N3DV and Meeting Room, respectively.
| Object | Flow | LD | N3DV PSNR ↑ | N3DV DSSIM ↓ | N3DV s/frm ↓ | Meeting Room PSNR ↑ | Meeting Room DSSIM ↓ | Meeting Room s/frm ↓ | Average PSNR ↑ | Average DSSIM ↓ | Average s/frm ↓ |
| × | × | × | 32.877 | 0.020 | 2.00 | 30.088 | 0.028 | 1.96 | 31.482 | 0.024 | 1.98 |
| ✓ | × | × | 32.882 | 0.020 | 2.20 | 30.116 | 0.028 | 2.14 | 31.499 | 0.024 | 2.17 |
| × | ✓ | × | 32.913 | 0.020 | 2.37 | 30.263 | 0.027 | 2.31 | 31.588 | 0.023 | 2.34 |
| × | × | ✓ | 33.059 | 0.020 | 2.28 | 30.109 | 0.028 | 2.24 | 31.584 | 0.024 | 2.26 |
| ✓ | ✓ | × | 32.927 | 0.020 | 2.33 | 30.267 | 0.027 | 2.27 | 31.597 | 0.023 | 2.30 |
| × | ✓ | ✓ | 33.018 | 0.019 | 2.29 | 30.292 | 0.027 | 2.30 | 31.655 | 0.023 | 2.30 |
| ✓ | × | ✓ | 32.921 | 0.020 | 2.33 | 30.163 | 0.028 | 2.16 | 31.542 | 0.024 | 2.25 |
| ✓ | ✓ | ✓ | 33.046 | 0.019 | 2.30 | 30.290 | 0.027 | 2.18 | 31.668 | 0.023 | 2.24 |
Table 2. Component ablation on N3DV and Meeting Room. Checkmarks indicate enabled components. The first row corresponds to IGS (FPS), and the last row to our full method. Average equally weights the two datasets. LD: Local Deformation
Table 3 compares Local Deformation (LD) and anchor reallocation. For LD configuration, No correction disables LD, while Rigid replacement uses estimated rigid motion only. LD MLP w/o Rigid learns per-anchor corrections without conditioning them on the estimated rigid rotations. For anchor reallocation, FPS selects anchors for spatial coverage, while Object based () allocates anchors by object size without motion guidance. Ours combines LD conditioned on rigid motion with motion-guided anchor reallocation, achieving the highest average PSNR of 31.668 dB.
| Configuration | PSNR ↑ N3DV | PSNR ↑ MR | Average PSNR ↑ | Average DSSIM ↓ |
| Local deformation configuration | ||||
| No correction | 32.927 | 30.267 | 31.597 | 0.023 |
| Rigid replacement | 32.988 | 30.204 | 31.596 | 0.024 |
| LD MLP w/o Rigid | 32.991 | 30.280 | 31.636 | 0.023 |
| Anchor Reallocation | ||||
| FPS | 32.882 | 30.116 | 31.499 | 0.024 |
| Object based () | 33.026 | 30.280 | 31.653 | 0.023 |
| Ours | 33.046 | 30.290 | 31.668 | 0.023 |
Table 3. Ablation of local deformation and anchor allocation on N3DV and Meeting Room (MR). Average metrics equally weight the two datasets. LD: Local Deformation
Cross-domain temporal robustness. Meeting Room is excluded from local deformation-head training, so its improvement over IGS demonstrates transfer beyond the training datasets without retraining the head. Beyond average quality, Figure 6 shows smaller PSNR degradation between keyframes. The mean drop in PSNR from frames 0 to 4 from keyframe decreases from 0.622 to 0.565 dB on N3DV and from 1.927 to 1.662 dB on Meeting Room, corresponding to reductions of 9.2% and 13.8%. Together with the full-sequence trajectories in Figure 5, these results support improved quality preservation between refinement updates, including on the unseen dataset.
Figure 5. Temporal reconstruction quality with selected online baselines. Faint traces show raw values; solid curves show a 45-frame EMA. Full comparisons appear in Table 1.
Figure 6. Mean PSNR drop relative to keyframes on N3DV and Meeting Room; lower is better.
Number of anchors. On cross-domain dataset Meeting Room evaluation, OCLDA-GS outperforms IGS by 0.113 dB using eight times fewer motion anchors. Table 4 shows consistent PSNR gains on N3DV, Meeting Room, and Google Immersive across all anchor budgets. This further demonstrates the advantage of anchor re-allocation under tighter budgets.
| Dataset | 1,024 IGS | 1,024 Ours | 4,096 IGS | 4,096 Ours | 8,192 IGS | 8,192 Ours |
| N3DV | 32.741 | 32.822 | 32.851 | 32.962 | 32.877 | 33.046 |
| Meeting Room | 29.860 | 30.201 | 30.057 | 30.263 | 30.088 | 30.290 |
| Google Immersive | 23.637 | 23.766 | 23.593 | 23.598 | 23.599 | 23.798 |
Table 4. Motion-anchor budget ablation across the datasets. PSNR (↑) averages evaluated targets per scene, followed by an equal-scene average within each dataset. Shading indicates the highest PSNR per dataset.
Refinement efficiency. Figure 7 examines the PSNR–runtime tradeoff across four keyframe refinement budgets. On N3DV, OCLDA-GS with 100 refinement steps achieves 32.995 dB at 2.13 s/frame, compared with 32.952 dB at 2.95 s/frame for IGS with 200 steps. This yields slightly higher PSNR with 27.8% less reconstruction time. On Meeting Room, Ours with 200 steps reaches 29.777 dB, within 0.057 dB of IGS with 400 steps, while taking 33.5% less time. These results indicate that the proposed propagation can reduce the refinement budget needed to obtain comparable PSNR.

Figure 7. Refinement budget and reconstruction quality. Numbers denote keyframe refinement steps.
Limitations
OCLDA-GS depends on the quality of object groups. Inaccurate grouping can limit object-aware motion propagation. More robust object grouping could alleviate this limitation. Similarly, the rigid-motion mechanism assumes that each object group results in a useful dominant rigid transformation, which may be inaccurate for articulated objects. Finally, our learned local deformation correction updates Gaussian positions only but does not correct errors in orientation, scale, and appearance.
Conclusions
We presented OCLDA-GS, an object-centric Gaussian streaming framework that combines flow-guided anchor selection, object-constrained motion propagation, and learned local deformation correction conditioned on object-wise rigid motion. Under a fixed anchor budget, these components improve motion modeling without per-frame Gaussian refinement. Experiments across three dynamic multi-view datasets demonstrate competitive reconstruction quality and efficiency, with improvements over IGS on N3DV and Meeting Room and reduced quality degradation between keyframes. These results highlight the importance of object-aware motion modeling for efficient Gaussian streaming.
LLM Usage Statement
OpenAI ChatGPT was used to obtain reviewer-style feedback on the manuscript or major portions of it. It assisted with editing and refining author-produced text, including related work and citation presentation, and with refining author-produced material concerning experimental design, execution, analysis of results, and figures or diagrams. For code and implementation, ChatGPT generated work that the authors reviewed, edited, or approved. No LLMs or generative AI tools were used for synthetic data, dataset creation, annotation, or developing research ideas, hypotheses, claims, or conclusions. The authors take full responsibility for all submitted content, regardless of any LLM or AI-tool use.
References
- [1]Michael Broxton, John Flynn, Ryan Overbeck, Daniel Erickson, Peter Hedman, Matthew DuVall, Jason Dourgarian, Jay Busch, Matt Whalen, and Paul Debevec. Immersive light field video with a layered mesh representation. ACM Transactions on Graphics, 39(4):86:1–86:15, 2020. 1, 5DOI
- [2]Ang Cao and Justin Johnson. HexPlane: A fast representation for dynamic scenes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 130–141, 2023. 2arxiv.org/abs/2301.09632
- [3]Jiazhong Chen, Jiemin Fang, Chen Yang, Lingxi Xie, Xiaopeng Zhang, Wei Shen, and Qi Tian. Segment any 3D Gaussians. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 1971–1979, 2025. 5arxiv.org/abs/2312.00860
- [4]Jiacong Chen, Qingyu Mao, Youneng Bao, Xiandong Meng, Fanyang Meng, Ronggang Wang, and Yongsheng Liang. Motion matters: Compact gaussian streaming for free-viewpoint video reconstruction. In Advances in Neural Information Processing Systems, 2025. 2, 3arxiv.org/abs/2505.16533
- [5]Sara Fridovich-Keil, Giacomo Meanti, Frederik Rahbæk Warburg, Benjamin Recht, and Angjoo Kanazawa. K-Planes: Explicit radiance fields in space, time, and appearance. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12479–12488, 2023. 2
- [6]Jiahe Fu, Qiankun Gao, Chengxiang Wen, Yanmin Wu, Siwei Ma, Jiaqi Zhang, and Jian Zhang. ReCon-GS: Continuum-preserved gaussian streaming for fast and compact reconstruction of dynamic scenes. In Advances in Neural Information Processing Systems, 2025. 2, 5
- [7]Qiankun Gao, Jiarui Meng, Chengxiang Wen, Jie Chen, and Jian Zhang. HiCoM: Hierarchical coherent motion for dynamic streamable scenes with 3D Gaussian Splatting. In Advances in Neural Information Processing Systems, 2024. 2DOI
- [8]Sharath Girish, Tianye Li, Amrita Mazumdar, Abhinav Shrivastava, David Luebke, and Shalini De Mello. QUEEN: Quantized Efficient ENcoding of dynamic gaussians for streaming free-viewpoint videos. In Advances in Neural Information Processing Systems, 2024. 2, 5arxiv.org/abs/2412.04469
- [9]Bing He, Yunuo Chen, Guo Lu, Qi Wang, Qunshan Gu, Rong Xie, Li Song, and Wenjun Zhang. H3D-DGS: Exploring heterogeneous 3D motion representation for deformable 3D Gaussian Splatting. In Advances in Neural Information Processing Systems, 2025. 2, 3arxiv.org/abs/2408.13036
- [10]Qiang Hu, Zihan Zheng, Houqiang Zhong, Sihua Fu, Li Song, Xiaoyun Zhang, Guangtao Zhai, and Yanfeng Wang. 4DGC: Rate-aware 4D Gaussian compression for efficient streamable free-viewpoint video. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 875–885, 2025. 2arxiv.org/abs/2503.18421
- [11]Yi-Hua Huang, Yang-Tian Sun, Ziyi Yang, Xiaoyang Lyu, Yan-Pei Cao, and Xiaojuan Qi. SC-GS: Sparse-controlled gaussian splatting for editable dynamic scenes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4220–4230, 2024. 2, 3arxiv.org/abs/2312.14937
- [12]Wolfgang Kabsch. A solution for the best rotation to relate two sets of vectors. Acta Crystallographica Section A, 32(5):922–923, 1976. 4
- [13]Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis. 3D Gaussian Splatting for real-time radiance field rendering. ACM Transactions on Graphics, 42(4), 2023. 1, 2arxiv.org/abs/2308.04079
- [14]Sangwoon Kwak, Joonsoo Kim, Jun Young Jeong, Won-Sik Cheong, Jihyong Oh, and Munchurl Kim. MoDec-GS: Global-to-local motion decomposition and temporal interval adjustment for compact dynamic 3D Gaussian Splatting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11338–11348, 2025. 3arxiv.org/abs/2501.03714
- [15]Hao Li, Sicheng Li, Xiang Gao, Abdououhaiti Batuer, Lu Yu, and Yiyi Liao. GIFStream: 4D Gaussian-based immersive video with feature stream. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 21761–21770, 2025. 2arxiv.org/abs/2505.07539
- [16]Lingzhi Li, Zhen Shen, Zhongshu Wang, Li Shen, and Ping Tan. Streaming radiance fields for 3D video synthesis. In Advances in Neural Information Processing Systems, pages 13485–13498, 2022. 2, 5arxiv.org/abs/2210.14831
- [17]Tianye Li, Mira Slavcheva, Michael Zollhöfer, Simon Green, Christoph Lassner, Changil Kim, Tanner Schmidt, Steven Lovegrove, Michael Goesele, Richard Newcombe, and Zhaoyang Lv. Neural 3D video synthesis from multi-view video. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5521–5531, 2022. 5
- [18]Zhengqi Li, Simon Niklaus, Noah Snavely, and Oliver Wang. Neural scene flow fields for space-time view synthesis of dynamic scenes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6498–6508, 2021. 2arxiv.org/abs/2011.13084
- [19]Zhan Li, Zhang Chen, Zhong Li, and Yi Xu. Spacetime gaussian feature splatting for real-time dynamic view synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8508–8520, 2024. 2, 5arxiv.org/abs/2312.16812
- [20]Jonathon Luiten, Georgios Kopanas, Bastian Leibe, and Deva Ramanan. Dynamic 3D gaussians: Tracking by persistent dynamic view synthesis. In International Conference on 3D Vision, 2024. 1, 3arxiv.org/abs/2308.09713
- [21]Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren Ng. NeRF: Representing scenes as neural radiance fields for view synthesis. In European Conference on Computer Vision, 2020.arxiv.org/abs/2003.08934
- [22]Keunhong Park, Utkarsh Sinha, Jonathan T. Barron, Sofien Bouaziz, Dan B Goldman, Steven M. Seitz, and Ricardo Martin-Brualla. Nerfies: Deformable neural radiance fields. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 5865–5874, 2021.arxiv.org/abs/2011.12948
- [23]Keunhong Park, Utkarsh Sinha, Peter Hedman, Jonathan T. Barron, Sofien Bouaziz, Dan B Goldman, Ricardo Martin-Brualla, and Steven M. Seitz. HyperNeRF: A higher-dimensional representation for topologically varying neural radiance fields. ACM Transactions on Graphics, 40(6), 2021.arxiv.org/abs/2106.13228
- [24]Albert Pumarola, Enric Corona, Gerard Pons-Moll, and Francesc Moreno-Noguer. D-NeRF: Neural radiance fields for dynamic scenes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10318–10327, 2021.arxiv.org/abs/2011.13961
- [25]Liangchen Song, Anpei Chen, Zhong Li, Zhang Chen, Lele Chen, Junsong Yuan, Yi Xu, and Andreas Geiger. NeRFPlayer: A streamable dynamic scene representation with decomposed neural radiance fields. IEEE Transactions on Visualization and Computer Graphics, 29(5):2732–2742, 2023.DOI
- [26]Robert W. Sumner, Johannes Schmid, and Mark Pauly. Embedded deformation for shape manipulation. ACM Transactions on Graphics, 26(3), 2007.
- [27]Jiakai Sun, Han Jiao, Guangyuan Li, Zhijie Zhang, Lei Zhao, and Wei Xing. 3DGStream: On-the-fly training of 3D Gaussians for efficient streaming of photo-realistic free-viewpoint videos. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20675–20685, 2024.arxiv.org/abs/2403.01444
- [28]Zachary Teed and Jia Deng. RAFT: Recurrent all-pairs field transforms for optical flow. In European Conference on Computer Vision, 2020.arxiv.org/abs/2003.12039
- [29]Liao Wang, Qiang Hu, Qihan He, Ziyu Wang, Jingyi Yu, Tinne Tuytelaars, Lan Xu, and Minye Wu. Neural residual radiance fields for streamably free-viewpoint videos. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 76–87, 2023.arxiv.org/abs/2304.04452
- [30]Liao Wang, Kaixin Yao, Chengcheng Guo, Zhirui Zhang, Qiang Hu, Jingyi Yu, Lan Xu, and Minye Wu. VideoRF: Rendering dynamic radiance fields as 2D feature video streams. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 470–481, 2024.arxiv.org/abs/2312.01407
- [31]Zhou Wang and Alan C. Bovik. Mean squared error: Love it or leave it? a new look at signal fidelity measures. IEEE Signal Processing Magazine, 26(1):98–117, 2009.DOI
- [32]Zhou Wang, Alan C. Bovik, Hamid R. Sheikh, and Eero P. Simoncelli. Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4):600–612, 2004.pubmed.ncbi.nlm.nih.gov/15376593
- [33]Guanjun Wu, Toaran Yi, Jiemin Fang, Lingxi Xie, Xiaopeng Zhang, Wei Wei, Wenyu Liu, Qi Tian, and Xinggang Wang. 4D Gaussian Splatting for real-time dynamic scene rendering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20310–20320, 2024.arxiv.org/abs/2310.08528
- [34]Jiahao Wu, Rui Peng, Zhiyan Wang, Lu Xiao, Luyang Tang, Jinbo Yan, Kaiqiang Xiong, and Ronggang Wang. Swift4D: Adaptive divide-and-conquer gaussian splatting for compact and efficient reconstruction of dynamic scene. In International Conference on Learning Representations, 2025.
- [35]Jinbo Yan, Rui Peng, Zhiyan Wang, Luyang Tang, Jiayu Yang, Jie Liang, Jiahao Wu, and Ronggang Wang. Instant Gaussian Stream: Fast and generalizable streaming of dynamic scene reconstruction via gaussian splatting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16520–16531, 2025.arxiv.org/abs/2503.16979
- [36]Ziyi Yang, Xinyu Gao, Wen Zhou, Shaohui Jiao, Yuqing Zhang, and Xiaogang Jin. Deformable 3D Gaussians for high-fidelity monocular dynamic scene reconstruction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20331–20341, 2024.arxiv.org/abs/2309.13101
- [37]Zeyu Yang, Hongye Yang, Zijie Pan, and Li Zhang. Real-time photorealistic dynamic scene representation and rendering with 4D Gaussian Splatting. In International Conference on Learning Representations, 2024.arxiv.org/abs/2310.10642
- [38]Ruijie Zhu, Yanzhe Liang, Hanzhi Chang, Jiacheng Deng, Jiahao Lu, Wenfei Yang, Tianzhu Zhang, and Yongdong Zhang. MotionGS: Exploring explicit motion guidance for deformable 3D Gaussian Splatting. In Advances in Neural Information Processing Systems, 2024.arxiv.org/abs/2410.07707


