WorksComputer Science
LingGS: Decoupling Feed-Forward Geometry Prediction from Camera Tracking for High-Fidelity Gaussian SLAM
Abstract
Achieving high-quality reconstruction and responsive camera tracking remains challenging for monocular RGB Gaussian splatting SLAM under a limited optimization budget. We present an online system that addresses this challenge by combining learned geometric priors with explicit geometric camera tracking, which can substantially outperform learned camera prediction in terms of tracking accuracy. A frozen streaming 3D foundation model serves as a virtual RGB-D sensor, providing depth estimation from monocular video. An explicit geometric solver then estimates camera motion through feature reprojection, and a streaming Gaussian mapper integrates the resulting observations into a dense scene representation. We further show that geometric tracking improves camera pose accuracy across representative streaming geometry backbones, demonstrating the broader applicability of our approach. Extensive experiments on indoor and outdoor datasets show that our method significantly improves reconstruction quality over existing approaches, achieving about 7 FPS on a single RTX 4090 and delivering 2-3 times the throughput of the evaluated prior SLAM systems.
Introduction
Monocular Gaussian SLAM aims to estimate the camera pose and build a renderable scene map online using only an RGB image stream. This capability supports immersive interaction, live scene capture, and remote interaction in AR/VR and robotics. 3D Gaussian splatting (GS) [16] provides an explicit scene representation with efficient rendering and differentiable mapping, making it a useful representation for dense SLAM. However, unlike RGB-D Gaussian SLAM methods [15, 35, 39, 24, 45] (Ha et al. 2024), without measured depth, the monocular GS SLAM system must jointly recover camera pose, scene geometry, and the appearance, within a limited computational budget.
Methods that rely on sparse feature points or incremental map optimization struggle to achieve high-fidelity mapping [22, 14]. Learned dense priors help fill this gap, but integrating them into a consistent map introduces further challenges. Dense depth prediction and camera joint optimization add online computational cost [11, 5], while per-frame depth predictions do not ensure cross-frame scale consistency and require additional scale recovery and geometric alignment [6] (Yu et al. 2026). Consistent depth and camera poses are both essential to this task. Depth determines where Gaussians are initialized, while camera poses determine whether observations from different frames align. Poor initialization requires additional map refinement, and inaccurate poses introduce conflicting observations that can blur the reconstructed scene. The challenge is therefore to obtain consistent dense depth and camera poses together within a limited online computational budget.
Streaming 3D foundation models offer a promising starting point. By maintaining temporal context, they predict consistent dense depth and camera poses as images arrive [47] (Su et al. 2026; X. Chen et al. 2026; L.-Z. Chen et al. 2026). Their depth predictions provide dense surface estimates for the Gaussian initialization, reducing the need to recover the scene structure from sparse feature points. Yet this advantage depends on the quality of the accompanying poses. We observe that directly using the models’ predicted poses can lead to cross-frame misalignment, limiting reconstruction quality even when useful depth estimates are available. To this end, we introduce an explicit geometry pose solver that combines 2D feature points with predicted depth and utilizes them to jointly constrain camera pose estimation.
Our system requires no additional training and runs at ∼7 FPS on a single RTX 4090 GPU. It achieves higher rendering quality than the previous baselines across indoor and outdoor datasets, with the lowest depth error and competitive tracking error. Overall, our main contributions are:
Depth-assisted geometric tracking for streaming 3D models. We introduce an explicit geometry solver for camera tracking that combines 2D feature points with predicted depth. Intensive experiments show that our geometry solver can provide better tracking than the learned streaming camera head.
An efficient monocular Gaussian SLAM system. We build a faster high-fidelity monocular GS SLAM system combining the geometric solver, a frozen streaming model, and a Gaussian mapper, leading to higher appearance quality. Across evaluated datasets, our system can improve rendering PSNR by up to 3.1 dB.
Generalization of geometry solver across streaming models. We apply our geometry solver to multiple representative streaming models. We show that the geometry solver reduces trajectory error by up to ∼97% relative to the models’ native camera head, demonstrating its effectiveness across different backbones and providing insight for future streaming model design.
Related Work
Visual tracking without depth prior. These methods recover scene geometry from RGB observations through visual tracking and map optimization. MonoGS Matsuki et al. [22] estimates poses by photometric alignment with the Gaussian map, making tracking dependent on the quality of the evolving reconstruction. Photo-SLAM Huang et al. [14] combines ORB-SLAM3 Campos et al. [2] with Gaussian mapping. CaRtGS Feng et al. [7], RP-SLAM Bai et al. [1], and SEGS-SLAM Wen et al. [34] further develop this architecture through improvements to computation allocation, map updates, and scene modeling. MGSO Y. S. Hu et al. [12] instead initializes Gaussians from geometry recovered by direct photometric odometry. Without learned dense geometric priors, surface recovery relies on multi-view image observations and parallax. Regions with weak visual constraints may remain insufficiently reconstructed at initialization, requiring interpolation or further geometric refinement during mapping.
Learned geometry for tracking and mapping. Learned depth and pointmap predictions provide richer surface information for tracking and mapping. Splat-SLAM Sandström et al. [26], DROID-Splat Homeyer et al. [11], and HI-SLAM2 W. Zhang et al. [40] integrate learned dense depth and trajectory refinement. MGS-SLAM P. Zhu et al. [46] uses multi-view depth supervision, while SplatMAP Y. Hu et al. [12] updates Gaussian positions using refined poses and depths from dense SLAM, and uses reliability masks to prune or add Gaussians. Some methods strengthen geometric consistency through map-to-tracker feedback or optical flow constraints [44, 27], and suppress unreliable observations in dynamic scenes [42] (M. Li et al. 2025). These designs improve geometric estimation, but dense tracking and iterative refinement add computation to the online pipeline, limiting their application. Feed-forward prediction offers a way to reduce this burden. StreamGS Y. Li et al. (2025) and Flash-Mono Z. Zhang et al. [41] predict poses and Gaussian attributes before limited map refinement, but require intensive model retraining. Pointmap and multi-view foundation models also support tracking and mapping in OpenGS-SLAM Yu et al. (2025), ARTDECO G. Li et al. (2026) and S3PO-GS Cheng et al. [5], but with a high tracking cost. GeoGS-SLAM Gao et al. [8], VGGT-GS SLAM Han et al. [10], and M Ren et al. [25] combine learned predictions with geometric or photometric optimization to refine poses and scene estimates by processing frames in batches. OTF Meuleman et al. [23] explores efficient incremental Gaussian construction, but loses the consistent geometry reconstruction.
Combining predicted geometry with explicit pose estimation is particularly relevant to our work. GigaSLAM Deng et al. [6] and KiloGS-SLAM Yu et al. (2026) use predicted depth with geometric solvers, while KiloGS-SLAM additionally activates learned pixel matching when needed, but they suffer from the scale drift. MonoEM-GS Kruzhkov et al. [17] uses the iterative closest point (ICP) to align predicted pointmap with the Gaussian representation. These approaches demonstrate the value of learned geometry for explicit camera estimation. Our work focuses on using streaming consistent depth predictions for camera tracking and Gaussian mapping.
Method
As illustrated in Figure 1, given a monocular RGB sequence , our system combines learned geometric priors with explicit geometric camera tracking to incrementally estimate camera poses and build a Gaussian scene representation. The LingBot-Map L.-Z. Chen et al. [3] first establishes a fixed camera calibration from the initial frames and subsequently provides depth as a virtual depth sensor. The geometric solver (§ 3.1) combines these geometric measurements with image features to estimate camera poses by minimizing reprojection error. The Gaussian mapper (§ 3.2) integrates through keyframe selection, Laplacian-guided Gaussian insertion, and 30 refinement steps using RGB and depth losses with keyframe replay.
Figure 1. Overview of our online Gaussian splatting SLAM system. (a) A frozen streaming foundation model estimates camera intrinsics from the initial RGB frames. (b) Predicted depth and image features support geometric camera tracking through correspondence matching and reprojection error minimization. (c) The Gaussian mapper integrates RGB frames, predicted depth, and estimated poses through keyframe selection, Laplacian-guided Gaussian insertion, and 30-step refinement using RGB and depth losses with keyframe replay.
Geometry Solver
Online Gaussian mapping requires a camera pose for each incoming frame to integrate its observations into a common reconstruction. We build our geometry solver on ORB-SLAM3 Campos et al. [2]; Huang et al. [14], using the depth predicted by the LingBot-Map as a virtual RGB-D measurement. At frame , the solver receives the current RGB image , the predicted depth map . Together with the sparse map established from previous observations, these inputs are used to estimate the world-to-camera pose , which transforms a world point into the current camera frame as , where . At initialization, the first camera frame defines the reconstruction frame, with . Feature locations with valid predicted depth are back-projected to initialize the sparse map. We retain ORB-SLAM3’s feature extraction, matching, and local bundle adjustment, while specifying how predicted depth contributes to pose estimation. In particular, depth enters the tracking objective as a direct observation with a constant relative-error noise model.
Camera pose estimation with a direct depth observation. Each frame extracts oriented FAST and rotated BRIEF (ORB) image features on an 8-level pyramid. Following ORB-SLAM3, we obtain a set of 2D–3D correspondences between image features and local map points using projection-guided matching. Keeping the map points fixed, we refine the current pose by minimizing a robust combination of image reprojection and depth residuals:
where is the pinhole projection with the frozen , extracts the third coordinate, is the model’s depth at the keypoint, is the Huber kernel, and
with the pyramid-level pixel standard deviation. Pose estimation alternates between optimization and outlier re-classification. In each of four outer rounds, the solver refines the current pose using the active feature correspondences, recomputes their residuals and updates their inlier status according to the weighted squared errors. This procedure reduces the influence of incorrect matches on subsequent optimization rounds. We retain the image-plane reprojection residuals and their analytical pose Jacobians from ORB-SLAM3. ORB-SLAM3 represents RGB-D measurements as virtual stereo observations, allowing it to reuse the stereo observation model and optimization framework, but this modeling choice becomes less natural when depth is supplied directly by a feed-forward geometry model rather than inferred from stereo disparity; more details are in Appendix § 6.§ 1. For the predicted depth, we instead adopt a constant-relative-uncertainty model and use a direct depth residual with the corresponding uncertainty weight. Our modification replaces the third, virtual-stereo residual with a direct depth residual and updates its Jacobian and information weight accordingly. We therefore describe only the modified depth component below.
For a matched map point , let denote the observed valid depth at its matched 2D image correspondence. Given a camera pose , the reprojected camera-space depth is . The direct depth residual is
To linearize this residual around the current pose, we introduce a local pose increment and apply a left perturbation , where
Here, and respectively denote the rotational and translational increments, following [29, 2].
The operator maps a six-dimensional increment to its matrix representation in , and denotes the skew-symmetric cross-product matrix. The observed depth and map point remain fixed during pose-only optimization, so only the reprojected camera-space depth depends on the increment. The corresponding Jacobian is therefore
This Jacobian describes how a small correction to the current pose changes the depth residual. The depth Jacobian is combined with the image-plane reprojection Jacobians to form the full residual Jacobian. Together with the information matrix and the Huber robust weights, these Jacobians define the linearized pose optimization problem. After solving for the increment , we update the pose as . This update is repeated within the optimization rounds described above, while keeping the map points and depth observations fixed like ORB-SLAM3.
Gaussian Mapper
The mapper incrementally reconstructs the scene as a set of 3D Gaussians,
where is the center, is a unit quaternion, contains the principal-axis scales, is the opacity, and is the RGB color. The map is updated at selected keyframes through render-aware Gaussian insertion followed by joint refinement with keyframe replay following OTF ([23]. All updates use only the current observation and previously stored keyframes.
Laplacian-guided Gaussian insertion
New Gaussians are allocated preferentially to image structure that is insufficiently represented by the current map. Let denote the rendered RGB image at pose , and denote the pixel. We use to denote the absolute discrete Laplacian of the grayscale image, normalized by its maximum, with a zero output for a zero Laplacian. The insertion weights are
where . This score favors valid pixels whose observed high-frequency response exceeds that of the rendering. It therefore provides a heuristic for identifying missing or underresolved detail, rather than a complete measure of photometric disagreement. For an empty map, is zero, and the weights reduce to the observed image Laplacian restricted to valid depth.
Geometric initialization
Each sampled pixel with depth initializes a Gaussian. Let denote its homogeneous image coordinate. The Gaussian center is obtained by back-projecting the depth and transforming the resulting point into the reconstruction frame. The remaining attributes are initialized as
The scale follows the pixel-footprint initialization of OTF, with a nominal image-space scale of two pixels. Although initialized isotropically, the three scale components are optimized independently. The low initial opacity permits newly inserted primitives to make a limited contribution before refinement.
Refinement with keyframe replay
After inserting new Gaussians, the mapper refines the Gaussian map for optimization steps. At each step, the newest keyframe is selected with probability through a dedicated sampling branch; otherwise, a keyframe is sampled uniformly from all stored keyframes, including the newest.
For the sampled keyframe , the differentiable gsplat rasterizer ((Ye et al., 2025)) produces a color image and a rendered depth map . We minimize
where is the mean absolute RGB reconstruction error. The depth loss compares the rendered depth with the stored model-predicted depth prior in inverse-depth space:
where denotes the set of valid depth-supervision pixels for keyframe . For small depth errors, the inverse-depth discrepancy is approximately the absolute depth error divided by the squared reference depth. This emphasizes nearby geometry relative to equally sized absolute depth errors at greater distances. Together with the photometric terms, the depth loss encourages appearance refinement to remain consistent with the predicted scene geometry.
Experiment
Setup and Evaluation
Datasets
We evaluate on four datasets, including indoor and outdoor datasets. 7-Scenes ([28]) has seven indoor RGB sequences with ground truth Kinect depth. Oxford Spires ([32]) consists of six outdoor clips of 1000–2001 frames selected from outdoor sites: Keble College-(02,03,04), Observatory Quarter-01, Christ Church-02, and Bodleian Library-02, with poses and depth reprojected from the point cloud of a terrestrial laser scan. From RE10K ([43]) we select 60 short indoor and outdoor real-estate clips with 71–279 frames. Eleven handheld sequences (fr1, fr2, fr3) are selected from the TUM-D ([30]) dataset. All frames are processed at 518x392 resolution across all datasets. We evaluate all methods using a single NVIDIA RTX4090 GPU with an AMD Ryzen Threadripper PRO 3995WX 64-Cores CPU.
Evaluation metrics
We evaluate novel-view synthesis (NVS) quality on held-out views using PSNR, SSIM, and LPIPS. We measure global trajectory accuracy using the root-mean-square Absolute Trajectory Error (ATE) after aligning the estimated trajectory to the reference trajectory with a Sim(3) Umeyama alignment ([33]). We also report the Relative Pose Error (RPE) between consecutive frames, including translational error (RPE) and rotational error (RPE). To evaluate reconstruction geometry, we compute the mean L1 error between the rendered and ground-truth depth on held-out views. Finally, we report end-to-end throughput in frames per second (FPS), calculated by dividing the number of frames in each sequence by its total reconstruction time.
Baselines
We compare against MonoGS ([22]), S3PO-GS ([5]), Photo-SLAM ([14]), and OTF ([23]). MonoGS uses 3D Gaussians for monocular tracking and mapping, while S3PO-GS targets monocular SLAM with globally scale-consistent Gaussian point maps. Photo-SLAM combines feature-based tracking with Gaussian splatting for photorealistic mapping, whereas OTF incrementally reconstructs large-scale scenes from unposed image sequences with large parallax. All baselines are evaluated using their publicly available implementations on identical input frames. To compare with Photo-SLAM fairly, we increase its mapping iterations from 8 to a maximum of 200 while retaining concurrent tracking and mapping, maintaining a processing rate of FPS.
Novel View Synthesis and Tracking
Table 1 shows that our method consistently achieves the best rendering quality on all datasets in terms of NVS metrics. It also maintains competitive trajectory accuracy, obtaining the best ATE on Oxford Spires and TUM-D, and the second-best ATE on 7-Scenes and RE10K. The few cases in which Photo-SLAM or S3PO-GS achieves slightly lower tracking error are consistent with their larger computational budgets for pose estimation: Photo-SLAM performs extensive local-map maintenance when they have larger mapping iterations, whereas S3PO-GS employs a computationally intensive tracking procedure that aligns each frame with its Gaussian point map. The qualitative comparisons in Figure 3 further show that our reconstructions preserve finer structures and sharper appearance, while several baselines exhibit noticeable blurring, artifacts, or geometric inconsistency. For 7-Scenes, Oxford Spires and TUM-D, we additionally visualize rendered depth. The 7-Scenes and TUM-D references provide dense depth, whereas the Oxford Spires reference is obtained by projecting the scanned point cloud and is therefore sparse. For a fair comparison, the same valid-depth mask is applied to all methods. The trajectory examples in Figure 4 confirm that our estimates remain among the most accurate across three datasets, omitting RE10K because it is too short, with particularly clear improvements on the Oxford Spires and TUM-D sequences.
| Method | (a) 7-Scenes | (b) Oxford Spires | ||||||||||||
| PSNR↑ | SSIM↑ | LPIPS↓ | ATE↓ | RPE ↓ | RPE ↓ | FPS↑ | PSNR↑ | SSIM↑ | LPIPS↓ | ATE↓ | RPE ↓ | RPE ↓ | FPS↑ | |
| MonoGS | 18.94 | 0.681 | 0.501 | 29.90 | 0.776 | 0.290 | 3.28 | 20.40 | 0.635 | 0.571 | 595.47 | 10.58 | 0.321 | 3.21 |
| S3PO-GS | 19.76 | 0.713 | 0.470 | 7.30 | 0.480 | 0.213 | 1.74 | 21.63 | 0.699 | 0.506 | <u>287.81</u> | 2.802 | 0.134 | 1.31 |
| Photo-SLAM | <u>21.91</u> | 0.741 | 0.400 | 19.78 | 0.566 | 0.205 | 5.99 | 22.05 | 0.717 | 0.465 | 567.71 | <u>2.308</u> | 0.074 | 5.99 |
| OTF | 20.50 | 0.744 | 0.443 | 39.34 | 0.801 | 0.269 | 7.57 | <u>23.19</u> | 0.713 | 0.466 | 624.06 | 8.592 | 0.231 | 8.24 |
| Ours | 24.48 | 0.839 | 0.293 | <u>8.58</u> | <u>0.524</u> | 0.189 | <u>6.99</u> | 26.35 | 0.817 | 0.332 | 73.81 | 0.943 | <u>0.095</u> | 6.40 |
| Method | (c) RE10K | (d) TUM-D | ||||||||||||
| PSNR↑ | SSIM↑ | LPIPS↓ | ATE↓ | RPE ↓ | RPE ↓ | FPS↑ | PSNR↑ | SSIM↑ | LPIPS↓ | ATE↓ | RPE ↓ | RPE ↓ | FPS↑ | |
| MonoGS | 22.97 | 0.793 | 0.309 | 10.56 | 0.726 | 0.113 | 1.21 | 14.31 | 0.481 | 0.601 | 92.17 | 0.94 | 0.571 | 3.33 |
| S3PO-GS | 23.12 | 0.797 | 0.298 | 3.47 | 0.553 | 0.130 | 1.29 | 17.00 | 0.621 | 0.557 | <u>56.57</u> | <u>0.74</u> | 0.387 | 1.30 |
| Photo-SLAM | 23.86 | 0.802 | 0.287 | 1.90 | 0.647 | 0.085 | 5.94 | 13.68 | 0.496 | 0.601 | 59.96 | 0.89 | 0.330 | <u>5.99</u> |
| OTF | <u>25.20</u> | <u>0.859</u> | <u>0.245</u> | 8.79 | 1.064 | 0.150 | <u>6.35</u> | <u>19.58</u> | <u>0.708</u> | <u>0.462</u> | 88.55 | 0.94 | <u>0.323</u> | 6.54 |
| Ours | 25.73 | 0.873 | 0.210 | <u>2.84</u> | <u>0.645</u> | <u>0.088</u> | 7.94 | 20.89 | 0.758 | 0.387 | 41.33 | 0.66 | 0.288 | 5.01 |
Table 1. Rendering quality, trajectory accuracy, and efficiency among all datasets. The best results are in bold, and the second-best results are underlined. ATE and RPE are reported in centimeters, and RPE is reported in degrees. For Photo-SLAM, it fails on the Pumpkin sequence of 7-Scenes and we only use 6 scenes for its average.

Figure 3. Qualitative rendering and depth comparison. One representative sequence is selected from each dataset. For 7-Scenes, TUM-D and Oxford Spires, we show both novel-view RGB rendering and rendered depth; RE10K is shown only in RGB because reference depth is unavailable. The 7-Scenes and TUM-D datasets provide dense reference depth, whereas the Oxford Spires reference depth is sparsely projected from the scanned point cloud. All methods are compared using the same valid-depth mask.

Figure 4. Qualitative trajectory comparison. We visualize one representative sequence from three datasets after trajectory alignment. Our estimated trajectories closely follow the ground truth. Because OTF is designed primarily for novel-view synthesis and estimates poses only for keyframes and held-out views, its trajectory error is computed only over this available pose subset rather than over all input frames.
Depth reconstruction accuracy. Table 2 complements the RGB evaluation by measuring the rendered depth. Our method achieves the lowest mean L1 error on all three datasets, demonstrating consistent geometric improvements across both indoor and outdoor environments.
| Method | 7-Scenes↓ | TUM-D↓ | Oxford Spires↓ |
| MonoGS | 0.52 | 1.29 | 9.12 |
| S3PO-GS | <u>0.17</u> | <u>0.57</u> | <u>4.47</u> |
| Photo-SLAM | 0.36 | 1.66 | 8.54 |
| OTF | 1.00 | 2.35 | 14.12 |
| Ours | 0.15 | 0.33 | 3.18 |
Table 2. Scale-aligned rendered-depth accuracy. Mean depth L1 error on 7-Scenes, TUM-D, and Oxford Spires. All values are reported in meters; lower is better.
Compared with S3PO-GS, the strongest baseline across all three datasets, our method reduces the depth error by 11.8% on 7-Scenes, 42.1% on TUM-D, and 28.9% on Oxford Spires, based on the rounded values reported in the table. Compared with Photo-SLAM, the error decreases from 0.36 to 0.15 m on 7-Scenes, from 1.66 to 0.33 m on TUM-D, and from 8.54 to 3.18 m on Oxford Spires. Notably, OTF achieves competitive novel-view rendering quality, but exhibits the highest depth error on all three datasets. This discrepancy highlights a limitation of assessing 3D Gaussian maps through RGB rendering alone: visually plausible novel views do not necessarily imply geometrically accurate reconstruction, since the observed appearance can still be explained by Gaussians that are incorrectly distributed in 3D space.
Ablation study
Ablation on camera tracking. Table 3 isolates the effect of the pose on the reconstruction. Four alternatives are compared with our solver: (A) Virtual stereo, the RGB-D front-end in ORB-SLAM3, which turns the predicted depth into a virtual right-image coordinate and optimizes the stereo residual shown in Eq. (13); (B) 2D reprojection, the same front-end with depth used only to back-project the new map points, so that tracking sees the 2D reprojection residual only, like the monocular RGB ORB-SLAM3; (C) a purely geometric point-to-plane ICP of the predicted depth against a local model and (D) the backbone’s own regressed camera head. Compared with virtual stereo (A), our direct-depth residual produces more accurate camera poses while preserving the same PSNR: ATE decreases from 79.5 to 73.8 cm, while translational RPE decreases from 1.123 to 0.943 cm. This result indicates that directly modeling the uncertainty of predicted depth provides a more effective geometric constraint than encoding it through a virtual-stereo observation. The comparison with 2D reprojection (B) further demonstrates the importance of explicitly including depth in pose optimization. Removing the depth residual reduces PSNR by 3.45 dB, increases ATE from 73.8 to 162.1 cm, and increases translational RPE from 0.943 to 4.723 cm. Predicted depth is therefore useful not only for initializing map points, but also for constraining the camera pose at every frame. The purely unoptimized CPU ICP variant (C) is also substantially less accurate than our solver, increasing ATE from 73.8 to 307.2 cm, translational RPE from 0.943 to 15.14 cm, and rotational RPE from to . This result suggests that point-to-plane alignment alone is less robust to predicted-depth noise and weak geometric constraints. Finally, the backbone camera head (D) is fast but produces substantially less accurate trajectories, with 5.98 cm ATE and 15.39 cm translational RPE. This suggests that its regressed poses are not as tightly coupled to the predicted scene geometry as those obtained through explicit geometric optimization. Our solver adds only 6.6 ms of tracking time relative to the camera head, while reducing ATE by 87.6% and translational RPE by 93.9%. The end-to-end throughput remains comparable, decreasing only from 7.04 to 6.40 FPS.
| Pose source | PSNR↑ | SSIM↑ | LPIPS↓ | ATE (cm)↓ | RPE (cm)↓ | RPE (°)↓ | Track (ms)↓ | FPS↑ |
| A: Virtual stereo | 26.35 | 0.815 | 0.334 | 79.5 | 1.123 | 0.104 | 16.2 | 6.20 |
| B: 2D reprojection | 22.90 | 0.709 | 0.446 | 162.1 | 4.723 | 0.140 | 13.9 | 6.51 |
| C: ICP | 21.37 | 0.648 | 0.482 | 307.2 | 15.14 | 0.482 | 138.6 | 5.47 |
| D: Camera head | 20.23 | 0.629 | 0.510 | 598.5 | 15.39 | 0.259 | 9.5 | 7.04 |
| Ours | 26.35 | 0.817 | 0.332 | 73.8 | 0.943 | 0.095 | 16.1 | 6.40 |
Table 3. Ablation on camera pose estimation over Oxford Spires. For variant C, we use Open3D’s CPU point-to-plane ICP as an unoptimized reference implementation; its runtime does not represent the computational limit of ICP.
Ablation on backbones. Table 4 shows that replacing the regressed camera head with the geometry solver consistently improves all trajectory metrics across the four streaming backbones on Kベル College-02. The solver reduces ATE by 31.0–96.7%, translational RPE by 64.3–97.1%, and rotational RPE by 55.9–78.8%. These gains require only 16.1 ms of tracking per frame, corresponding to an additional 5.5–11.9 ms over the camera heads. The results demonstrate that our solver transfers across different backbone architectures and produces camera motion that is more consistent with their predicted scene geometry at modest computational cost.
| Backbone | Pose source | ATE (m)↓ | RPE (cm)↓ | RPE (°)↓ | Track (ms)↓ |
| XStreamVGGT (Su et al., 2026) | Camera head | 23.277 | 48.95 | 0.4 | 9.6 |
| Geometry solver | 0.773 | 1.44 | 0.085 | 16.1 | |
| STream3R (Lan et al., [18]) | Camera head | 25.560 | 17.67 | 0.319 | 10.6 |
| Geometry solver | 2.332 | 1.74 | 0.098 | 16.1 | |
| TTT3R (X. Chen et al., [4]) | Camera head | 10.57 | 6.52 | 0.204 | 4.2 |
| Geometry solver | 7.29 | 2.33 | 0.09 | 16.1 | |
| Lingbot-map (L.-Z. Chen et al., [3]) | Camera head | 8.224 | 13.74 | 0.202 | 9.5 |
| Geometry solver | 1.224 | 0.758 | 0.059 | 16.1 |
Table 4. Geometry-based pose estimation across streaming backbones on Keble College-02.
Ablation on mapping iterations. The number of synchronous refinement steps per keyframe is the one mapper parameter that trades quality against speed. We show the ablation in Figure 2. Quality improves quickly and then saturates: 20→30 steps gain 0.93 dB, 30→75 a further ∼0.88 dB, and everything beyond 100 steps changes only marginally (27.11–27.15 dB). We report throughout as the speed-oriented operating point.
Figure 2. Ablation on refinement steps per keyframe. Means over the six Oxford Spires sequences. Rendering quality saturates at – (dashed line). The filled marker is the default used in all main tables.
Conclusion
We present LingGS, an online monocular Gaussian SLAM system that decouples feed-forward geometry prediction from camera tracking. A frozen streaming 3D foundation model serves as a virtual RGB-D sensor, while an explicit geometric solver estimates camera poses by jointly minimizing image reprojection and direct depth residuals. Combined with incremental Gaussian mapping, this design enables high-fidelity reconstruction. Experiments across indoor and outdoor datasets demonstrate improved rendering quality and geometric accuracy while maintaining efficient processing and competitive tracking accuracy. Further experiments across multiple streaming backbones support the broader applicability of our geometric solver.
Our current system does not address loop closure or dynamic scenes. Its reliance on ORB feature extraction and matching can also compromise tracking in repetitive or weakly textured environments, where correspondences may be ambiguous or reliable features scarce. Addressing these limitations with learned prior is an important direction for improving long-term consistency and robustness. More broadly, our findings motivate future streaming 3D models that integrate learned geometric priors with explicit geometric optimization to improve pose accuracy and geometric consistency in online reconstruction.
Appendix
Relation to ORB-SLAM3's Virtual-Stereo Model
ORB-SLAM3 handles RGB-D observations by converting depth measurements into virtual right-image coordinates and reusing its stereo reprojection model. Although the information matrix is defined in pixel coordinates and does not explicitly depend on depth, the projection model introduces a depth-dependent sensitivity to depth errors. We first elucidate the visual-stereo mechanism in ORB-SLAM3 and compare it with our direct depth residual.
Calibrated stereo geometry and observations. Consider a calibrated pinhole stereo pair with shared calibrated intrinsics and baseline . The right camera center lies at in the left camera frame, and the two camera frames have aligned axes.
Let denote a map point in homogeneous world coordinates. Under a world-to-left-camera pose , the point in left camera space is defined as:
The same point has coordinates in the right camera frame. Here, and are expressed in the same length unit.
The predicted image coordinates are
For a matched stereo feature, let denote its observed coordinates in the left image and its observed horizontal coordinate in the right image. Rectification makes corresponding points share the same vertical coordinate in the ideal model. ORB-SLAM3, therefore, uses the three-component observation and the residual
Explicit weighting in pixel coordinates. ORB-SLAM3 assigns the stereo observation a diagonal covariance model,
Here, is an assumed pixel localization standard deviation in base-level image coordinates. For a feature detected at pyramid level ,
where is the pyramid scale factor and is zero-based. We use .
This covariance is assigned from the feature scale, rather than estimated from its current reprojection residual. It contains no explicit depth-dependent term. The weighted error is used within the robust optimization and outlier-classification procedure. In particular, ORB-SLAM3 does not compute a separate depth variance and insert it into .
RGB-D observations and the virtual right coordinate
For RGB-D inputs, the RGB image takes the role of the left image. Given a valid observed depth , ORB-SLAM3 constructs a virtual right-image observation:
The depth denotes the coordinate along the camera’s optical axis and uses the same length unit as the baseline. No physical right image is required.
Define the left-image reprojection residuals and the direct depth residual as
The observed quantities are fixed, whereas the predicted coordinates and depend on . For readability, this dependence is omitted from the scalar residual notation.
Substituting Eq. (16) into the right-image reprojection residual gives
Writing
the virtual-stereo residual becomes
Its third component combines horizontal reprojection error with an inverse-depth discrepancy. It is not a depth or disparity residual alone.
Depth sensitivity induced by the residual. Substituting Eq. (20) into the pixel-weighted cost yields the exact expression
Expanding the square gives
This expression explains how depth affects the cost despite the depth-independent pixel information matrix. The depth residual is multiplied by before the residual is squared. Near a consistent estimate, , so that
Thus, at fixed calibration and feature pyramid level, the coefficient of the squared depth residual scales locally as . This is a property induced by the residual parameterization, not an additional information weight explicitly assigned by ORB-SLAM3. Furthermore, Eq. (22) contains a horizontal–depth cross term. The full virtual-stereo cost therefore cannot be identified with an independent scalar depth penalty.
Comparison with direct-depth weighting. Our per-frame pose-only formulation uses the direct depth residual as the third component:
as defined in Eqs. (1) and (2). Its nominal weighted cost is
Unlike the virtual-stereo formulation, the third residual does not contain the horizontal reprojection residual. Here, the depth-dependent information weight is explicitly specified:
It corresponds to an assumed constant relative depth uncertainty, . We use . The observed depth remains fixed during pose-only optimization, so this weight is also fixed within each optimization problem.
Thus, the two formulations introduce depth dependence in different ways. Virtual stereo uses a pixel-domain weight together with a projection sensitivity that induces a local scaling. Our formulation directly penalizes depth error with an explicitly chosen weight. We evaluate direct-depth, virtual-stereo, and reprojection-only tracking in § 4.3.
AI use statement
In this work, we used generative AI tools to provide feedback on research methodology and experiments and to assist with the implementation of methods. Additionally, we used generative AI tools for brainstorming, code editing, literature search, and improving the readability of the manuscript. All AI-assisted methodological suggestions, code, experiment results, and manuscript content were independently reviewed and verified by the authors. We take responsibility for the final content of this work.
Ethics statement
This work focuses on visual SLAM and 3D scene reconstruction using publicly available research datasets. It does not involve human-subject studies or the collection of personally identifiable information. We are not aware of any specific ethical concerns beyond the general considerations associated with computer vision and 3D reconstruction technologies.
Reproducibility statement
Our source code and configuration files will be publicly released upon acceptance of the paper. To facilitate reproducibility, we provide detailed descriptions of the proposed tracking and mapping pipelines, implementation details, experimental settings, and evaluation protocols in the main paper and appendix. Additional ablations and qualitative results are also provided to support the reported findings.
Acknowledgements
This work was supported by the Guangdong Basic and Applied Basic Research Foundation (No. 2026A1515011138) and the King Abdullah University of Science and Technology (KAUST) research fund under grant BAS/1/1680-01-01.
References
- [1]Bai, Lizhi et al. (2026). “RP-SLAM: Real-time Photorealistic SLAM with Efficient 3D Gaussian Splatting”. In: IEEE Transactions on Visualization and Computer Graphics 32.2, pp. 1452–1466.arxiv.org/abs/2412.09868
- [2]Campos, Carlos et al. (2021). “ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual-Inertial, and Multimap SLAM”. In: IEEE Transactions on Robotics 37.6, pp. 1874–1890.DOI
- [3]Chen, Lin-Zhuo et al. (2026). “LingBot-Map: Geometric Context Transformer for Streaming 3D Reconstruction”. In: arXiv preprint arXiv:2604.14141.arxiv.org/abs/2604.14141
- [4]Chen, Xingyu et al. (2026). “TTT3R: 3D Reconstruction as Test-Time Training”. In: ICLR.arxiv.org/abs/2509.26645
- [5]Cheng, Chong et al. (2025). “Outdoor Monocular SLAM with Global Scale-Consistent 3D Gaussian Pointmaps”. In: ICCV, pp. 26035–26044.arxiv.org/abs/2507.03737
- [6]Deng, Kai et al. (2025). “GigaSLAM: Large-Scale Monocular SLAM with Hierarchical Gaussian Splatting”. In: SIGGRAPH Asia Conference Papers, pp. 1–10.arxiv.org/abs/2503.08071
- [7]Feng, Dapeng et al. (2025). “CaRtGS: Computational Alignment for Real-Time Gaussian Splatting SLAM”. In: IEEE Robotics and Automation Letters 10.5, pp. 4340–4347.DOI
- [8]Gao, Ruilan, Letian Jin, and Yu Zhang (2026). “GeoGS-SLAM: Online Monocular Reconstruction Using Gaussian Splatting with Geometric Priors”. In: arXiv preprint arXiv:2607.11184.arxiv.org/abs/2607.11184
- [10]Han, Yuhang et al. (2026). “VGGT-GS SLAM: Uncalibrated Monocular Gaussian Splatting SLAM with Feed-Forward Priors”. In: arXiv preprint arXiv:2609.19628.arxiv.org/abs/2609.19628
- [11]Homeyer, Christian, Leon Begrinistain, and Christoph Schröner (2025). “DROMD-Splat: Combining end-to-end SLAM with 3D Gaussian Splatting”. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops.DOI
- [12]Hu, Yan Song et al. (2025). “MGSO: Monocular Real-time Photometric SLAM with Efficient 3D Gaussian Splatting”. In: ICRA, pp. 11061–11067.DOI
- [14]Huang, Huajian et al. (2024). “Photo-SLAM: Real-time Simultaneous Localization and Photorealistic Mapping for Monocular, Stereo, and RGB-D Cameras”. In: CVPR.arxiv.org/abs/2311.16728
- [15]Keetha, Nikhil et al. (2024). “SplaTAM: Splat, Track & Map 3D Gaussians for Dense RGB-D SLAM”. In: CVPR.arxiv.org/abs/2312.02126
- [16]Kerbl, Bernhard et al. (2023). “3D Gaussian Splatting for Real-Time Radiance Field Rendering”. In: ACM Trans. Graphics (SIGGRAPH) 42.4.arxiv.org/abs/2308.04079
- [17]Kruzhkov, Evgenii and Sven Behnke (2026). “MonoEM-GS: Monocular Expectation-Maximization Gaussian Splatting SLAM”. In: arXiv preprint arXiv:2604.10593.arxiv.org/abs/2604.10593
- [18]Lan, Yushi et al. (2026). “STream3M: Scalable Sequential 3D Reconstruction with Causal Transformer”. In: ICLR, pp. 42746–42768.arxiv.org/abs/2508.10893
- [22]Matsuki, Hidenobu et al. (2024). “Gaussian Splatting SLAM”. In: CVPR.arxiv.org/abs/2312.06741
- [23]Meuleman, Andreas et al. (2025). “On-the-fly Reconstruction for Large-Scale Novel View Synthesis from Unposed Images”. In: ACM Trans. Graph. (SIGGRAPH) 44.4.DOI
- [24]Peng, Zhexi et al. (2024). “RTG-SLAM: Real-time 3D Reconstruction at Scale using Gaussian Splatting”. In: ACM SIGGRAPH Conference Papers.
- [25]Ren, Kerui et al. (2026). “M³: Dense Matching Meets Multi-View Foundation Models for Monocular Gaussian Splatting SLAM”. In: arXiv preprint arXiv:2603.16844.arxiv.org/abs/2603.16844
- [26]Sandström, Erik et al. (2024). “Splat-SLAM: Globally Optimized RGB-only SLAM with 3D Gaussians”. In: arXiv preprint arXiv:2405.16544.arxiv.org/abs/2405.16544
- [27]Seo, Dong-Uk et al. (2026). “GaussianFlow SLAM: Monocular Gaussian Splatting SLAM Guided by GaussianFlow”. In: IEEE Robotics and Automation Letters 11.6, pp. 7644–7651.
- [28]Shotton, Jamie et al. (2013). “Scene Coordinate Regression Forests for Camera Relocalization in RGB-D Images”. In: CVPR, pp. 2930–2937.DOI
- [29]Solà, Joan, Jeremie Deray, and Dinesh Atchuthan (2018). “A Micro Lie Theory for State Estimation in Robotics”. In: arXiv preprint arXiv:1812.01537.arxiv.org/abs/1812.01537
- [30]Sturm, Jürgen et al. (2012). “A Benchmark for the Evaluation of RGB-D SLAM Systems”. In: IROS, pp. 573–580.DOI
- [32]Tao, Yifu et al. (2026). “The Oxford Spires Dataset: Benchmarking Large-Scale LiDAR-Visual Localisation, Reconstruction and Radiance Field Methods”. In: International Journal of Robotics Research 45.6, pp. 839–857.arxiv.org/abs/2411.10546
- [33]Umeyama, Shinji (1991). “Least-Squares Estimation of Transformation Parameters Between Two Point Patterns”. In: IEEE Transactions on Pattern Analysis and Machine Intelligence 13.4, pp. 376–380.DOI
- [34]Wen, Tianqi, Zhiang Liu, and Yongchun Fang (2025). “SEGS-SLAM: Structure-enhanced 3D Gaussian Splatting SLAM with Appearance Embedding”. In: ICCV, pp. 28103–28113.arxiv.org/abs/2501.05242
- [35]Yan, Chi et al. (2024). “GS-SLAM: Dense Visual SLAM with 3D Gaussian Splatting”. In: CVPR.arxiv.org/abs/2311.11700
- [39]Yugay, Vladimir et al. (2023). “Gaussian-SLAM: Photo-realistic Dense SLAM with Gaussian Splatting”. In: arXiv preprint arXiv:2312.10070.arxiv.org/abs/2312.10070
- [40]Zhang, Wei et al. (2025). “HI-SLAM2: Geometry-Aware Gaussian SLAM for Fast Monocular Scene Reconstruction”. In: IEEE Transactions on Robotics 41, pp. 6478–6493.
- [41]Zhang, Zicheng et al. (2026). “Flash-Mono: Feed-Forward Accelerated Gaussian Splatting Monocular SLAM”. In: ICLR.arxiv.org/abs/2604.03092
- [42]Zheng, Jianhao et al. (2025). “WildGS-SLAM: Monocular Gaussian Splatting SLAM in Dynamic Environments”. In: CVPR.arxiv.org/abs/2504.03886
- [43]Zhou, Tinghui et al. (2018). “Stereo Magnification: Learning View Synthesis using Multiplane Images”. In: ACM Trans. Graph. (SIGGRAPH) 37.4.arxiv.org/abs/1805.09817
- [44]Zhu, Fan et al. (2026). “MyGO-Splat: Multi-Objective Closed-Loop Geometric Feedback for RGB-Only Gaussian SLAM”. In: arXiv preprint arXiv:2606.29738.arxiv.org/abs/2606.29738
- [45]Zhu, Liyuan et al. (2025). “LoopSplat: Loop Closure by Registering 3D Gaussian Splats”. In: 3DV.
- [46]Zhu, Pengcheng et al. (2024). “MGS-SLAM: Monocular Sparse Tracking and Gaussian Mapping with Depth Smooth Regularization”. In: IEEE Robotics and Automation Letters 9.11, pp. 9486–9493.arxiv.org/abs/2405.06241
- [47]Zhuo, Dong et al. (2026). “Streaming Visual Geometry Transformer”. In: ICLR, pp. 88055–88072.

