betaFeedback

WorksComputer Science

HiP3R: Hierarchical Pose Graphs for Long-Horizon 3D Reconstruction

Abstract

Feed-forward models accurately estimate camera motion and 3D geometry from short video windows. However, computational and memory costs prevent direct inference on sequences containing thousands of frames. Long videos must therefore be processed as overlapping chunks. This division produces independent local reconstructions with inconsistent coordinate frames and duplicate estimates of shared cameras. Achieving global consistency requires propagating constraints from adjacent overlaps and distant revisits across the sequence. This requirement naturally motivates pose graph optimization. However, the choice of graph level creates a trade-off. Chunk-level graphs efficiently resolve scale drift and close large loops. Yet, their rigid transformations cannot correct internal errors or conflicting frame predictions. Conversely, frame-level optimization offers local flexibility, but scaling it globally is computationally expensive and can break metric scale. HiP3R resolves this trade-off using a hierarchical pose graph. HiP3R first optimizes a robust chunk-level Sim(3)\mathrm{Sim}(3) graph to establish the global layout and scale. It then holds this layout fixed and estimates one shared SE(3)\mathrm{SE}(3) camera per frame, turning overlapping window predictions into a single trajectory. Experiments across multiple benchmarks demonstrate the effectiveness of HiP3R in improving camera trajectories and 3D reconstruction. Our method applies to both chunk-based and streaming backends without additional training or fine-tuning. The codes are available at https://kkakkkka.github.io/HiP3R.

Introduction

Recovering globally consistent 3D geometry and camera trajectories from video is fundamental to computer vision. This capability supports autonomous navigation, robotic mapping, and spatial artificial intelligence. Recent 3D foundation models [28, 25, 26] have significantly advanced visual geometry. By jointly predicting camera poses and dense pointmaps in a single forward pass, they bypass classical handcrafted matching. These models achieve high accuracy on short video clips without per-scene calibration or optimization. However, the quadratic complexity of visual attention limits them to short temporal windows. Directly scaling transformers to extended multi-kilometer sequences remains too expensive.

To process long videos, modern systems adopt a divide-and-conquer strategy [6]. Long sequences are divided into overlapping video chunks, which are processed independently and stitched together. While chunking bounds GPU memory, sequential alignment introduces severe computational and numerical bottlenecks. Dense overlapping points share high spatial redundancy, making all-pixel alignment computationally expensive. Furthermore, pairwise alignment errors compound along the sequential chain, causing severe scale and orientation drift. Loop closures are essential to eliminate cumulative drift and enforce global topological consistency. However, when the chains are not yet aligned, applying multiple long-range loop constraints at once can make the non-convex optimization unstable. The solver may diverge or converge to poor local minima.

As shown in Figure 1, even after global chunk placement is resolved, a conflict remains at the frame level. Existing alignment methods [6, 33, 31] optimize similarity transformations per chunk. However, a single similarity transform preserves internal camera configurations up to scale. Consequently, it cannot resolve cross-window camera-pose inconsistency. Overlapping chunks produce multiple, conflicting pose estimates of the same frame under differing visual contexts. A chunk is an inference unit, whereas the true trajectory contains only one camera pose per frame. Arbitrarily selecting one prediction discards multi-view evidence and causes velocity jumps across chunk boundaries. Conversely, full bundle adjustment is too expensive, while relative motion alone drifts from the recovered global layout.

Overlapping predictions recover local geometry in inconsistent frames

Figure 1. Overlapping predictions recover local geometry in inconsistent frames. Coarse layout optimization corrects global placement and scale, but keeps each chunk’s internal cameras. Shared-camera trajectory refinement estimates one trajectory from predictions and within-chunk motion.

To overcome these challenges, we introduce HIP3R, a Hierarchical Pose-graph optimization method for long-horizon 3D Reconstruction. HIP3R decomposes reconstruction into coarse chunk layout recovery and fine frame trajectory refinement. The coarse stage optimizes a chunk-level Sim(3) pose graph over adjacent overlaps and distant loop closures. This stage establishes a globally consistent layout and eliminates accumulated scale drift. Holding this coarse layout fixed, the fine stage optimizes a frame-level shared-camera SE(3) pose graph, representing each physical frame with a single camera variable to resolve cross-window pose inconsistency. Placement priors keep that camera on the recovered layout, while within-chunk motion factors resolve conflicting predictions from overlapping windows. We evaluate HIP3R across seven public benchmarks spanning diverse indoor and outdoor environments. Under matched backends and windows, HIP3R improves long-horizon trajectory accuracy over loop-enabled VGGT-Long. Controlled ablations show that bounded point sampling reduces graph construction and optimization time with little change in accuracy. Additional evaluations show that the method also works with streaming backends such as LingBot-MAP.

We summarize our main contributions as follows:

  • We show that chunk-level similarity alignment alone cannot ensure cross-window camera-pose consistency, as overlapping windows may yield conflicting estimates for the same frame.

  • We develop a hierarchical pose graph that first recovers a globally consistent chunk layout and scale, then estimates one shared SE(3) camera pose for each frame.

  • We introduce progressive loop closure and bounded point sampling to achieve stable and scalable optimization on long sequences without retraining.

Related Work

Feed-forward 3D vision models. Feed-forward 3D models have emerged as an alternative to classical Structure-from-Motion (SfM). Pioneering architectures like DUSt3R [28] and MASt3R [12] directly regress dense pointmaps from uncalibrated image pairs. Recent multi-view transformers such as VGGT [25] and VGGT-Omega [26] extend feed-forward predictions across broader view collections. Subsequent models explore permutation invariance [29], sparse attention [22], and learned scene actions (Wang & Agapito, 2026). These models achieve high local geometric accuracy without per-scene numerical optimization. However, quadratic attention restricts them to short windows, so long sequences yield overlapping local reconstructions.

Streaming and recurrent video geometry. To extend temporal context beyond static windows, another line of research develops streaming and recurrent architectures. Early methods such as Spann3R (Wang & Agapito, 2025) and CUT3R [27] maintain persistent spatial memory states. Recent methods like StreamVGGT [34], STream3R [11], and WinT3R [14] process video frames causally with temporal key-value caching. LingBot-MAP [3] structures geometric context across anchor frames, local windows, and trajectory memory. Other approaches explore test-time training, memory compression, and feature retrieval [4, 32, 30, 9, 15, 35]. Causal streaming still accumulates drift on long trajectories. As post-processing, HIP3R adds global loops and per-frame pose consistency without retraining.

Multi-window fusion and Structure-from-Motion. Scaling feed-forward predictions to extended trajectories connects to classical global Structure-from-Motion and pose graph optimization. Chunk-based methods like VGGT-Long [6], VGGT-Align [33], and VGGT-Motion [31] align adjacent chunks via pairwise Sim(3) alignment. A rigid Sim(3) per chunk preserves internal cameras up to scale and cannot resolve overlapping-frame disagreements. GlueMap [16] couples feed-forward star reconstructions with global motion averaging and bundle adjustment over virtual tracks. HIP3R resolves overlapping camera predictions through a shared frame-level pose graph after chunk alignment, without optimizing 3D tracks.

Method

Problem Formulation

Let I={It}t=1T\mathcal{I} = \{I_t\}_{t=1}^{T} denote an input video sequence of TT frames. We partition I\mathcal{I} into KK overlapping chunks {Ck}k=1K\{\mathcal{C}_k\}_{k=1}^{K}, each of at most CC frames with overlap OO. Chunk starts advance by C−OC-O frames; the final chunk may be shorter than CC. Each chunk is processed independently by a feed-forward 3D vision model [26]. For chunk Ck\mathcal{C}_k, the backend predicts dense pointmaps Xk,iX_{k,i}, confidence maps γk,i\gamma_{k,i}, and camera-to-chunk poses Pk,iP_{k,i}. Here i∈{1,…,∣Ck∣}i \in\{1,\ldots,|\mathcal{C}_k|\} indexes local frames within the chunk. Let g(k,i)∈{1,…,T}g(k,i) \in\{1,\ldots,T\} map local frame (k,i)(k,i) to its global video index. The goal is to recover a single camera trajectory F={Ft∈SE(3)}t=1T\mathcal{F} = \{F_t \in\mathrm{SE}(3)\}_{t=1}^{T} together with consistent 3D pointmaps. Figure 2 gives an overview.

Decoupled optimization. Directly optimizing all camera and point variables across thousands of frames is expensive. Unconstrained bundle adjustment loses the efficiency of feed-forward models. HIP3R uses a two-level hierarchical pose graph. Stage I solves a chunk-level Sim(3) pose graph for a globally aligned layout (§ 3.2). Stage II then optimizes a frame-level SE(3)SE(3) pose graph with one shared camera variable per frame while holding the layout fixed (§ 3.3).

Coarse Layout Optimization with Progressive Loop Scheduling

We formulate global layout recovery as optimization over a similarity pose graph Gc=(Vc,Ec)\mathcal{G}_c = (\mathcal{V}_c, \mathcal{E}_c). Each node k∈Vck \in\mathcal{V}_c represents a chunk CkC_k and is associated with a similarity transformation Ak=(sk,Rk,tk)∈Sim⁡(3)A_k = (s_k, R_k, t_k) \in\operatorname{Sim}(3). This transformation maps local chunk coordinates to a common world coordinate system as Ak(x)=skRkx+tkA_k(x) = s_k R_k x + t_k. An edge (i,j)∈Ec(i,j) \in\mathcal{E}_c represents a measured relative transformation Z^ij\widehat{Z}_{ij} from the coordinate system of chunk ii to that of chunk jj. The corresponding predicted relative transformation is Aj−1AiA_j^{-1} A_i. Overlap edges connect consecutive chunks, while loop edges connect retrieved chunk pairs that observe the same physical scene. Relative constraints determine scale only up to a global factor, so we fix A1A_1 as the world frame and scale reference. Scale errors in independent chunk predictions accumulate on long trajectories, so we optimize the remaining sks_k jointly with rotation and translation.

Dense overlapping 3D points are spatially redundant, so all-pixel alignment is unnecessary. For adjacent chunks, identical pixels in overlapping frames yield paired 3D predictions. We estimate relative similarities with confidence-weighted iteratively reweighted least squares (IRLS) rather than unweighted Umeyama alignment on all dense pixels [6]. Each iteration runs a weighted Umeyama alignment [21] and updates Huber weights from spatial residuals. We bound alignment cost by limiting the number of point correspondences used in each fit.

Visual place recognition proposes candidate loop closures across temporal separations [8]. For each selected pair of revisiting frames, we combine temporal neighborhoods from both endpoints into one backend input. The resulting joint reconstruction connects the two visits in a common local coordinate system. We align the predicted geometry at each endpoint to the corresponding original chunk, obtaining transformations BiB_i and BjB_j from the joint reconstruction to chunks ii and jj, respectively. The relative loop measurement is Z^ij=BjBi−1\widehat{Z}_{ij} = B_j B_i^{-1}. We assign each loop constraint a bounded weight wijw_{ij} based on the registration residuals at its two endpoints, reducing the influence of poorly aligned loops. Appendix D specifies loop retrieval and window settings, point sampling, pairwise alignment, and loop weighting.

Combining sequential overlap and loop constraints, we define the objective over active edges Ec\mathcal{E}_c. Let Eijc=Z^ij−1Aj−1AiE_{ij}^c = \widehat{Z}_{ij}^{-1} A_j^{-1} A_i denote the relative similarity error. We define the residual vector as

rijc=(Log⁡SO(3)(R(Eijc)),τ(Eijc),log⁡s(Eijc))∈R7,(1)r_{ij}^c = \left(\operatorname{Log}_{\mathrm{SO}(3)}(R(E_{ij}^c)), \tau(E_{ij}^c), \log s(E_{ij}^c)\right) \in\mathbb{R}^{7}, \tag*{(1)}

where τ\tau extracts Euclidean translation error, and log⁡s\log s evaluates log-scale error. Because feed-forward backends predict geometry without analytical pose covariances, wijw_{ij} provides a data-driven confidence weight per edge. The coarse layout objective minimizes the robust loss

A⋆=arg min⁡A ∑(i,j)∈Ec∑ℓ=17ρδ(wij rij,ℓc),(2)A^\star= \underset{A}{\operatorname{arg\,min}}\ \sum_{(i,j)\in\mathcal{E}_c} \sum_{\ell=1}^{7} \rho_\delta\left(\sqrt{w_{ij}}\,r_{ij,\ell}^c\right), \tag*{(2)}

where ρδ\rho_\delta denotes the Huber penalty, applied coordinate-wise to suppress large outliers.

Activating all distant loops on an unaligned chain can make the non-convex optimization unstable. We initialize chunk placements from sequential alignments, then introduce loop constraints in order of increasing temporal span. Adjacent edges remain active throughout, and each stage starts from the solution of the preceding stage. When most loop edges have short temporal spans, additional stages introduce the longer-span constraints more gradually.

We solve Eq. (2) with bounded trust-region reflective least squares, limiting log-scale changes relative to initialization. Each edge residual depends only on the transformations of the two connected chunks, yielding a sparse Jacobian. We compute the corresponding Jacobian blocks using automatic differentiation. Appendix D provides the parameterization, scheduling rules, and restart settings.

Shared-Camera Trajectory Refinement

Given the recovered layout A⋆A^\star, overlapping frames still have multiple conflicting pose estimates. Selecting one prediction discards alternative observations and causes velocity jumps across chunk boundaries. We therefore introduce a shared camera-to-world variable Ft∈SE(3)F_t \in\mathrm{SE}(3) for each frame tt. We represent each camera pose Pk,iP_{k,i} by its rotation Rk,i∈SO(3)R_{k,i} \in\mathrm{SO}(3) and camera center ck,i∈R3c_{k,i} \in\mathbb{R}^3, writing Pk,i=[Rk,i∣ck,i]P_{k,i} = [R_{k,i} \mid c_{k,i}]. Its placement Qk,iQ_{k,i} in the world frame is

R(Qk,i)=RkRk,i,c(Qk,i)=skRkck,i+tk,(3)R(Q_{k,i}) = R_k R_{k,i}, \qquad c(Q_{k,i}) = s_k R_k c_{k,i} + t_k, \tag*{(3)}

with Ak=Ak⋆A_k = A_k^\star. The chunk scale applies to camera centers. The shared pose is rigid with no scale variable. For each shared camera, we use one coarse-placed prediction as its initial pose. The placement priors retain the predictions from every chunk that observes that frame.

As shown in Figure 2, the camera graph uses two types of geometric factors. For consecutive frames i,ji,j retained as graph nodes within chunk kk, define the relative motion measurement Mk,i→j=Qk,j−1Qk,iM_{k,i\to j} = Q_{k,j}^{-1}Q_{k,i}. With Ef=Mk,i→j−1Fg(k,j)−1Fg(k,i)E^f = M_{k,i\to j}^{-1}F_{g(k,j)}^{-1}F_{g(k,i)}, the relative motion edge residual is

Overview of HIP3R

Figure 2. Overview of HIP3R. Frozen backends predict overlapping chunks. Stage I builds a chunk-level Sim(3) graph from overlap and loop constraints to recover layout and scale. Stage II estimates one shared SE(3) camera per frame under that layout.

rk,i,jf=(Log⁡SO(3)R(Ef), τ(Ef))∈R6.(4)r^f_{k,i,j} = \left(\operatorname{Log}_{\mathrm{SO}(3)} R(E^f),\ \tau(E^f)\right) \in\mathbb{R}^6. \tag*{(4)}

Here τ\tau extracts Euclidean translation, and the rotation residual uses the standard rotation logarithm [19]. Each placed prediction also provides a unary chunk pose prior on its shared camera, with Ep=Qk,i−1Fg(k,i)E^p = Q_{k,i}^{-1}F_{g(k,i)}, and

rk,ip=(Log⁡SO(3)R(Ep), τ(Ep))∈R6.(5)r^p_{k,i} = \left(\operatorname{Log}_{\mathrm{SO}(3)} R(E^p),\ \tau(E^p)\right) \in\mathbb{R}^6. \tag*{(5)}

When a physical frame appears in multiple overlapping chunks, their respective chunk pose priors jointly constrain the same shared variable FtF_t. Relative motion edges connected to this node join local chunk trajectories across chunk boundaries. The fine objective therefore combines both relative motion constraints and placement priors,

F⋆=arg⁡min⁡F ∑(k,i,j)∈Ef∑ℓ=16ρδ(λf rk,i,j,ℓf(A⋆,F))+∑(k,i)∈Ep∑ℓ=16ρδ(λp rk,i,ℓp(A⋆,F)).(6)F^\star= \underset{F}{\arg\min}\ \sum_{(k,i,j)\in\mathcal{E}_f}\sum_{\ell=1}^{6}\rho_\delta\left(\sqrt{\lambda_f}\,r^f_{k,i,j,\ell}(A^\star,F)\right) + \sum_{(k,i)\in\mathcal{E}_p}\sum_{\ell=1}^{6}\rho_\delta\left(\sqrt{\lambda_p}\,r^p_{k,i,\ell}(A^\star,F)\right). \tag*{(6)}

Motion constraints preserve relative poses within each chunk, while placement priors keep the refined trajectory close to the recovered global layout. Initialization and solver settings are given in Appendix § 11, and closed-form Jacobian blocks in Appendix § 9.

For long sequences, we optimize a keyframe subgraph for efficiency. Let QtQ_t denote the coarse-placed pose used to initialize frame tt, and define its correction as Δt=FtQt−1\Delta_t = F_t Q_t^{-1}. We interpolate corrections between keyframes and set Ft=ΔtQtF_t = \Delta_t Q_t. This recovers a trajectory for all frames while retaining their local pose predictions as the baseline. To update the geometry, we first place each pointmap in world coordinates using Ak⋆A_k^\star, then apply the rigid correction Fg(k,i)Qk,i−1F_{g(k,i)}Q_{k,i}^{-1}. The corrected pointmaps follow the shared camera poses while preserving the predicted depth and recovered chunk scale. Appendix § 10 describes keyframe selection and correction interpolation.

Experiments

We evaluate trajectory accuracy across seven public benchmarks comprising 61 diverse sequences. The benchmark spans indoor room-scale environments, including 7-Scenes [18] and Neural RGB-D (Azizović et al., 2022). It also includes high-precision multi-scale scenes from ETH3D [17] and Tanks and Temples [10]. Finally, it includes extended outdoor driving and campus traversals from Oxford Spires [20], KITTI [7], and VBR [2]. Dense surface geometry is evaluated on ETH3D, Neural RGB-D, and 7-Scenes, which provide dense 3D ground truth. Appendix § 12 details dataset compositions and sequence splits. Baselines in Table 1 use released checkpoints and recommended window policies on identical sequences. VGGT-Long and VGGT-Align are evaluated from public implementations, while VGGT-Motion [31] is not publicly available.

MethodMean ATE ↓\downarrowMean RPErot\mathrm{RPE}_{\mathrm{rot}} ↓\downarrowMean RPEtrans\mathrm{RPE}_{\mathrm{trans}} ↓\downarrow
CUT3R [27]44.8944.4691.4600
TTT3R [4]23.7755.0770.8077
LoGeR [32]40.6912.0861.1035
VGGT [25]20.9143.1591.6396
ZipMap [9]28.2302.7920.9454
LingBot-MAP [3]8.2881.8181.4152
VGGT-Long [6]22.7131.4490.2537
VGGT-Align [33]25.5841.7142.3608
VGGT-Omega [26]18.5551.8200.9878
π3\pi^3 [29]24.0291.5920.3614
GlueMap [16]45.2811.5611.9132
Scal3R [30]27.6121.3760.1703
HiP3R (LingBot-MAP)4.0171.0550.1824
HiP3R (VGGT-Omega)4.9060.5900.0622

Table 1. Trajectory evaluation across indoor and outdoor benchmarks. Values are dataset-macro averages of ATE (m), RPErot\mathrm{RPE}_{\mathrm{rot}} (∘^\circ), and RPEtrans\mathrm{RPE}_{\mathrm{trans}} (m). Bold marks the best result.

We evaluate trajectory accuracy using ATE, RPErot_{\mathrm{rot}}, and RPEtrans_{\mathrm{trans}} (lower is better). ATE is translation RMSE after a per-sequence Sim(3) alignment that corrects rotation, translation, and scale. Relative errors use all consecutive frame pairs (Δ=1\Delta=1 frame). Rotation RPE is in degrees, and translation RPE applies the same Sim(3) correction. Pairwise pose AUC (higher is better) evaluates all i<ji < j camera pairs using the maximum of relative rotation and translation-direction angular errors at thresholds of 5∘5^\circ, 15∘15^\circ, and 30∘30^\circ. Reconstructed 3D pointmaps are evaluated using the mean of accuracy and completeness (Chamfer, meters) after Umeyama and ICP. F1 uses a hit distance of 0.25 m on ETH3D and 0.05 m on Neural RGB-D and 7-Scenes. Appendix H provides per-dataset geometric breakdowns.

We apply HiP3R to the camera poses and dense geometry produced by VGGT-Omega [26] and LingBot-MAP [3]. Both backends use the same graph formulation without fine-tuning. Unless otherwise stated, we use VGGT-Omega 1B with a chunk size of 60 frames and an overlap size of 30 frames. LingBot-MAP uses windowed inference with a chunk size of 120 frames and an overlap size of 32 frames.

We use the same refinement weights across backends. For short sequences, the fine stage optimizes all camera poses; for long sequences, it optimizes a keyframe subgraph and interpolates pose corrections for the remaining frames. Appendix details the keyframe selection rules and solver settings.

Benchmark Evaluation across Public Datasets

Table 1 reports tracking metrics macro-averaged across the seven datasets. With VGGT-Omega, HiP3R reduces ATE by 74%, from 18.555 to 4.906 m, RPErot\mathrm{RPE}_{\mathrm{rot}} from 1.820∘1.820^\circ to 0.590∘0.590^\circ, and RPEtrans\mathrm{RPE}_{\mathrm{trans}} by 94%, from 0.9878 to 0.0622 m. On long traversals, sequential stitching accumulates scale and orientation drift. Progressive loop scheduling in the coarse layout stage (§ 3.2) corrects global drift and drives much of the ATE reduction in Table 1. Shared-camera refinement (§ 3.3) further improves relative pose accuracy while preserving the layout. On compact indoor benchmarks such as 7-Scenes and Neural RGB-D, native VGGT-Omega already achieves ATE of 0.037 m and 0.019 m, leaving limited room for further global correction. With LingBot-MAP, HiP3R reduces macro ATE by 52%, from 8.288 to 4.017 m, without retraining.

Controlled Long-Horizon Reconstruction

We compare HiP3R with two VGGT-Long [6] variants, with and without loop closures. All three use the same VGGT-Omega backend and the default chunk and overlap sizes specified above. Table 2 reports ATE on KITTI, VBR, and Oxford and the mean over these three datasets. For reference, it also includes the released VGGT-Long and VGGT-Align [33] results using VGGT predictions.

BackendMethodOxfordKITTIVBRMean
VGGT-OmegaBaseline9.86177.4241.9143.06
VGGT-Long (w/o loops)1.777110.385.5065.86
VGGT-Long (w/ loops)1.66510.1941.2717.71
+ HiP3R (Ours)1.5799.73022.4411.25
VGGTVGGT-Long (released)9.45148.8499.8352.71
VGGT-Align8.56769.8099.8759.41

Table 2. Trajectory comparison on KITTI, VBR, and Oxford, alongside the long-horizon mean. ATE ↓

HiP3R achieves lower ATE than both VGGT-Long variants using VGGT-Omega on all three datasets. Loop closures reduce VGGT-Long ATE from 110.3 to 10.19 m on KITTI and from 85.50 to 41.27 m on VBR. HiP3R further reduces these errors to 9.73 and 22.44 m, nearly halving the remaining VBR error. VGGT-Long applies loops as rigid Sim(3) transforms to entire chunks. HiP3R instead jointly optimizes a sparse global layout graph with progressive loop scheduling. Appendix K shows that the complete solver further improves accuracy and convergence efficiency, while Appendix F reports stage-wise runtimes.

End-to-End Pose Accuracy and 3D Geometry

Inconsistent poses place walls and corridors at conflicting locations, causing duplicated surfaces and ghosting. Table 3 reports pose AUC and surface metrics on both backends. With VGGT-Omega, HiP3R reduces Chamfer by 64%, from 0.0759 to 0.0272 m, and raises F1 from 65.92 to 91.26. The stage-wise comparison in Appendix § 20, Table 21, shows that coarse alignment accounts for most of the Chamfer reduction, with improvements from shared-camera refinement on both backends.

MethodAUC@5 ↑\uparrowAUC@15 ↑\uparrowAUC@30 ↑\uparrowChamfer ↓\downarrowF1 ↑\uparrow
VGGT-Omega29.7750.2463.220.075965.92
HIP3R (VGGT-Omega)50.7567.9875.210.027291.26
LingBot-MAP17.8338.6153.190.142361.13
HIP3R (LingBot-MAP)22.0151.3566.900.104274.60

Table 3. Pose AUC and dense geometry on both backends.

VGGT-Omega OutputChamfer ↓\downarrowF1 ↑\uparrowLingBot-MAP OutputChamfer ↓\downarrowF1 ↑\uparrow
Native0.075965.92Native0.142361.13
Layout Only0.027391.20Layout Only0.104974.49
Full0.027291.26Full0.104274.60

Table 21. Dense geometry across optimization stages. F1 is reported in percent.

Qualitative Dense 3D Reconstruction

Figure 3 compares VGGT-Omega and HiP3R (VGGT-Omega) on Oxford Bodleian Library and Keble College. Sequential placement drifts across long temporal gaps, reconstructing unclosed loops as offset surfaces several meters apart. Coarse layout optimization aligns distant revisits and collapses these surfaces onto single facades. As shown in the local views, shared-camera refinement further reconciles overlapping camera predictions. Figure 4 shows four long sequences with identical VGGT-Omega points. Native placement stretches streets into duplicated structures, while HiP3R recovers closed street networks with lower trajectory error.

Qualitative comparison of input videos, global 3D reconstruction, and local reconstruction details

Figure 3. Qualitative comparison between VGGT-Omega chunk placement and HiP3R.

Figure 4. Trajectory comparison on four long sequences with identical VGGT-Omega points.

Ablation Studies

Ablations isolate point sampling, progressive loop scheduling, and placement-motion interaction.

Point sampling bound. Table 4 reports point sampling on cached predictions. Bounding sampled points cuts graph build and solve time from 12232.18 to 2004.20 s (83.62%), while ATE changes from 21.7751 to 21.7569 m. Total time includes measurement construction and layout solves.

ConfigurationATE ↓\downarrowGraph Build & Solve (s) ↓\downarrowFinal solve (s)
Baseline21.775112232.181404.54
+ Bounded sampling21.75692004.201345.43
+ Fit weighting21.77702077.491387.14
+ Progressive loops21.52411477.59688.02
Complete17.4708753.4341.28

Table 4. Ablation of point sampling and progressive layout solver on extended sequences. Rows add components cumulatively. Appendix K details all test conditions.

Progressive loop scheduling and solver stabilization. Table 4 compares residual-fit weighting, progressive loop scheduling, and the complete solver. Progressive loop scheduling reduces graph build and solve time from 2077.49 to 1477.59 s. The complete configuration reaches 17.4708 m ATE in 753.43 s, with a 41.28 s final solve. Progressive scheduling mainly reduces runtime in this comparison, while the complete configuration improves both runtime and trajectory accuracy. Appendix K details the additional solver components evaluated together in the complete configuration.

Interaction of placement priors and motion constraints. Table 5 compares four ways of forming a camera trajectory, Single, Unary, Motion, and Shared, from identical predictions and optimized coarse layouts, with Layout Only as a reference. Layout Only takes the first available prediction for each frame in chunk order. Single selects the prediction farthest from its chunk boundaries. Neither performs frame-level optimization. Unary uses overlapping pose observations in a robust unary-only objective without relative-motion factors. Motion uses within-chunk motion factors without placement priors. Shared combines both factors as in Eq. (6). On both backends, Shared reduces relative errors over Single and Unary, with a small ATE gain. Motion yields slightly lower relative errors, but without placement priors, ATE increases to 11.678 and 6.501 m. Placement priors preserve the recovered layout, while motion factors improve local smoothness.

VGGT-OmegaLingBot-MAP
ATERPErot_{\mathrm{rot}}RPEtrans_{\mathrm{trans}}ATERPErot_{\mathrm{rot}}RPEtrans_{\mathrm{trans}}
Layout Only4.91430.64120.08174.05171.18370.2126
Single4.90790.62380.06954.02561.11310.1930
Unary4.90670.60100.06744.01921.08710.1828
Motion11.67760.58770.06196.50071.04640.1816
Shared4.90610.58990.06224.01681.05490.1824

Table 5. Trajectory comparison on identical predictions and one optimized coarse layout.

Stage-wise results across layout recovery and trajectory refinement. Table 6 reports the Layout Only and Full outputs on identical VGGT-Omega predictions. Layout Only recovers global layout and scale, reducing ATE by 87% on KITTI, 84% on Oxford, and 46% on VBR. Full then adds shared-camera trajectory refinement on that fixed layout. It reduces RPErot_{\mathrm{rot}} by 24% on KITTI, 30% on Oxford, and 5% on VBR, and RPEtrans_{\mathrm{trans}} by 29%, 40%, and 22%.

DatasetStageATE ↓RPErot_{\mathrm{rot}} (°) ↓RPEtrans_{\mathrm{trans}} ↓InferenceStage solve
KITTI (T=4541T=4541)Baseline77.420.8952.358825.745–
KITTI (T=4541T=4541)Layout Only9.750 (−87%-87\%)0.202 (−77%-77\%)0.131 (−94%-94\%)896.7855.40
KITTI (T=4541T=4541)Full9.730 (−0.2%-0.2\%)0.154 (−24%-24\%)0.093 (−29%-29\%)896.7850.08
VBR (T≈12,383T\approx12{,}383)Baseline41.912.1170.2562283.438–
VBR (T≈12,383T\approx12{,}383)Layout Only22.44 (−46%-46\%)0.617 (−71%-71\%)0.075 (−71%-71\%)2511.460200.0
VBR (T≈12,383T\approx12{,}383)Full22.44 (−0.0%-0.0\%)0.587 (−5%-5\%)0.059 (−22%-22\%)2511.4600.16
Oxford (T≈320T\approx320)Baseline9.8615.7574.04453.297–
Oxford (T≈320T\approx320)Layout Only1.599 (−84%-84\%)0.579 (−90%-90\%)0.173 (−96%-96\%)54.9190.03
Oxford (T≈320T\approx320)Full1.579 (−1%-1\%)0.408 (−30%-30\%)0.104 (−40%-40\%)54.9190.02

Table 6. Stage-wise results on identical VGGT-Omega predictions. Layout Only is the direct coarse-layout output; Full adds shared-camera trajectory refinement on the same layout. Oxford and VBR timings are means per sequence. Inference times include all inference required for each output; solver times are reported per stage. Error subscripts give reductions relative to the preceding stage.

Conclusion

HIP3R turns overlapping local predictions into a globally consistent reconstruction of long video sequences through a hierarchical pose graph. The method first recovers global layout and scale through a chunk-level Sim(3) graph, then refines a shared-camera SE(3) graph with the chunk transforms held fixed. This separation allows distant loop closures to correct accumulated drift while reconciling conflicting predictions of the same camera across windows. Evaluations on indoor and outdoor benchmarks show improvements in overall trajectory accuracy and surface reconstruction with both VGGT-Omega and LingBot-MAP. Extending this hierarchy to online updates and improving its robustness to unreliable loop closures are promising directions for future work.

Appendix

In this supplementary material, we provide mathematical derivations, implementation details, and extended experimental analyses. Appendix B details closed-form Jacobian derivations and graph coloring. Appendix C specifies the keyframe subgraph and residual interpolation for long sequences. Appendix D details public solver settings and optimization hyperparameters. Appendix E provides dataset compositions and evaluation splits across all seven benchmarks. Appendix F breaks down stage-wise execution times across competing methods. Appendices G and H report per-dataset tracking and dense 3D geometry results. Finally, Appendices I through M present window diagnostics, solver ablations, and dense geometry comparisons.

Graph coloring and closed-form Jacobian blocks

This appendix provides mathematical derivations for the sparse Jacobian blocks in § 3.3. Derivative identities follow standard Lie-group calculus [19]. We first analyze Jacobian sparsity and compare finite-difference graph coloring with analytic derivatives. We then derive closed-form blocks for SE⁡(3)\operatorname{SE}(3) residuals and describe automatic differentiation for Sim⁡(3)\operatorname{Sim}(3) residuals.

Jacobian sparsity and manifold structure. Nonlinear least-squares iterations linearize r(x)r(x) at current poses. Jacobian J=∂r/∂xJ = \partial r/\partial x has one row per residual coordinate and one column per pose parameter. For a 20,000-frame graph, nn reaches 10510^5 and mm spans millions. A dense JJ consists mostly of zeros. A relative residual between frames aa and bb depends only on those two cameras under fixed chunk transforms.

Graph coloring baseline for finite differences. Without closed-form derivatives, JJ can be approximated via finite differences. Evaluating columns individually costs nn residual passes. Sparsity enables a more efficient approach. Two columns are structurally orthogonal if they share no residual rows. Perturbing orthogonal variables simultaneously isolates each column after differencing. Each residual experiences at most one perturbation [5]. Coloring the column-intersection graph groups mutually orthogonal columns together. The number of residual sweeps equals the chromatic number. This depends on graph topology instead of total parameter count. On a 735-frame scene, coloring cuts post-processing from 10,228 to 172 s. The closed-form derivatives below eliminate these colored sweeps for SE⁡(3)\operatorname{SE}(3) residuals.

Lie group calculus and rotation identities. Let [u]×[u]_\times denote the skew-symmetric matrix where [u]×v=u×v[u]_\times v = u \times v. Let Exp⁡\operatorname{Exp} and Log⁡\operatorname{Log} denote the SO⁡(3)\operatorname{SO}(3) exponential and logarithm. We use three standard Lie identities [19].

Exp⁡(ϕ+δ)≈Exp⁡(ϕ)Exp⁡(Jr(ϕ)δ),(7)\operatorname{Exp}(\phi+\delta) \approx\operatorname{Exp}(\phi)\operatorname{Exp}(J_r(\phi)\delta), \tag*{(7)}
Log⁡(RExp⁡(δ))≈Log⁡(R)+Jr−1(Log⁡(R))δ,Log⁡(Exp⁡(δ)R)≈Log⁡(R)+Jl−1(Log⁡(R))δ,AExp⁡(u)=Exp⁡(Au)A,A∈SO⁡(3).(8)\begin{aligned} \operatorname{Log}(R\operatorname{Exp}(\delta)) &\approx\operatorname{Log}(R) + J_r^{-1}(\operatorname{Log}(R))\delta,\\ \operatorname{Log}(\operatorname{Exp}(\delta)R) &\approx\operatorname{Log}(R) + J_l^{-1}(\operatorname{Log}(R))\delta,\\ A\operatorname{Exp}(u) &= \operatorname{Exp}(Au)A,\qquad A \in\operatorname{SO}(3). \tag*{(8)} \end{aligned}

The right Jacobian and its inverse have the closed forms (θ=∥ϕ∥)(\theta= \lVert\phi\rVert)

Jr(ϕ)=I−1−cos⁡θθ2[ϕ]×+θ−sin⁡θθ3[ϕ]×2,Jr−1(ϕ)=I+12[ϕ]×+(1θ2−1+cos⁡θ2θsin⁡θ)[ϕ]×2,(9)J_r(\phi) = I - \frac{1-\cos\theta}{\theta^2}[\phi]_\times+ \frac{\theta-\sin\theta}{\theta^3}[\phi]_\times^2,\qquad J_r^{-1}(\phi) = I + \frac{1}{2}[\phi]_\times+ \left(\frac{1}{\theta^2} - \frac{1+\cos\theta}{2\theta\sin\theta}\right)[\phi]_\times^2, \tag*{(9)}

and Jl−1(ω)=Jr−1(ω)TJ_l^{-1}(\omega) = J_r^{-1}(\omega)^\mathsf{T}. Identity (7) differentiates global rotation vectors, where frame tt carries xt=(ϕt,τt)∈R6x_t = (\phi_t,\tau_t) \in\mathbb{R}^6 with Rt=Exp⁡(ϕt)R_t = \operatorname{Exp}(\phi_t).

Closed-form factorized SE⁡(3)\operatorname{SE}(3) residual. Consider a fine edge between source aa and target bb with measurement M=(RM,tM)M = (R_M,t_M). Expanding M−1Fb−1FaM^{-1}F_b^{-1}F_a into rotation and translation yields

rR=Log⁡(RMTRbTRa),rt=RMT(RbT(τa−τb)−tM).(10)r_R = \operatorname{Log}\left(R_M^\mathsf{T}R_b^\mathsf{T}R_a\right),\qquad r_t = R_M^\mathsf{T}\left(R_b^\mathsf{T}(\tau_a-\tau_b)-t_M\right). \tag*{(10)}

This factorization provides favorable sparsity. Specifically, rRr_R never depends on translation, while rtr_t never depends on source rotation. A dense 6×126 \times12 local Jacobian exceeds residual requirements. After the coarse stage, MM remains constant.

Analytic rotation derivatives. Perturbing ϕa→ϕa+δ\phi_a \to\phi_a + \delta and applying (7) gives ER→ERExp⁡(Jr(ϕa)δ)E_R \to E_R \operatorname{Exp}(J_r(\phi_a)\delta), so by (8)

∂rR∂ϕa=Jr−1(rR)Jr(ϕa).(11)\frac{\partial r_R}{\partial\phi_a}=J_r^{-1}(r_R)J_r(\phi_a). \tag*{(11)}

Perturbing ϕb\phi_b gives RbT→Exp⁡(−Jr(ϕb)δ)RbTR_b^{\mathsf{T}} \to\operatorname{Exp}(-J_r(\phi_b)\delta)R_b^{\mathsf{T}}. Transporting the perturbation through RMTR_M^{\mathsf{T}} with (9) yields ER→Exp⁡(−RMTJr(ϕb)δ)ERE_R \to\operatorname{Exp}(-R_M^{\mathsf{T}}J_r(\phi_b)\delta)E_R and hence

∂rR∂ϕb=−Jl−1(rR)RMTJr(ϕb).(12)\frac{\partial r_R}{\partial\phi_b}=-J_l^{-1}(r_R)R_M^{\mathsf{T}}J_r(\phi_b). \tag*{(12)}

Each block multiplies at most three 3×33 \times3 matrices. These include the inverse log Jacobian, a frame-transport rotation, and the target right Jacobian.

Analytic translation derivatives. The translation residual is linear in translations.

∂rt∂τa=RMTRbT,∂rt∂τb=−RMTRbT,(13)\frac{\partial r_t}{\partial\tau_a}=R_M^{\mathsf{T}}R_b^{\mathsf{T}}, \qquad\frac{\partial r_t}{\partial\tau_b}=-R_M^{\mathsf{T}}R_b^{\mathsf{T}}, \tag*{(13)}

depending on rotation only through the target. Using −[u]×v=[v]×u-[u]_\times v=[v]_\times u,

∂rt∂ϕb=RMT[RbT(τa−τb)]×Jr(ϕb),∂rt∂ϕa=0.(14)\frac{\partial r_t}{\partial\phi_b}=R_M^{\mathsf{T}}\left[R_b^{\mathsf{T}}(\tau_a-\tau_b)\right]_\times J_r(\phi_b), \qquad\frac{\partial r_t}{\partial\phi_a}=0. \tag*{(14)}

A fine edge contributes five nonzero 3×33 \times3 blocks. The prior residual from Eq. (5) with placement Q=(RQ,tQ)Q=(R_Q,t_Q) factorizes similarly. Specifically, rRp=Log⁡(RQTRg)r_R^p=\operatorname{Log}(R_Q^{\mathsf{T}}R_g) and rtp=RQT(τg−tQ)r_t^p=R_Q^{\mathsf{T}}(\tau_g-t_Q). This yields two diagonal blocks.

∂rRp∂ϕg=Jr−1(rRp)Jr(ϕg),∂rtp∂τg=RQT.(15)\frac{\partial r_R^p}{\partial\phi_g}=J_r^{-1}(r_R^p)J_r(\phi_g), \qquad\frac{\partial r_t^p}{\partial\tau_g}=R_Q^{\mathsf{T}}. \tag*{(15)}

Sparsity pattern assembly and reuse. Quantities RtR_t and Jr(ϕt)J_r(\phi_t) appear in every factor incident to frame tt. They are computed once per iteration in batch and shared. Per-edge terms Jr−1(rR)J_r^{-1}(r_R) and Jl−1(rR)J_l^{-1}(r_R) are evaluated in batch from current residuals. The CSR sparsity structure depends only on graph topology, requiring a single initialization. Subsequent iterations update only numeric block values. Jacobian construction requires only a few batched 3×33 \times3 multiplications per edge, scaling linearly without residual sweeps.

Coarse Sim(3) residual formulation. Let Ai=(si,Ri,ti)A_i=(s_i,R_i,t_i) and Aj=(sj,Rj,tj)A_j=(s_j,R_j,t_j). Relative similarity Z^ij=(s^,R,t^)\widehat{Z}_{ij}=(\hat{s},R,\hat{t}) comes from confidence-weighted Umeyama alignment on overlap or loop pointmaps. The coarse residual in Eq. (2) is

Eij=Z^ij−1Aj−1Ai,r~ijc=wijrijc=wij[Log⁡SO(3)R(Eij)τ(Eij)log⁡s(Eij)]∈R7,(16)E_{ij}=\widehat{Z}_{ij}^{-1}A_j^{-1}A_i,\qquad\tilde{r}_{ij}^{c}=\sqrt{w_{ij}}r_{ij}^{c}=\sqrt{w_{ij}} \begin{bmatrix} \operatorname{Log}_{\mathrm{SO}(3)}R(E_{ij})\\ \tau(E_{ij})\\ \log s(E_{ij}) \end{bmatrix}\in\mathbb{R}^{7}, \tag*{(16)}

ordered as (ω,τ,log⁡s)(\omega,\tau,\log s) with ω∈R3\omega\in\mathbb{R}^{3} and translation τ∈R3\tau\in\mathbb{R}^{3}. Edge weights apply prior to the Huber penalty. Fixing A1A_1 leaves 7(K−1)7(K-1) active parameters. Huber loss is coordinate-wise with δ=1\delta=1. Adaptive Jacobi scaling normalizes parameter updates across rotation, translation, and scale.

Coarse Sim(3) blocks via automatic differentiation. Similarity composition couples scales, rotations, and translations between endpoints. We differentiate per-edge residuals using vectorized float64 automatic differentiation. This outputs one 7×147 \times14 block per edge, assembled into CSR format. The closed-form SE(3) blocks and automatically differentiated Sim(3) blocks match numerical derivatives within 1.1×10−81.1 \times10^{-8}. Because the fine stage holds A⋆A^\star fixed, chunk columns drop before the SE(3) solve.

Long-sequence keyframe subgraph

For sequences exceeding 2000 frames, optimizing camera variables at every single frame is computationally unnecessary. Camera motion across adjacent video frames is temporally smooth. Therefore, the fine stage solves the objective over an active keyframe subgraph. Dense per-frame corrections are recovered by interpolating their rotation and translation components.

Keyframe selection rules. Let TT denote frame count and s=5s=5 the default stride. Keyframe selection in this appendix uses zero-based frame indices. Target indices include {0,s,2s,…}\{0, s, 2s, \ldots\} and {T−1}\{T-1\}. Each chunk [a,b)[a,b) also contributes boundary frames aa and b−1b-1. For adjacent chunk overlap O={t:t∈Ck∩Ck+1}\mathcal{O}=\{t:t\in C_k\cap C_{k+1}\}, we check existing samples. If fewer than m=min⁡(3,∣O∣)m=\min(3,|\mathcal{O}|) indices lie in O\mathcal{O}, we insert mm linearly spaced frames. Within-chunk edges connect consecutive keyframes, and placement priors attach only to retained nodes.

Rotation and translation interpolation of pose corrections. Let QtQ_t denote the first-occurrence coarse pose for frame tt in chunk order. A solved keyframe pose FtiF_{t_i} defines correction Δti=FtiQti−1\Delta_{t_i}=F_{t_i}Q_{t_i}^{-1} in world coordinates. Between keyframes ti<ti+1t_i<t_{i+1} with weight α=(t−ti)/(ti+1−ti)\alpha=(t-t_i)/(t_{i+1}-t_i),

R(Δt)=Slerp⁡R(R(Δti),R(Δti+1),α),τ(Δt)=(1−α)τ(Δti)+ατ(Δti+1),(17)R(\Delta_t)=\operatorname{Slerp}_R\left(R(\Delta_{t_i}),R(\Delta_{t_{i+1}}),\alpha\right),\qquad\tau(\Delta_t)=(1-\alpha)\tau(\Delta_{t_i})+\alpha\tau(\Delta_{t_{i+1}}), \tag*{(17)}

and Ft=ΔtQtF_t=\Delta_tQ_t.

Stride sensitivity and comparisons. Overlap averaging computes consensus poses before applying residual interpolation, while omitting the fine optimization stage. Table 7 compares uniform s=10s=10 with motion-spaced s=10s=10. Motion spacing allocates targets by accumulated motion instead of frame indices. Table 8 further extends this evaluation across three long sequences. Table 9 evaluates dense versus keyframe graphs on the KITTI dataset.

Refinement graphATE ↓\downarrowRPErot↓_{\mathrm{rot}}\downarrowRPEtrans↓_{\mathrm{trans}}\downarrow∣K∣|\mathcal{K}|tft_f (s)
Overlap averaging9.7350.2020.131——
Uniform stride 109.7280.158 (−22%-22\%)0.094 (−28%-28\%)6050.055
Motion stride 109.7290.156 (−23%-23\%)0.094 (−28%-28\%)7250.104
Uniform stride 59.7300.154 (−24%-24\%)0.093 (−29%-29\%)10590.075

Table 7. Keyframe subgraph on KITTI after the same coarse A⋆A^\star.

SceneTTConsensus RPErot_{\mathrm{rot}}Consensus RPEtrans_{\mathrm{trans}}Uniform s=10s=10 RPErot_{\mathrm{rot}}Uniform s=10s=10 RPEtrans_{\mathrm{trans}}Uniform s=5s=5 RPErot_{\mathrm{rot}}Uniform s=5s=5 RPEtrans_{\mathrm{trans}}
campus_train012,0420.2040.1290.1220.1180.1190.117
spagna_train014,1410.5420.0480.5350.0350.5350.034
downtown222003.5090.0833.5070.0823.5070.082

Table 8. The same keyframe comparison on additional long sequences.

SceneTTDense FtF_t ATEDense FtF_t RPErot_{\mathrm{rot}}Dense FtF_t RPEtrans_{\mathrm{trans}}Uniform s=5s=5 ATEUniform s=5s=5 RPErot_{\mathrm{rot}}Uniform s=5s=5 RPEtrans_{\mathrm{trans}}
Courthouse11062.59910.820.5172.60010.810.517
downtown114273.2233.9400.08753.2233.9400.0876
KITTI45419.7310.1440.0869.7300.1540.093

Table 9. Dense fine graph versus uniform stride 5 on VGGT-Omega predictions and coarse A⋆A^\star.

Uniform stride s=5s=5 yields comparable or lower relative errors than s=10s=10 in these comparisons. On KITTI, solve times are 0.075 and 0.055 s, respectively.

Table 9 compares dense and keyframe graphs. On Courthouse and downtown1 (1106 and 1427 frames), stride 5 yields errors comparable to the dense graph. On KITTI (4541 frames), dense solving improves RPErot_{\mathrm{rot}} and RPEtrans_{\mathrm{trans}} by 6% and 8% with unchanged ATE. The default configuration retains the keyframe subgraph on long sequences.

Hyperparameters and solver settings

Table 10 lists the default settings for the main benchmark.

QuantityValue
VGGT-Omega default windowC=60C=60, O=30O=30 frames
LingBot-MAP default windowC=120C=120, O=32O=32 frames, windowed inference
Loop window20 frames per endpoint
SALAD retrievalsimilarity 0.85, top-5, NMS 25 frames, minimum chunk separation 1, one pair per chunk pair (first hit)
Loop-constraint addition4 configured span-ordered stages, expanded to at most 8 by adaptive continuation
Keyframe subgraph (T>2000T > 2000)uniform stride 5, all chunk endpoints, at least min⁡(3,∣O∣)\min(3, \lvert O\rvert) frames in each overlap OO, correction interpolation (Appendix § 10)
Refinement weights (Eq. (6))λf=1\lambda_f=1, λp=0.25\lambda_p=0.25, fixed across backends and windows
Coarse Sim(3)Huber fscale=1f_{\mathrm{scale}}=1, adaptive Jacobi scaling (x_scale='jac'), max_nfev=5000, ftol=xtol=gtol=10^{-8} (SciPy defaults)
Coarse continuationintermediate stages (100 evals), adaptive full solve (500), full fallback (5000)
Active-bound restartsat most 2 passes of 1000 evaluations, initial log-scale bounds ±log⁡100\pm\log100, recentered on restart
Shared-camera SE(3)fixed A⋆A^{\star}, Huber scale δ=1\delta=1, max_nfev=100, ftol=xtol=gtol=10^{-8}, translation_scale=1
Jacobianclosed-form 3×33 \times3 SE(3) blocks in CSR, coarse Sim(3) by float64 automatic differentiation

Table 10. Solver hyperparameters and optimization settings across backends. We use the public configurations of VGGT-Omega [26], LingBot-MAP [3], and SALAD [8].

Backend inputs. The graph uses camera poses and pointmaps from each inference chunk. When a backend provides depth rather than pointmaps, we unproject its depth predictions into 3D points. For LingBot-MAP, we decode camera-to-world poses and unproject depth within inference windows.

Trajectory evaluation. We compute trajectory errors with evo, enabling correct_scale for the per-sequence Sim(3) alignment used to evaluate ATE.

Point sampling and pairwise alignment. For adjacent alignment, we sample at most 2×1052 \times10^{5} point pairs evenly from the flattened arrays, excluding invalid points and pairs with non-positive confidence scores. Paired confidence scores are combined by their geometric mean. Alignment uses at most 10 IRLS iterations with an adaptive Huber threshold.

Loop-constraint weighting. The alignment fit statistic qijq_{ij} is the larger of the two endpoint IRLS residual 95th percentiles (p95p_{95}). If p95p_{95} is unavailable at either endpoint, we use the larger endpoint 90th percentile (p90p_{90}). We convert this statistic to a bounded edge weight,

wij=clip⁡(σσ+max⁡(qij,0),wmin⁡,wmax⁡)w_{ij}=\operatorname{clip}\left(\frac{\sigma}{\sigma+\max(q_{ij},0)},w_{\min},w_{\max}\right)

where clip⁡(x,a,b)=min⁡{b,max⁡{a,x}}\operatorname{clip}(x,a,b)=\min\{b,\max\{a,x\}\} restricts the weight to the interval [a,b][a,b]. We use σ=0.05\sigma=0.05, wmin⁡=0.05w_{\min}=0.05, and wmax⁡=1w_{\max}=1.

Camera initialization and placement priors. For each frame tt, we select its first occurrence in chunk order and apply the coarse placement to obtain QtQ_t. We initialize the shared camera as Ft=QtF_t=Q_t and use QtQ_t as the reference for correction interpolation in Appendix § 10. For every frame retained in the optimization, each chunk prediction Qk,iQ_{k,i} with g(k,i)=tg(k,i)=t contributes a separate placement prior to the same shared camera FtF_t, as defined in Eq. (5). The first camera is fixed to its coarse placement as the coordinate reference.

Solver configuration. Each free chunk is parameterized by an absolute rotation vector, a translation vector, and log-scale. Coarse graph optimization uses bounded trust-region reflective least squares. Adaptive Jacobi scaling accounts for differences in parameter sensitivity across rotation, translation, and scale. Initial log-scale bounds are centered on the sequential initialization. The fine stage holds the recovered chunk transformations fixed and optimizes camera poses, with the first camera held as the reference. Table 10 gives the loss scales, residual weights, evaluation budgets, and termination tolerances. Keyframe selection and correction interpolation follow Appendix § 10.

Adaptive schedule and restarts. Let LL denote the number of loop edges, nmin⁡n_{\min} the shortest span count, and S=4S = 4 the configured stage count. When loop spans vary and nmin⁡S≥L(S−1)n_{\min}S \geq L(S - 1), the schedule expands to min⁡(2S,L)\min(2S, L) stages. Longer-span constraints then enter through additional intermediate layouts. Intermediate stages and the adaptive full-graph solve use the separate budgets in Table 10. If the full solve reaches its evaluation ceiling without converging, the solver invokes the full-budget fallback. Active scale bounds trigger the restart routine instead. Restarts warm-start from the latest estimate and recenter scale bounds. They continue while the problem remains unconverged and bounds remain active, subject to the restart limits in Table 10.

Datasets and splits

We evaluate HiP3R across seven public benchmarks spanning 61 diverse sequences. Table 11 lists dataset compositions, sensor domains, and evaluation properties.

DatasetDomain#Seq.What it tests
7-ScenesIndoor RGB-DTextureless rooms, motion blur
ETH3DIndoor and outdoor38Mixed-scale laser-scanned scenes
Neural RGB-DIndoor RGB-DCluttered rooms, fine geometry
Tanks and TemplesOutdoor structures6Large objects and buildings
Oxford SpiresCampus indoor and outdoor9Loops and scale change (T≈320T \approx320)
KITTI seq00Driving1Long outdoor loop (T=4541T = 4541)
VBRCity-scale Rome7Extreme length (TT up to 18,846)
Total61

Table 11. Dataset composition and experimental splits.

Indoor room-scale and multi-scale sequences. These three datasets evaluate performance across local and multi-scale settings with varying sequence lengths and scene complexity. With the default VGGT-Omega chunk size of 60 frames and overlap size of 30 frames, 10 of the 11 ETH3D still sequences fit within a single chunk. For these sequences, the layout and shared-camera graphs remain inactive, and the Native, Layout Only, and Full trajectories coincide. Neural RGB-D spans 2–11 chunks per sequence, while 7-Scenes spans 3–6. 7-Scenes [18] consists of seven Kinect-captured rooms: Chess, Fire, Heads, Office, Pumpkin, Kitchen, and Stairs. ETH3D [17] includes high-resolution indoor and outdoor still sequences with laser-scanner ground truth, covering offices, reliefs, facades, playgrounds, and gardens. Neural RGB-D (Azinoǧlu et al., 2022) contains nine room-scale indoor scenes with structured-light depth, including kitchens, staircases, and apartments.

Outdoor structures and buildings. Tanks and Temples [10] evaluates Barn, Caterpillar, Courthouse, Ignatius, Meeting Room, and Truck.

Extended traversals and driving loops. Oxford Spires [20] provides nine campus sequences with LiDAR-inertial ground truth, subsampled to T≈320T \approx320 frames. The sequences mix outdoor courtyards, interiors, and staircases, with revisits after long temporal gaps. KITTI [7] uses odometry sequence 00 only (T=4541T = 4541), a standard benchmark for long driving loops. VBR [2] contains seven long urban sequences through Rome, 8,518–18,846 frames per sequence (dataset-mean T=12,383T = 12,383).

Additional keyframe and layout-solver experiments use the downtown1, downtown2, and downtown5 sequences from Droid-W [13].

Aggregation. Trajectory benchmark metrics are averaged within each dataset and then equally across the seven datasets. Dense geometry metrics use an equal-weight average over the 38 evaluated sequences, as detailed in Appendix § 15.

Reported pipeline-stage timings

Table 12 reports pipeline component times with inference on a single GPU and HiP3R graph solves on CPU. The HiP3R timings are calculated from Table 6, with tloop=tg−tgNt_{\mathrm{loop}} = t_g - t_g^{\mathrm{N}} and tsolve=tc+tft_{\mathrm{solve}} = t_c + t_f. For VGGT-Long, reported components include chunk inference, loop-window inference when enabled, and pairwise Sim(3) alignment. We report VGGT-Long with and without loops on the default VGGT-Omega windows, alongside its released configuration. GlueMap component times include Pi3 star forwards, tracking, and coarse mapping on one GPU.

MethodComponentOxford (T≈320T\approx320)KITTI seq00 (T=4541T=4541)VBR (T≈12,383T\approx12{,}383)
HiP3R (Ours)Local baseline chunk inference (tgNt_g^{\mathrm{N}})53.3825.72283.4
HiP3R (Ours)Loop-window inference (tloopt_{\mathrm{loop}})1.671.0228.0
HiP3R (Ours)CPU coarse + fine solves (tsolvet_{\mathrm{solve}})0.055.5200.2
HiP3R (Ours)Component sum55.0902.32711.6
VGGT-Long (w/ loops), on Omega windowsChunk inference57.2924.22664.0
VGGT-Long (w/ loops), on Omega windowsLoop-window inference0.575.3252.7
VGGT-Long (w/ loops), on Omega windowsChunk alignment (Sim(3) IRLS)139.92882.09489.5
VGGT-Long (w/ loops), on Omega windowsComponent sum197.63881.512,406.2
VGGT-Long (w/o loops), on Omega windowsChunk inference67.5910.12919.0
VGGT-Long (w/o loops), on Omega windowsChunk alignment (Sim(3) IRLS)185.62457.18237.7
VGGT-Long (w/o loops), on Omega windowsComponent sum253.13367.211,156.7
VGGT-Long (released)Chunk inference67.4794.62329.4
VGGT-Long (released)Chunk alignment (Sim(3) IRLS)169.22228.85036.7
VGGT-Long (released)Component sum236.63023.37366.2
GlueMap (Pi3 coarse)Star forward (Pi3)280.92913.27803.2
GlueMap (Pi3 coarse)VGGSfM tracking996.211,947.838,762.2
GlueMap (Pi3 coarse)Coarse mapping215.21935.85077.2
GlueMap (Pi3 coarse)Component sum1492.316,796.851,642.6

Table 12. Mean pipeline component times (s) on Oxford, KITTI, and VBR. For HiP3R, component sums include chunk and loop inference and coarse and fine solves.

The large alignment times in VGGT-Long show pairwise 3D point alignment to be a dominant computational bottleneck. Contrastly, HiP3R accelerates pairwise alignment via bounded sampling while using a fast sparse graph solver. Dense point tracking accounts for most of GlueMap runtime.

Per-dataset comparison

Table 1 in the main text summarizes macro-averaged tracking metrics across all seven public benchmarks. Table 13, Table 14, and Table 15 report detailed per-dataset tracking errors for ATE, RPErot_{\mathrm{rot}}, and RPEtrans_{\mathrm{trans}}, respectively. In each column, bold and underline mark the best and second-best results across all evaluated methods. On VGGT-Omega, HiP3R improves relative rotation and translation accuracy across all seven datasets. Trajectory ATE improvements are largest on extended outdoor traversals (KITTI and VBR). In these environments, global loop constraints effectively eliminate accumulated drift. On compact indoor scenes, baseline predictions already achieve centimeter-level trajectory accuracy. For LingBot-MAP, HiP3R acts as a modular post-processing method. It reduces macro-averaged ATE from 8.288 to 4.017 m.

Method7SETHKITTINRGBDOxfordT&TVBR
VGGT-Omega ([26])0.03700.11877.420.01909.8610.51841.914
π3\pi^{3} ([29])0.0574<u>0.104</u>68.220.024412.060.46787.269
GlueMap ([16])0.06240.810193.50.023216.65<u>0.521</u>105.4
Scal3R ([30])0.07200.23686.250.04714.5730.572101.535
VGGT-Long ([6])0.07430.20448.840.03749.4510.55099.832
VGGT-Align ([33])0.07310.20969.800.03748.5670.53799.867
LingBot-MAP ([3])0.07890.44123.400.05542.3910.51831.129
ZipMap ([9])0.07020.236127.220.073528.330.69240.987
VGGT ([25])0.07050.23779.870.16421.400.53644.123
LoGeR ([32])0.1180.370174.960.1385.5221.078102.650
TTT3R ([4])0.1041.18946.930.19112.511.302104.199
CUT3R ([27])0.2961.376190.190.54616.261.839103.748
HiP3R (VGGT-Omega)<u>0.0381</u>0.0546<u>9.730</u><u>0.0219</u>1.5790.47622.443
HiP3R (LingBot-MAP)0.09010.4886.0630.126<u>2.346</u>1.02617.978

Table 13. Per-dataset ATE on the seven public datasets (lower is better). NRGBD is Neural RGB-D, 7S is 7-Scenes, T&T is Tanks and Temples. Bold and underline mark the full-table best and second-best values in each column.

Method7SETHKITTINRGBDOxfordT&TVBR
VGGT-Omega ([26])0.5010.8680.8950.1465.7572.4582.117
π3\pi^{3} ([29])0.510<u>0.729</u>0.218<u>0.0999</u>6.9122.036<u>0.636</u>
GlueMap ([16])0.5071.7550.5350.09844.3372.3081.388
Scal3R ([30])0.5202.036<u>0.215</u><u>0.0957</u>2.193<u>1.985</u>2.589
VGGT-Long ([6])0.5351.2570.2640.1185.1352.0270.806
VGGT-Align ([33])0.5361.2582.1410.1185.1092.0270.808
LingBot-MAP ([3])0.5793.3451.0510.2541.1062.1624.226
ZipMap ([9])0.6321.0981.4950.16111.842.5311.786
VGGT ([25])0.6051.2842.1860.23212.302.8492.656
LoGeR ([32])0.7021.7912.5520.407<u>2.185</u>3.0433.919
TTT3R ([4])0.71121.190.4410.3054.7983.4574.638
CUT3R ([27])0.90710.916.4850.8514.6442.9794.510
HiP3R (VGGT-Omega)0.4680.4760.1540.07100.4081.9650.587
HiP3R (LingBot-MAP)0.5862.2870.2750.280<u>1.080</u>2.2090.668

Table 14. Per-dataset RPErot_{\mathrm{rot}} on the same seven datasets (lower is better). Bold and underline mark the full-table best and second-best values in each column.

Method7SETHKITTINRGBDOxfordT&TVBR
VGGT-Omega ([26])0.01720.1012.3580.01094.0440.1280.2557
π3\pi^{3} ([29])0.0158<u>0.091</u><u>0.150</u>0.00911.999<u>0.106</u>0.1591
GlueMap ([16])0.01450.5831.8050.00792.2680.1158.599
Scal3R ([30])0.01760.2380.3350.0092<u>0.259</u>0.1070.2260
VGGT-Long ([6])0.01880.1900.2210.00980.9970.1100.2292
VGGT-Align ([33])0.01880.19114.880.00981.0530.1100.2633
LingBot-MAP ([3])0.02050.4956.0040.01890.5230.1242.7208
ZipMap ([9])0.02170.2393.0390.01603.0050.1440.1530
VGGT ([25])0.02210.2444.7080.01965.9550.1650.3632
LoGeR ([32])0.02230.2894.1100.01800.5470.1822.5563
TTT3R ([4])0.02631.2290.3290.02390.8030.1813.0620
CUT3R ([27])0.03510.7994.9640.03971.0540.1723.1564
HIP3R (VGGT-Omega)<u>0.0148</u>0.05670.09310.00490.1040.1030.0591
HIP3R (LingBot-MAP)0.02110.4650.2310.01960.2850.147<u>0.108</u>

Table 15. Per-dataset RPEtrans\mathrm{RPE}_{\mathrm{trans}} on the same seven datasets (lower is better). Bold and underline mark the full-table best and second-best values in each column.

Per-dataset dense 3D geometry

This appendix provides per-dataset surface reconstruction metrics for the dense geometry benchmark. The dense-geometry evaluation set comprises 38 sequences with dense 3D ground truth. It includes 11 sequences from ETH3D, 9 from Neural RGB-D, and 18 from 7-Scenes. After Umeyama alignment, ICP uses a correspondence threshold of 0.25 m on ETH3D and 0.10 m on Neural RGB-D and 7-Scenes. To weight each physical evaluation scene equally, Table 3 reports sequence-macro averages over these 38 sequences. Under unweighted dataset-macro averaging across the three datasets, VGGT-Omega baseline versus HiP3R scores are 0.0796 versus 0.0269 for Chamfer distance, and 65.97 versus 92.06 for F1 score. LingBot-MAP baseline versus HiP3R scores are 0.1439 versus 0.1014 for Chamfer distance, and 61.34 versus 74.58 for F1 score. Table 21 isolates the relative geometric contributions of layout alignment and camera refinement.

DatasetChamfer ↓\downarrow BaselineChamfer ↓\downarrow HIP3RF1 ↑\uparrow BaselineF1 ↑\uparrow HIP3R
ETH3D0.03390.033498.9299.00
Neural RGB-D0.13100.020540.5390.05
7-Scenes0.07390.026958.4587.13

Table 16. Dense 3D reconstruction metrics (Chamfer distance and F1 score) on VGGT-Omega versus HIP3R evaluated on datasets with dense 3D ground truth.

Chunk and overlap experiments

Table 18 compares selected chunk and overlap sizes against fixed defaults in an exploratory sweep. The default chunk size and overlap size are 60 and 30 frames for VGGT-Omega, and 120 and 32 frames for LingBot-MAP. Each window configuration produces its own chunk predictions and optimized coarse layout, which is then held fixed during camera processing. The coarse solve follows Appendix § 11, with a limit of 5000 function evaluations.

DatasetVGGT-Omega (Wang et al., [26]) C/OVGGT-Omega (Wang et al., [26]) ATEVGGT-Omega (Wang et al., [26]) vs 60/30LingBot-MAP (Chen et al., [3]) C/OLingBot-MAP (Chen et al., [3]) ATELingBot-MAP (Chen et al., [3]) vs 120/32
7-Scenes90/300.03660.0381120/80.08010.0793
ETH3D60/300.05460.0546120/320.4790.479
KITTI60/169.3529.736120/323.5083.508
NRGBD60/300.02190.0219160/320.06080.0776
Oxford60/301.5791.579120/321.6511.651
T&T60/300.4760.476120/320.5870.587
VBR120/3019.3022.45120/3220.3920.39

Table 18. Chunk and overlap comparison. C/OC/O denotes chunk size and overlap size in frames. The “vs” columns report ATE with the indicated default sizes. Bold marks a change from Table 10.

Both window configurations use dense fine optimization for T≤2000T \le2000 and overlap consensus for longer sequences. Overlap consensus averages rotations and translations at stride-10 frames, retains the final frame, and interpolates pose corrections to the remaining frames. For long sequences, this replaces the stride-5 keyframe optimization used in the main benchmark.

On KITTI, LingBot-MAP achieves 3.508 m with overlap consensus, compared with 6.063 m for Full in Table 13. Selected versus default macro-averaged ATE across the seven datasets is 4.403 versus 4.908 m for VGGT-Omega, and 3.822 versus 3.825 m for LingBot-MAP.

Ablation on the coarse-only layout

Table 19 compares camera accuracy after the coarse layout stage alone against the Full pipeline on three long sequences. Shared-camera refinement lowers RPErot\mathrm{RPE}_{\mathrm{rot}} and RPEtrans\mathrm{RPE}_{\mathrm{trans}} on all three sequences while leaving ATE close to the coarse-stage result.

SequenceTTLayout Only ATELayout Only RPErot\mathrm{RPE}_{\mathrm{rot}}Layout Only RPEtrans\mathrm{RPE}_{\mathrm{trans}}Full ATEFull RPErot\mathrm{RPE}_{\mathrm{rot}}Full RPEtrans\mathrm{RPE}_{\mathrm{trans}}
KITTI seq0045419.7500.2020.1319.7300.1540.093
spagna_train014,14110.2560.5420.04810.2610.5350.034
campus_train111,6717.3810.2300.0767.3870.1570.045

Table 19. Trajectory accuracy of Layout Only versus Full on the same feed-forward predictions.

DatasetChamfer ↓\downarrow BaselineChamfer ↓\downarrow HIP3RF1 ↑\uparrow BaselineF1 ↑\uparrow HIP3R
ETH3D0.12950.129586.4986.49
Neural RGB-D0.16200.067542.6665.24
7-Scenes0.14030.107154.8772.01

Table 17. Dense 3D reconstruction metrics (Chamfer distance and F1 score) on LingBot-MAP versus HIP3R evaluated on datasets with dense 3D ground truth.

Controlled layout-solver comparison

The layout-solver study uses ten long sequences: KITTI sequence 00, all seven VBR sequences, and Droid-W downtown2 and downtown5 (Li et al., [13]). All evaluated configurations share identical model predictions, candidate revisit pairs, and initialization. Camera refinement and point corrections are disabled to isolate layout optimization. The full coarse solve allows at most 5000 function evaluations. The five configurations evaluate baseline pairwise alignment, bounded point sampling, residual-fit weighting, progressive loop scheduling, and the complete optimization solver. The final step combines adaptive continuation, Huber loss, and bounded restarts. Appendix § 11 specifies the continuation and restart rules and their evaluation budgets.

We test solver robustness against noisy loop measurements by perturbing loop closures. Specifically, synthetic noise is injected into 25% of detected loop edges using a fixed random seed. For each selected edge, we apply a 10∘10^{\circ} rotation error. We also inject a 25% translation offset and a 0.10 log-scale distortion. The evaluation confirms efficiency gains from bounded sampling, alongside accuracy and runtime gains from the complete solver. The complete configuration reduces clean mean ATE by 19.77% relative to the baseline. This improvement concentrates in VBR, where mean error falls from 28.6017 to 22.4461 m. Peak host memory remains comparable across configurations.

Placement and motion ablation settings

The four controls in Table 5 evaluate different strategies for resolving multiple camera predictions under identical optimized coarse layouts. Layout Only selects the first available coarse-placed pose for each frame in chunk order. Single selects the occurrence that maximizes the minimum distance, in frames, to either boundary of its chunk. Ties are broken by the smaller chunk index. Both Layout Only and Single directly output the selected coarse-placed poses without frame-level optimization. Unary computes a robust pose average over all coarse-placed occurrences of each frame using deterministic Huber IRLS, without relative-motion factors. Motion optimizes within-chunk relative-motion factors without placement priors. It fixes the first camera to its coarse placement and uses λf=1\lambda_f = 1 and λp=0\lambda_p = 0. This control allows up to 5000 function evaluations with ftol = xtol = gtol = 10^{-8}. Shared jointly optimizes placement priors and relative-motion constraints as formulated in Eq. (6), and corresponds to Full in the main benchmark. All configurations use the same evaluation protocol, and results are macro-averaged across all seven datasets.

ConfigurationATERPErot\mathrm{RPE}_{\mathrm{rot}}RPEtrans\mathrm{RPE}_{\mathrm{trans}}Graph Build & Solve (s)Final solve (s)
Clean loop measurements
Baseline21.77511.310940.1129612232.181404.54
+ Bounded sampling21.75691.310930.112802004.201345.43
+ Fit weighting21.77701.310840.112622077.491387.14
+ Progressive loop scheduling21.52411.310770.110701477.59688.02
Complete17.47081.309720.09834753.4341.28
Perturbed loop measurements
Baseline20.47671.334440.110789343.001226.62
+ Bounded sampling20.52691.334470.111091641.881220.56
+ Fit weighting21.22261.330610.116271798.381284.72
+ Progressive loop scheduling19.19151.329760.105681032.27461.17
Complete16.92981.329510.10280794.1331.31

Table 20. Ablation of layout-solver components on extended sequences. Total time includes measurement construction and all layout solves. Final solve is the duration of the final least-squares call.

Dense geometry across optimization stages

Table 21 extends the trajectory comparison in Table 6 to dense surface geometry on ETH3D, Neural RGB-D, and 7-Scenes. Native sequentially stitches chunks through their overlaps without coarse or fine graph optimization. Layout Only applies the optimized coarse chunk placements without frame-level corrections. Full uses the same coarse layout and adds shared-camera corrections to the pointmaps, preserving their predicted depth and chunk scale. Shared-camera refinement produces additional gains in Chamfer and F1 on both backends. Native sequentially stitches chunks through their overlaps without coarse or fine graph optimization. Layout Only applies the optimized coarse chunk placements without frame-level corrections. Full uses the same coarse layout and adds shared-camera corrections to the pointmaps, preserving their predicted depth and chunk scale. Shared-camera refinement produces additional gains in Chamfer and F1 on both backends.

References

  1. [2]Leonardo Brizi, Emanuele Giacomini, Luca Di Giammarino, Simone Ferrari, Omar Salem, Lorenzo De Rebotti, and Giorgio Grisetti. VBR: A vision benchmark in Rome. In ICRA, 2024.arxiv.org/abs/2404.11322
  2. [3]Lin-Zhuo Chen, Jian Gao, Yihang Chen, Ka Leong Cheng, Yipengjing Sun, Liangxiao Hu, Nan Xue, Xing Zhu, Yujun Shen, Yao Yao, and Yinghao Xu. Geometric context transformer for streaming 3D reconstruction. arXiv preprint arXiv:2604.14141, 2026a.arxiv.org/abs/2604.14141
  3. [4]Xingyu Chen, Yue Chen, Yuliang Xiu, Andreas Geiger, and Anpei Chen. TTT3R: 3D reconstruction as test-time training. In ICLR, 2026b.arxiv.org/abs/2509.26645
  4. [5]A. R. Curtis, M. J. D. Powell, and J. K. Reid. On the estimation of sparse Jacobian matrices. IMA Journal of Applied Mathematics, 13(1):117–119, 1974.DOI
  5. [6]Kai Deng, Zexin Ti, Jiawei Xu, Jian Yang, and Jin Xie. VGGT-Long: Chunk it, loop it, align it – pushing VGGT’s limits on kilometer-scale long RGB sequences. arXiv preprint arXiv:2507.16443, 2025.arxiv.org/abs/2507.16443
  6. [7]Andreas Geiger, Philip Lenz, and Raquel Urtasun. Are we ready for autonomous driving? The KITTI vision benchmark suite. In CVPR, 2012.
  7. [8]Sergio Izquierdo and Javier Civera. Optimal transport aggregation for visual place recognition. In CVPR, 2024.arxiv.org/abs/2311.15937
  8. [9]Haian Jin, Rundi Wu, Tianyuan Zhang, Ruiqi Gao, Jonathan T. Barron, Noah Snavely, and Aleksander Holynski. ZipMap: Linear-time stateful 3D reconstruction via test-time training. In CVPR, 2026.
  9. [10]Arno Knapitsch, Jaesik Park, Qian-Yi Zhou, and Vladlen Koltun. Tanks and temples: Benchmarking large-scale scene reconstruction. ACM Transactions on Graphics, 36(4), 2017.
  10. [11]Yushi Lan, Yihang Luo, Fangzhou Hong, Shangchen Zhou, Honghua Chen, Zhaoyang Lyu, Shuai Yang, Bo Dai, Chen Change Loy, and Xingang Pan. Stream3R: Scalable sequential 3D reconstruction with causal transformer. In ICLR, 2026.arxiv.org/abs/2508.10893
  11. [12]Vincent Leroy, Yohan Cabon, and Jerome Revaud. Grounding image matching in 3D with MASt3R. In ECCV, 2024.arxiv.org/abs/2406.09756
  12. [13]Moyang Li, Zihan Zhu, Marc Pollefeys, and Dániel Baráth. DROID-SLAM in the wild. In CVPR, 2026a.
  13. [14]Zizun Li, Jianjun Zhou, Yifan Wang, Haoyu Guo, Wenzheng Chang, Yang Zhou, Haoyi Zhu, Junyi Chen, Chunhua Shen, and Tong He. WinT3R: Window-based streaming reconstruction with camera token pool. In ICLR, 2026b.arxiv.org/abs/2509.05296
  14. [15]Xuanyi Liu, Chunan Yu, Deyi Ji, Qi Zhu, Lingyun Sun, Xuanfu Li, Jin Ma, Tianrun Chen, and Lanyun Zhu. StreamCacheVGGT: Streaming visual geometry transformers with robust scoring and hybrid cache compression. arXiv preprint arXiv:2604.15237, 2026.arxiv.org/abs/2604.15237
  15. [16]Linfei Pan, Johannes Schönberger, and Marc Pollefeys. Global structure-from-motion meets feed-forward reconstruction. arXiv preprint arXiv:2605.26103, 2026.arxiv.org/abs/2605.26103
  16. [17]Thomas Schöps, Johannes L. Schönberger, Silvano Galliani, Torsten Sattler, Konrad Schindler, Marc Pollefeys, and Andreas Geiger. A multi-view stereo benchmark with high-resolution images and multi-camera videos. In CVPR, 2017.DOI
  17. [18]Jamie Shotton, Ben Glocker, Christopher Zach, Shahram Izadi, Antonio Criminisi, and Andrew Fitzgibbon. Scene coordinate regression forests for camera relocalization in RGB-D images. In CVPR, 2013.DOI
  18. [19]Joan Solà, Jeremie Deray, and Dinesh Atchuthan. A micro Lie theory for state estimation in robotics. arXiv preprint arXiv:1812.01537, 2018.arxiv.org/abs/1812.01537
  19. [20]Yifu Tao, Miguel Ángel Muñoz-Bañón, Lintong Zhang, Jiahao Wang, Lanke Frank Tarimo Fu, and Maurice Fallon. The Oxford Spires dataset: Benchmarking large-scale LiDAR-visual localization, reconstruction and radiance field methods. The International Journal of Robotics Research, 2025.DOI
  20. [21]Shinji Umeyama. Least-squares estimation of transformation parameters between two point patterns. IEEE Transactions on Pattern Analysis and Machine Intelligence, 13(4):376–380, 1991.DOI
  21. [22]Chung-Shien Brian Wang, Christian Schmidt, Jens Piekenbrinck, and Bastian Leibe. Block-sparse global attention for efficient multi-view geometry transformers. In CVPR, 2026a.arxiv.org/abs/2509.07120
  22. [25]Jianyuan Wang, Minghao Chen, Nikita Karaev, Andrea Vedaldi, Christian Rupprecht, and David Novotny. VGGT: Visual geometry grounded transformer. In CVPR, 2025a.arxiv.org/abs/2503.11651
  23. [26]Jianyuan Wang, Minghao Chen, Shangzhan Zhang, Nikita Karaev, Johannes Schönberger, Patrick Labatut, Piotr Bojanowski, David Novotny, Andrea Vedaldi, and Christian Rupprecht. VGGT-Ω. In CVPR, 2026b.
  24. [27]Qianqian Wang, Yifei Zhang, Aleksander Holynski, Alexei A. Efros, and Angjoo Kanazawa. Continuous 3D perception model with persistent state. In CVPR, 2025b.arxiv.org/abs/2501.12387
  25. [28]Shuzhe Wang, Vincent Leroy, Yohann Cabon, Boris Chidlovskii, and Jerome Revaud. DUSt3R: Geometric 3D vision made easy. In CVPR, 2024.arxiv.org/abs/2312.14132
  26. [29]Yifan Wang, Jianjun Zhou, Haoyi Zhu, Wenzheng Chang, Yang Zhou, Zizun Li, Junyi Chen, Jiangmiao Pang, Chunhua Shen, and Tong He. π³: Permutation-equivariant visual geometry learning. arXiv preprint arXiv:2507.13347, 2025c.arxiv.org/abs/2507.13347
  27. [30]Tao Xie, Peishan Yang, Yudong Jin, Yingfeng Cai, Wei Yin, Weiqiang Ren, Qian Zhang, Wei Hua, Sida Peng, Xiaoyang Guo, and Xiaowei Zhou. Scal3R: Scalable test-time training for large-scale 3D reconstruction. In CVPR, 2026.
  28. [31]Zhuang Xiong, Chen Zhang, Qingshan Xu, and Wenbing Tao. VGGT-Motion: Motion-aware calibration-free monocular SLAM for long-range consistency. arXiv preprint arXiv:2602.05508, 2026.arxiv.org/abs/2602.05508
  29. [32]Junyi Zhang, Charles Herrmann, Junhwa Hur, Chen Sun, Ming-Hsuan Yang, Forrester Cole, Trevor Darrell, and Deqing Sun. LoGeR: Long-context geometric reconstruction with hybrid memory. arXiv preprint arXiv:2603.03269, 2026a.arxiv.org/abs/2603.03269
  30. [33]Wei Zhang, Yihang Wu, Songhua Li, and Qi Wang. VGGT-Align: Bridging local reconstruction and global consistency for long-sequence 3D reconstruction. arXiv preprint arXiv:2608.15260, 2026b.DOI
  31. [34]Dong Zhuo, Wenzhao Zheng, Jiahe Guo, Yuqi Wu, Jie Zhou, and Jiwen Lu. Streaming visual geometry transformer. In ICLR, 2026.
  32. [35]Zichen Zou, Xiaosong Jia, Zuxuan Wu, and Yu-Gang Jiang. Attention Itself Could Retrieve. RetrieVeVGGT: Training-free long context streaming 3D reconstruction via query-key similarity retrieval. arXiv preprint arXiv:2605.09644, 2026.

Paper details

vlnc:W_8pc4yufd
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents