WorksComputer Science
Deep Bundle Adjustment
Abstract
We present DeepBA (Deep Bundle Adjustment), a graph neural network (GNN) that recovers camera poses, intrinsics, and scene structure from 2D matches across an arbitrary number of views. Rather than solving a non-linear least-squares problem per scene, DeepBA amortizes bundle adjustment into a learned graph network that iteratively refines its estimate from geometric residuals. Where classical SfM computes its updates analytically and needs careful initialization, DeepBA learns them and converges from a cold start. It is trained with scale-invariant supervision (ground-truth rotation, translation direction, and intrinsics, plus reprojection self-consistency), so datasets with arbitrary metric units can be mixed without per-scene normalization. DeepBA is decoupled from the matcher. At 2.04M parameters, it outperforms prior learned SfM from matches on scene reconstruction and on most pose metrics, is competitive with classical SfM at moderate accuracy thresholds and overtakes it in wall time as the view count grows, and approaches monolithic image-to-geometry transformers such as VGGT (1.2B parameters) with greater modularity. Project~page: https://cphjsn.github.io/deep-ba/
Introduction
Recovering camera poses and scene geometry from a set of images is a foundational problem in 3D computer vision, addressed by structure-from-motion (SfM) [40] and simultaneous localization and mapping (SLAM) [34]. Most modern SfM pipelines and keyframe-based SLAM systems rely on the same classical solver, bundle adjustment (BA) [14, 52], which jointly refines cameras and 3D points by minimizing reprojection residuals via non-linear least squares. Bundle adjustment is near-optimal given a good initialization, but its non-convex landscape is fragile. Incremental SfM obtains such an initialization through heuristics and per-scene tuning [40], yet these heuristics break down on weak baselines or planar structure [51]. Meanwhile, learning-based semi-dense [45] and dense [8, 9] matchers have made matches far more reliable than SIFT-style keypoints [32], shifting the bottleneck from matching to the solver. However, feeding them into a COLMAP-style pipeline does not automatically help, as its rigid track building is overwhelmed by the volume and its initialization stays brittle. This mismatch motivated a line of feed-forward 3D networks (DUSt3R [58], MASt3R [24], and VGGT [57]) that regress geometry directly from images, trading the classical pipeline’s modularity for robustness. There is a missing middle: a learned solver that amortizes bundle adjustment itself, while leaving the matcher decoupled and solving any number of views at once. Classical bundle adjustment already operates on a graph of cameras and their 2D matches, so a graph neural network is the natural architecture for that structure.
We introduce DeepBA (Deep Bundle Adjustment, Figure 1), a GNN that takes 2D matches from an off-the-shelf matcher and jointly infers absolute camera poses, intrinsics, and 3D points. DeepBA replaces the per-scene non-linear least-squares solver with a learned Markovian iterative refinement. Each iteration recomputes geometric features from the current state and predicts an updated state via shared GNN weights. The number of iterations is a free choice at test time, independent of the training schedule: accuracy rises with the iteration count until it saturates, mirroring an iterative classical solver. It operates directly on pairwise matches, with no separate track-building stage. Our contributions are:

Figure 1. DeepBA runs the bundle-adjustment loop with a learned update, on a track-free state. From a view graph built by any off-the-shelf matcher (Input), a weight-shared GNN iteratively refines camera poses, intrinsics, and depth from a trivial cold start . The example is the full 240-camera Mip-NeRF 360 [1] counter scene, with the estimated cameras in green.
Amortized cold-start bundle adjustment via a GNN. We formulate bundle adjustment as a learned iterative refinement over the view graph of any number of views, with a Markovian update that lets the user trade runtime for accuracy at test time. It requires no initial poses or intrinsics and converges from a cold start, where classical BA fails.
Scale-invariant, anchor-free, multi-dataset supervision. Chordal rotation, direction-only translation, intrinsics, self-consistent reprojection and 3D losses train DeepBA on datasets with arbitrary metric units without per-scene normalization. Because DeepBA predicts state updates rather than geometry, it needs no anchor camera, unlike VGGT [57], which uses the first camera as its reference frame.
Robustness across matcher families. DeepBA sees only 2D matches, so a trained network can run from sparse to dense matching without retraining. We demonstrate this extensively, across every matcher family.
Competitive accuracy at a fraction of the size. With 2.04M parameters (~ fewer than the strongest prior match-based network, GASfM [2] at 145M), DeepBA outperforms learned SfM from matches on most metrics
and is competitive with classical SfM at moderate thresholds, approaching image-to-geometry transformers such as VGGT (~1.2B parameters).
Related Work
Classical structure-from-motion and bundle adjustment. The classical pipeline for multi-view reconstruction proceeds by feature extraction, pairwise matching, geometric verification, incremental pose estimation, and a final non-linear refinement via BA [14, 52]. COLMAP [40] is its standard implementation and the baseline most work compares against. GLOMAP [37] replaces its incremental stage with global rotation averaging followed by a positioning step that solves for all cameras and points at once. That drops the seed-and-order heuristics, but leans on the pairwise rotations instead. Bundle adjustment’s accuracy hinges on initialization [31] and hand-tuned heuristics, making it brittle on weak baselines, planar or low-texture scenes, and repetitive structure [15, 16, 51]. A long line of work has widened bundle adjustment’s basin of attraction with alternative error parameterizations and convex relaxations [13, 17, 18, 20, 60], which improves cold-start convergence. These remain per-scene optimizations and still face the mirror ambiguity and the global-scale gauge freedom of unconstrained multi-view geometry [35, 44]. Recent neural pipelines inherit the same structure, still running an optimization once per scene at test time. For example, Detector-Free SfM [15] adapts track building to dense matchers, and FlowMap [43] fits poses, intrinsics, and depth by gradient descent through a dense-flow objective. TFD-SfM [3] removes track building altogether. It lifts both pixels of every correspondence to their own 3D landmarks and optimizes the reconstruction from a cold start. Similar to TFD-SfM, DeepBA is track-free but amortizes the per-scene optimization into a learned refinement from a cold start.
Feed-forward 3D reconstruction. Recent works remove the classical pipeline, training neural networks to regress geometry directly from images. PoseDiffusion [55] diffuses camera poses alone. DUSt3R [58] and MASt3R [24] produce per-camera pointmaps in a shared frame, with MASt3R adding an explicit matching head for downstream localization. Both are pair-only: extending them to many views needs a global-alignment step (e.g. MASt3R-SfM [7]) that re-introduces a per-scene non-linear optimization. VG-GSfM [56] combines deep pose and triangulator modules with a differentiable bundle adjustment back-end, while VGGT [57] pushes furthest, absorbing the matcher, visual encoder, and geometry solver into a single transformer that ingests dozens of views in one forward pass, at the cost of model size, training-data scale, and tight perception–geometry coupling. DeepBA instead operates on matches rather than image features and reasons jointly over all views, needing no fusion of pairwise reconstructions. Keeping the matcher external rather than learning one, it is smaller than VGGT and trainable on substantially less data.
Iterative learned refinement. RAFT [48] introduced iterative refinement with a fixed update network applied for a variable number of iterations, recomputing features from the current estimate at each one. The template has been extended to stereo, scene flow, and visual SLAM [29, 49, 50]. What makes the template work is supervising the intermediate iterations. The network cannot then rely on one large correction, and instead learns a contraction, an update that improves the estimate every time it is applied. The number of iterations therefore becomes a test-time trade between accuracy and runtime. DeepBA carries this template into the multi-view geometry setting with a weight-shared GNN (a Markovian update, without a recurrent gate). The closest learned refiner is DROID-SLAM [49], which targets calibrated, temporally-ordered video with a coupled dense feature network, whereas DeepBA handles unordered, wide-baseline collections and recovers intrinsics from a trivial initialization.
Learning-based SfM from matches. The work closest to ours also learns SfM directly from 2D matches rather than from pixels. Most of it regresses the reconstruction with a network and ends in a classical bundle adjustment. Deep Permutation Equivariant SfM [33] and its successor RESfM [23] regress cameras and sparse structure by minimizing reprojection error. Both fine-tune the network on each test scene before the final bundle adjustment. GASfM [2] instead re-places the equivariant layers of [33] with graph attention over the same point tracks. It is feed-forward, restricted to a calibrated or purely projective regime, and refined by a final classical bundle adjustment. LS-Net [4] learns a non-linear least-squares solver for dense, calibrated two-view stereo. Learning-to-Bundle-Adjust [46] predicts BA updates with a graph network, refining the local map of an initialized vehicular SLAM system. BA-Net [47] starts its poses at the identity but runs a dense feature-metric BA on calibrated short clips, with a learned Levenberg–Marquardt damping and an image-dependent depth basis. PoGO-Net [25] is a graph network for rotation/motion averaging that consumes relative poses, not raw matches, and recovers only orientations. Unlike these methods, DeepBA learns the update of poses, intrinsics and depth itself. It recomputes geometric features of the matches from its current estimate at every refinement iteration. It starts every scene from the same initial state.
Method
We introduce DeepBA, a graph neural network that performs amortized bundle adjustment via iterative refinement over a graph of cameras and their pairwise matches (Figure 2).

Figure 2. DeepBA architecture. Left: one refinement iteration: the scene state and view graph are encoded into , re-initialized every iteration (Markovian), mixed by MPC blocks, and decoded into the state delta composed via (Eq. (3)); the block is weight-shared across the iterations. Right: one MPC block; its three message-passing steps are specified in § 3.3.
Problem Setting and Notation
Given unordered images of a static scene, DeepBA maps pairwise 2D matches to per-image estimates of absolute pose, intrinsics, and a depth map (3D points are recovered by unprojection). The matches are image-point pairs from an off-the-shelf matcher. The inputs form a view graph in which is the set of cameras, is the set of pairs of cameras connected by at least 50 matches, and is the set of all matches over all pairs of the view graph. Each camera carries an absolute pose , a log field of view (log-FOV) (our default camera model, more detail in Sec. A.2), and a log-depth map on a fixed grid with . Each pair carries a relative pose in a decoupled parameterization: a unit translation direction and a rotation vector . Together, these form the scene state
The absolute poses, FOVs and depth maps already define the reconstruction. We still keep a relative pose per pair, which the network refines at every iteration under ground-truth (GT) supervision, while the absolute poses receive no GT supervision. The motivation of this design comes from pose-graph optimization. The absolute poses are updated to agree with the current relative poses instead of being pulled directly toward the GT, so they follow the gradual refinement of the relative poses. A relative pose is only a rotation and a translation direction, so its GT supervision needs no metric scale, while the reprojection error sets the translation lengths of the absolute poses. At every refinement iteration, the network predicts a delta
with in Lie-algebra coordinates, and composes it onto the current state,
where acts component-wise (exact update rules and conventions in Sec. A.1).
Per-Iteration Geometric Features
At every refinement iteration, DeepBA computes three hidden states from scratch (Markovian: no hidden state crosses iterations): the per-pair and per-match encode closed-form geometric features of the current scene state, while the per-camera takes no geometric input and starts from a single learned embedding shared by every camera. All three hidden states have dimension .
Pair Geometric Features
The pair geometric features give the network pairwise information. Although a pair stores a single relative pose, we featurize it in both directions so that message passing in the GNN reaches both of its cameras: we construct a 14-dim feature per direction, for the forward edge and for the inverse , and encode each into its own per-pair hidden state, and . The forward feature is
where the pose-graph residual measures the disagreement between the directly predicted relative pose and the counterpart derived from the two absolute poses (exact form in Sec. A.3).
Match Geometric Features
The match geometric features tell the network how well the current state explains each match. Each match , with endpoints in images and , contributes a 12-dim feature per direction— for the forward edge and for the inverse—and is encoded into its own per-match hidden state and . For brevity, we drop the edge subscript and write throughout the remainder of this subsection. The forward feature is
Beyond , the remaining entries are closed-form functions of the current state (full formulas in Sec. A.4). The epipolar vector is the point-to-epipolar-line offset under the directly predicted relative pose. The others are the 2D reprojection residual , the 3D consistency residual , and the log-magnitude of the destination point.
Iterative Refinement Network
State refinement block. At iteration , the state refinement block (Figure 2, left) runs three steps.
Step 1 (encode). The features enter per-type hidden states through a linear map,
while is set to the learned embedding .
Step 2 (mix). A stack of Match–Pair–Camera (MPC) blocks mixes the three tiers over the graph,
for . The blocks do not share weights.
Step 3 (decode). The prediction heads emit the state delta .
MPC block composition. Each MPC block (Figure 2, right) updates the three hidden states in three steps.
Step 1 (camerapair/match broadcast). Injects endpoint camera states into per-pair and per-match channels:
where represents the learned linear map .
Step 2 (matchpair aggregation). Matches are conditioned on their parent pair and aggregated into it by attention over the pair’s matches :
where is 4-head dot-product attention [54] (query , keys/values from ).
Step 3 (paircamera attention). Each camera attends over its incident directional pair states. The forward state reaches camera , the inverse state camera :
with the cameras paired with .
Prediction heads. Four heads read the hidden states of the final MPC block and emit the change of the state .
Absolute-pose head and log-FOV head. Two MLPs on produce and , respectively.
Relative-pose head. An MLP on produces in the decoupled parameterization. The change of relative pose is obtained by
Here is the output of the same head on the reversed state and predicts the change of . We use as a second prediction of the change of . (11) averages the two predictions, which makes the update antisymmetric by construction, .
Depth head. A small spatial decoder: each is scattered onto its destination camera’s grid at the match location, pooled by a bilinear per-cell mean, and decoded into the log-depth delta (details in Sec. A.5).
Training Loss
Scale-invariant pose supervision. We define a scale-invariant pose distance that combines a chordal rotation distance with a translation-direction cosine,
where is the geodesic angle between and on , computed from the relative rotation via , and is the angle between the unit-direction translations on , with . Because never consumes the magnitude of , no metric scale leaks into supervision. This enables us to mix datasets with arbitrary metric units without per-scene normalization. For every pair , we define two different residuals,
where is the stop-gradient operator. supervises the relative-pose head against GT. The term couples the absolute-pose head to the (GT-supervised) relative-pose head, supervising the absolute poses through multi-view consensus, a learned analogue of pose-graph optimization. The stop-gradient makes the absolute poses follow the relative poses, never the reverse. The pose loss averages both over all pairs:
Geometric losses
We further enforce self-consistency between the model’s outputs through a geometric reprojection loss that requires no ground-truth scale. Writing for the smooth- kernel [12, 19] of width (Sec. D), the 2D reprojection and 3D consistency losses average the robustified residual norms over all matches :
where and are the per-match residuals of § 3.2.2 and is a Cauchy weight on match , held constant during backpropagation, with the match’s Sampson distance under the GT epipolar geometry (kernel width ). As a result, are effectively evaluated on inliers only, encouraging the network to fit the correct matches and to learn to ignore the inaccurate ones.
The FOV loss applies the same kernel to the log-FOV error against GT,
Final loss. We run a Markov trajectory of iterations per training step and backpropagate through 24 of them (, Sec. D). In practice, even after the reconstruction has converged, the refiner keeps emitting updates that are pure global rotations and translations of the scene. We therefore add a fixed-point regularizer that penalizes the magnitude of these unnecessary, gauge-equivalent updates, pressing late iterations toward true stationarity. The total loss sums over a view graph,
Experiments
We evaluate DeepBA on multi-view camera pose across six benchmarks (§ 4.2) and on structure accuracy on DTU (§ 4.3), then ablate and analyze the upstream matcher, network capacity, outlier robustness, generalization beyond the training budget, and runtime scaling (§ 4.4). The appendix adds the accuracy-versus-iteration profile (Sec. E) and testing of camera localization on a frozen DeepBA map (Sec. G). In this paper, we treat all image collections as unordered and without known intrinsics unless explicitly noted.
Setup
Evaluation datasets. Following prior work [24, 56, 57, 58], we evaluate on six benchmarks that span diverse imaging conditions: CO3Dv2 [38], RealEstate10K [63], ETH3D [41], DTU [21], Mip-NeRF 360 [1], and IMC2021 [22]. Per-dataset scene selection and frame sampling are detailed in Sec. C. Five benchmarks (RealEstate10K, ETH3D, DTU, Mip-NeRF 360, and IMC2021) are absent from our training mixture, so they measure cross-dataset generalization; CO3Dv2 is in the mixture but is evaluated only on held-out scenes disjoint from the training set.
Training data. DeepBA is trained on a heterogeneous mixture of 3D datasets: MegaDepth [26], CO3Dv2 [38], ScanNet [5], BlendedMVS [61], DL3DV [28], and Calle [10, 36], spanning indoor/outdoor, object- and scene-centric captures. Their arbitrary and incompatible metric units mix safely under DeepBA’s scale-invariant supervision (§ 3.4), without per-scene normalization.
Baselines. We compare against three families: classical SfM (COLMAP [40], GLOMAP [37]); learned image-to-geometry (PoseDiffusion [55], DUSt3R [58], MASt3R [24], MASt3R-SfM [7], VGGSfM [56], VGGT [57]); and learning-based SfM from matches (GASfM [2], RESfM [23]).
Metrics. For pose estimation, following [24, 57], we report Relative Rotation/Translation Accuracy (RRA/RTA) and AUC@30°, the area under the joint accuracy curve up to 30°. For structure quality, we report Accuracy and Completeness against DTU’s structured-light reference scans at 1/2/5 cm. Both follow the standard definitions, given in Sec. B.
Implementation. DeepBA uses MPC blocks (2.04M parameters), trained for 50,000 steps with AdamW [30] (peak learning rate ) and loss weights , , on 4 NVIDIA L40S GPUs. Each step unrolls the refiner for all iterations but backpropagates only through a 24-iteration subset. The scene state is initialized input-independently (identity poses, zero log-FOV and log-depth) and perturbed with Gaussian noise during training. Unless stated otherwise (§ 4.4), we always evaluate on eLoFTR [59] (training mixes eLoFTR and SIFT), per-pair RANSAC-verified [11], with on every benchmark. Further training details are in Sec. D. Code and trained weights will be released upon publication.
Camera Pose Estimation
We evaluate camera pose on CO3Dv2 and RealEstate10K (Table 1) following the 10-frame protocol in VGGT [57].
| Method | CO3Dv2 RRA@15° | CO3Dv2 RTA@15° | CO3Dv2 AUC@30° | RealEstate10K AUC@30° | Params | Time |
| Classical SfM | ||||||
| COLMAP (SIFT) [40] | 86.3 | 79.9 | 74.9 | 46.5 | – | 2+3 s |
| GLOMAP (SIFT) [37] | 86.6 | 76.2 | 71.1 | 71.7 | – | 2+2 s |
| COLMAP (eLoFTR) [40] | 89.7 | 83.5 | 77.0 | 80.9 | – | 7+11 s |
| GLOMAP (eLoFTR) [37] | 82.1 | 72.3 | 67.6 | 72.7 | – | 7+5 s |
| Learned image-to-geometry | ||||||
| PoseDiffusion [55] | 87.7 | 80.0 | 71.6 | 39.6 | 39.0M | 11 s |
| DUSt3R [58] | 98.3 | 82.1 | 72.9 | 79.9 | 577.8M | 7 s |
| MASt3R [24] | 99.2 | 92.1 | 84.7 | 91.1 | 695.3M | 9 s |
| VGGSfM [56] | 99.4 | 85.8 | 85.0 | 89.5 | 281.2M | 10 s |
| VGGT [57] | 99.3 | 92.1 | 85.1 | 85.3 | 1.2B | 0.2 s |
| Learning-based SfM from matches | ||||||
| RESfM (SIFT) [23] | 62.7 | 55.5 | 51.1 | 48.9 | 0.7M | 2+13 s |
| GASfM (SIFT) [2] | 79.1 | 73.9 | 68.6 | 75.4 | 145.2M | 0.5+5 s |
| RESfM (eLoFTR) [23] | 61.1 | 54.0 | 48.2 | 63.7 | 0.7M | 7+13 s |
| GASfM (eLoFTR) [2] | 79.0 | 77.3 | 71.2 | 82.8 | 145.2M | 7+2 s |
| DeepBA | 97.4 | 89.2 | 78.6 | 84.0 | 2.04M | 7+3 s |
Table 1. Camera pose estimation on CO3Dv2 and RealEstate10K (10 frames per scene). Bold/underline: best/second per column within each group. †: ground-truth intrinsics plus a final classical bundle adjustment. Params: from each checkpoint. Time: seconds per scene on one H100, as matching + solver. ‡: same cached eLoFTR correspondences as DeepBA, given to COLMAP/GLOMAP/GASfM/RESfM as multi-view tracks (Sec. C).
CO3Dv2 covers object-centric, wide-baseline pairs, while RealEstate10K covers indoor and outdoor video motion. We additionally evaluate pose on four further multi-view benchmarks, DTU, Mip-NeRF 360, ETH3D, and IMC2021 (Table 2), each scored at the tighter @3°/@5°/@10° thresholds (per-dataset composition in Sec. C).
| Method | DTU [21] (49 img/scene) | Mip-NeRF 360 [1] (125–311 img/scene) | ETH3D [41] (14–76 img/scene) | IMC2021 [22] (5/10/25-view bags) | ||||||||||||||||||||
| RRA↑ @3° | RRA↑ @5° | RRA↑ @10° | RTA↑ @3° | RTA↑ @5° | RTA↑ @10° | RRA↑ @3° | RRA↑ @5° | RRA↑ @10° | RTA↑ @3° | RTA↑ @5° | RTA↑ @10° | RRA↑ @3° | RRA↑ @5° | RRA↑ @10° | RTA↑ @3° | RTA↑ @5° | RTA↑ @10° | RRA↑ @3° | RRA↑ @5° | RRA↑ @10° | RTA↑ @3° | RTA↑ @5° | RTA↑ @10° | |
| Classical SfM | ||||||||||||||||||||||||
| COLMAP (SIFT)[40] | 87.3 | 87.3 | 87.3 | 65.6 | 74.6 | 82.6 | 100.0 | 100.0 | 100.0 | 99.9 | 100.0 | 100.0 | 96.7 | 96.7 | 96.7 | 94.9 | 95.8 | 96.4 | 94.0 | 94.9 | 95.7 | 74.7 | 82.5 | 89.3 |
| GLOMAP (SIFT)[37] | 85.9 | 86.8 | 87.2 | 68.7 | 73.4 | 78.6 | 100.0 | 100.0 | 100.0 | 99.9 | 100.0 | 100.0 | 94.4 | 94.5 | 96.6 | 86.3 | 89.3 | 91.5 | 92.2 | 93.9 | 96.0 | 73.9 | 81.4 | 88.2 |
| COLMAP (eLoFTR)‡[40] | 100.0 | 100.0 | 100.0 | 91.8 | 97.1 | 99.5 | 100.0 | 100.0 | 100.0 | 99.6 | 99.8 | 100.0 | 76.5 | 76.6 | 76.9 | 69.1 | 72.8 | 75.5 | 95.8 | 96.7 | 97.2 | 75.4 | 84.3 | 91.5 |
| GLOMAP (eLoFTR)‡[37] | 97.1 | 97.1 | 97.9 | 89.7 | 94.8 | 96.8 | 100.0 | 100.0 | 100.0 | 99.7 | 99.9 | 100.0 | 58.4 | 59.7 | 61.6 | 50.2 | 53.5 | 56.9 | 95.8 | 96.7 | 97.8 | 74.9 | 83.6 | 91.2 |
| Learned image-to-geometry | ||||||||||||||||||||||||
| MASt3R-SfM [7] | 99.6 | 100.0 | 100.0 | 92.0 | 97.7 | 99.5 | 68.5 | 79.9 | 86.9 | 60.2 | 75.2 | 84.0 | 67.8 | 74.0 | 79.0 | 63.8 | 68.8 | 74.6 | 94.0 | 96.2 | 97.5 | 62.4 | 76.3 | 87.7 |
| VGGT [57] | 100.0 | 100.0 | 100.0 | 99.3 | 99.9 | 100.0 | 97.6 | 100.0 | 100.0 | 87.1 | 95.3 | 98.5 | 84.1 | 93.4 | 96.9 | 51.4 | 65.0 | 79.6 | 99.8 | 100.0 | 100.0 | 71.1 | 82.4 | 91.9 |
| Learning-based SfM from matches | ||||||||||||||||||||||||
| RESfM (SIFT)†[23] | 22.0 | 22.0 | 22.0 | 22.3 | 23.3 | 26.0 | 11.6 | 11.8 | 12.0 | 11.5 | 11.7 | 12.1 | 3.1 | 3.5 | 4.5 | 3.0 | 3.2 | 3.8 | 81.7 | 82.6 | 84.6 | 76.7 | 79.0 | 81.2 |
| GASfM (SIFT)†[2] | 38.3 | 39.9 | 40.8 | 39.1 | 40.5 | 44.1 | 32.4 | 35.2 | 41.1 | 31.7 | 35.4 | 44.3 | 8.8 | 9.5 | 12.1 | 8.3 | 9.0 | 10.7 | 74.7 | 80.4 | 86.3 | 56.1 | 63.9 | 75.3 |
| RESfM (eLoFTR)†,‡[23] | 36.6 | 36.7 | 36.8 | 36.8 | 37.2 | 39.3 | 10.7 | 11.1 | 11.3 | 10.5 | 10.8 | 12.0 | 19.1 | 19.3 | 19.6 | 18.3 | 18.8 | 19.4 | 81.0 | 82.2 | 84.8 | 75.5 | 77.9 | 79.8 |
| GASfM (eLoFTR)†,‡[2] | 36.9 | 37.6 | 38.8 | 37.4 | 38.6 | 42.4 | 49.6 | 51.9 | 54.7 | 50.8 | 53.5 | 57.6 | 21.6 | 23.9 | 27.2 | 21.0 | 23.9 | 29.4 | 89.6 | 91.0 | 92.2 | 83.1 | 87.3 | 90.8 |
| DeepBA | 73.6 | 95.6 | 100.0 | 69.5 | 90.8 | 99.6 | 53.3 | 66.7 | 73.5 | 58.6 | 68.2 | 86.5 | 33.8 | 53.4 | 69.1 | 18.4 | 28.8 | 49.1 | 86.9 | 95.6 | 99.0 | 49.4 | 68.9 | 87.3 |
Table 2. Multi-view pose accuracy (RRA/RTA) on DTU, Mip-NeRF 360, ETH3D, and IMC2021. Bold/underline: best/second per column within each group. Grey: image-to-geometry methods, for reference. †, ‡: as in Tab. 1.
CO3Dv2 and RealEstate10K. On both benchmarks (Table 1), DeepBA leads the learned methods from matches. On RealEstate10K, GASfM with the same eLoFTR correspondences comes closest (AUC@30° 82.8 vs. 84.0), while GASfM (SIFT) scores 75.4. Against classical SfM, DeepBA outperforms COLMAP and GLOMAP on every metric of both benchmarks, with both SIFT and the shared eLoFTR input.
DTU, Mip-NeRF 360, ETH3D and IMC2021. On these four benchmarks (Table 2), DeepBA leads the learned methods from matches on most metrics. GASfM and RESfM, given ground-truth intrinsics, are ahead only in a few cells, mostly on IMC2021 translation. Against classical SfM on the shared eLoFTR input, DeepBA comes close to both mappers at 10° on DTU and IMC2021 but falls behind at the tighter thresholds. On Mip-NeRF 360 the mappers are near the ceiling and DeepBA trails them (RRA@10° 73.5, RTA@10° 86.5). On ETH3D, COLMAP is well ahead (RTA@10° 75.5 against 49.1, see Sec. F.1). We also provide a qualitative comparison in Figure 3 and a failure analysis in Sec. F.

Figure 3. Qualitative comparison (rows: scan82, bicycle, delivery_area, and london_bridge; the number after each dataset is the number of input images); 3D points, estimated cameras (green), GT cameras (grey).
Camera localization. DeepBA’s robustness also enables camera localization (Sec. G). Reusing the same model with the reference cameras held fixed, DeepBA registers query views against a frozen map from the mean pose of the map cameras, with no PnP or external solver.
Structure Accuracy
Beyond poses, Table 3 evaluates the recovered structure on DTU against the official structured-light reference scans. Among the learning-based SfM methods from matches, DeepBA is the strongest in every column and the only one to produce a non-empty object reconstruction on all 15 scans—even though GASfM and RESfM use ground-truth intrinsics. The learned image-to-geometry methods are near the ceiling on this object-centric benchmark.
| Method | Accuracy (%)† @1 cm | Accuracy (%)† @2 cm | Accuracy (%)† @5 cm | Completeness (%)† @1 cm | Completeness (%)† @2 cm | Completeness (%)† @5 cm | Succ. |
| Classical SfM | |||||||
| COLMAP (SIFT) [40] | 94.2 | 97.6 | 99.4 | 80.6 | 95.2 | 99.6 | 15/15 |
| GLOMAP (SIFT) [37] | 43.0 | 49.3 | 65.0 | 30.1 | 40.7 | 51.6 | 11/15 |
| COLMAP (eLoFTR)‡ [40] | 91.7 | 98.2 | 99.8 | 96.6 | 99.7 | 100.0 | 15/15 |
| GLOMAP (eLoFTR)‡ [37] | 64.6 | 71.3 | 73.1 | 68.1 | 72.5 | 73.3 | 12/15 |
| Learned image-to-geometry | |||||||
| MASt3R-SfM [7] | 88.9 | 97.4 | 99.6 | 85.5 | 94.1 | 98.4 | 15/15 |
| VGGT [57] | 95.4 | 98.0 | 99.2 | 97.2 | 98.8 | 99.9 | 15/15 |
| Learning-based SfM from matches | |||||||
| RESfM (SIFT)†[23] | 9.3 | 21.7 | 38.0 | 0.9 | 3.9 | 17.4 | 10/15 |
| GASfM (SIFT)†[2] | 33.0 | 39.3 | 46.4 | 25.4 | 35.3 | 47.3 | 13/15 |
| RESfM (eLoFTR)†,‡[23] | 20.6 | 28.1 | 52.5 | 19.5 | 27.5 | 46.5 | 11/15 |
| GASfM (eLoFTR)†,‡[2] | 30.9 | 34.5 | 42.5 | 35.6 | 41.8 | 47.1 | 14/15 |
| DeepBA | 34.1 | 62.3 | 98.5 | 37.1 | 59.5 | 96.1 | 15/15 |
Table 3. Structure accuracy and completeness on DTU (15 test scans). Succ.: scans with a non-empty object reconstruction. Bold/underline: best/second on accuracy and completeness metrics within each group. †, ‡: as in Tab. 1. Grey: image-to-geometry methods, for reference.
Ablations and Analysis
Configuration ablations. Table 4 (left) ablates the configuration on CO3Dv2. Hidden width matters most: halving it to drops AUC@30° from 78.6 to 58.9, so we keep . Weakening the 3D consistency loss to is also costly (78.6 → 64.2), indicating the 3D term contributes beyond the pose losses. Enabling the per-iteration gauge strip leaves accuracy almost unchanged, so the gauge freedom needs no explicit fixing as expected. The last two rows change only the evaluation. Ground-truth focal lengths leave RRA almost unchanged but raise RTA, as they remove the focal-depth ambiguity (Sec. F.2). A classical BA, started from DeepBA’s output, raises all three metrics. This gain shows that DeepBA behaves similarly to BA but not identi-
| RRA @15°↑ | RTA @15°↑ | AUC @30°↑ | |
| Default | 97.4 | 89.2 | 78.6 |
| : 128 → 64 | 79.2 | 72.6 | 58.9 |
| : 0.1 → 0.01 | 81.7 | 76.6 | 64.2 |
| gauge strip: off → on | 97.1 | 88.8 | 78.3 |
| GT focal length | 97.5 | 90.9 | 81.5 |
| post classical BA | 98.0 | 91.0 | 84.6 |
Table 4. Ablations (left) and outlier robustness (right) on CO3Dv2 (protocol of Tab. 1). Left: each row changes one factor vs. the default, green/red marking gain/loss; the gauge strip removes the global rigid motion and rescales each iteration. Right: a fraction of matches is slid along the GT epipolar line (Sampson≈0).
| Corrupted fraction | RRA @15°↑ | RTA @15°↑ | AUC @30°↑ | |
| Default | 97.4 | 89.2 | 78.6 | |
| 5% | 95.5 | 86.8 | 74.5 | |
| 10% | 95.7 | 86.4 | 73.4 | |
| 50% | 86.8 | 77.5 | 62.4 | |
| 90% | 65.2 | 58.6 | 42.7 | |
| 100% | 63.4 | 56.3 | 40.7 |
::: cally, which leaves room for improvement.
Outlier robustness. A RANSAC-style geometric filter cannot remove wrong matches that lie on the true epipolar line (Sampson , e.g., from repetitive structure). We probe this regime on CO3Dv2 scenes (protocol of Table 1), sampling a growing fraction of the matches uniformly across all pairs of a scene and sliding the destination endpoint of each by 0–50% of the image width along its ground-truth epipolar line. Table 4 (right) shows slow degradation: against the clean run on the same CO3Dv2 scenes (78.6), 10% corruption costs 5.2 AUC@30° points and 50% costs 16.2. Even at 100%, accuracy stays at 40.7 rather than collapsing, because a match slid along its own epipolar line still constrains that pair’s rotation and translation direction.
Upstream matcher. DeepBA consumes only 2D matches, so the same trained network can run on any front-end without retraining. We evaluate it with five front-ends—sparse SIFT [32], the sparse learned SuperPoint+LightGlue (SP+LG) [6, 27] and ALIKED+LightGlue (ALIKED+LG) [27, 62], semi-dense eLoFTR [59], and dense DKM [8]—on the four multi-view benchmarks (Table 5). The four learned front-ends do not rank by match density, so density alone does not drive accuracy. SIFT gives the lowest accuracy on most of the metrics.
| Matcher | DTU [21] (49 img/scene) | Mip-NeRF 360 [1] (125–311 img/scene) | ETH3D [41] (14–76 img/scene) | IMC2021 [22] (5/10/25-view bags) | ||||||||||||||||||||
| RRA↑ @3° | RRA↑ @5° | RRA↑ @10° | RTA↑ @3° | RTA↑ @5° | RTA↑ @10° | RRA↑ @3° | RRA↑ @5° | RRA↑ @10° | RTA↑ @3° | RTA↑ @5° | RTA↑ @10° | RRA↑ @3° | RRA↑ @5° | RRA↑ @10° | RTA↑ @3° | RTA↑ @5° | RTA↑ @10° | RRA↑ @3° | RRA↑ @5° | RRA↑ @10° | RTA↑ @3° | RTA↑ @5° | RTA↑ @10° | |
| SIFT [32] | 32.2 | 41.3 | 50.7 | 25.2 | 39.5 | 54.1 | 14.2 | 21.4 | 29.4 | 10.0 | 18.7 | 29.8 | 31.5 | 45.2 | 61.3 | 14.5 | 25.8 | 44.4 | 57.0 | 67.6 | 73.3 | 21.9 | 34.3 | 52.1 |
| SP+LG [6, 27] | 69.5 | 93.4 | 100.0 | 60.3 | 87.1 | 99.5 | 64.0 | 72.2 | 82.0 | 64.3 | 77.7 | 91.7 | 35.2 | 55.7 | 72.5 | 21.5 | 35.9 | 57.6 | 87.1 | 96.1 | 99.1 | 46.7 | 66.7 | 86.2 |
| ALIKED+LG [27, 62] | 60.6 | 84.1 | 93.3 | 54.8 | 77.7 | 92.9 | 79.4 | 85.0 | 86.7 | 79.4 | 84.7 | 88.8 | 28.3 | 47.1 | 69.5 | 17.9 | 31.8 | 57.1 | 87.5 | 96.2 | 99.2 | 47.9 | 68.0 | 86.8 |
| eLoFTR [59] | 73.6 | 95.6 | 100.0 | 69.5 | 90.8 | 99.6 | 53.3 | 66.7 | 73.5 | 58.6 | 68.2 | 86.5 | 33.8 | 53.4 | 69.1 | 18.4 | 28.8 | 49.1 | 86.9 | 95.6 | 99.0 | 49.4 | 68.9 | 87.3 |
| DKM [8] | 65.5 | 92.9 | 99.7 | 74.6 | 95.7 | 99.7 | 61.3 | 68.9 | 74.5 | 63.0 | 71.3 | 90.0 | 36.1 | 52.4 | 66.1 | 26.3 | 41.4 | 59.5 | 86.6 | 95.2 | 98.0 | 51.2 | 70.3 | 87.4 |
Table 5. DeepBA generalizes across upstream matchers (§ 4.4). Bold/underline and protocols as in Table 2.
Generalization beyond the training budget. We train DeepBA with refinement iterations on view graphs of at most 48 cameras, yet at test time it extends beyond that: to more iterations and to larger scenes. On iterations, Figure 4 runs (10× the training budget) across checkpoints: early extrapolation is metastable and can degrade the estimate, but stabilizes by 40k steps. On size, DeepBA handles Mip-NeRF 360 scenes of up to 311 cameras (6.5\times its largest training graph).
Figure 4. Generalizing beyond the training iteration budget (RRA@10° left, RTA@10° right, on bicycle). Each training checkpoint is evaluated at (blue) and (red).
Runtime scaling. Figure 5 times DeepBA, COLMAP and GLOMAP on the same eLoFTR matches of the first images of bonsai, so the methods differ only in the solver (matching: 279 s at ). From on, DeepBA is the fastest of the three. At (3.4M matches), its iterations take 56 s with a GPU memory peak of 24 GB, against 988 s (0.9 GB) for COLMAP and 793 s (0.5 GB) for GLOMAP with their GPU solvers. DeepBA’s per-iteration cost is linear in the number of matches, 25 ms + 155 ns per match.
Figure 5. Left: time after matching on the first images of Mip-NeRF 360 bonsai, with every method using the same eLoFTR matches on one H100 (matching time dashed). Right: time per iteration vs. number of matches.
Conclusion
We presented DeepBA, a lightweight graph neural network that amortizes bundle adjustment. From the 2D matches of any off-the-shelf matcher and a trivial cold start, it jointly infers camera poses, intrinsics, and scene structure for arbitrarily many views. Across six benchmarks, DeepBA outperforms prior learned SfM from matches on most metrics with 71× fewer parameters than G&SM, the strongest of them, and approaches image-to-geometry transformers that consume the images. Four limits remain: the focal–depth ambiguity on planar scenes, a drift of the learned update on some scenes, the accuracy at tight thresholds, and the pinhole camera model. We hope DeepBA encourages further work on learned solvers for multi-view geometry.
Method Details
Lie-Group and Pose Conventions
We use and for the standard Lie-group logarithm and exponential between a rigid-body pose matrix and its -dim tangent vector. We overload for : . maps world coordinates to camera- coordinates, ; hence maps camera- to camera- coordinates.
Component-wise state update (, Eq. (3)). The absolute pose composes on the group, , and the log-FOV and log-depth maps update additively, , . The decoupled per-edge relative pose of Sec. 3.1 is updated component-wise. Writing , the rotation is composed on the group and the translation direction is added and renormalized onto :
Focal Length FOV
Writing for camera ’s field-of-view angle in radians, the log-FOV is ; the simple-pinhole intrinsic is
with image coordinates normalized to per axis and the aspect ratio of image , so is the horizontal field of view and pixels stay square.
Pose-Graph Residual
The per-edge pose-graph residual (Sec. 3.2.1) measures the disagreement between the directly predicted relative pose and the one derived from the absolute poses, . The rotation term is that disagreement in the tangent space,
The translation term compares directions only. Write for the unit-normalized translation of . Then is the minimal rotation carrying onto , written as an axis–angle -vector
Match-Feature Residuals
Throughout this subsection is a match on the pair , with endpoints in image and in image , written and in Sec. 3.2.2. Both are in the normalized image coordinates of Eq. (19), and denotes the homogeneous lift of a 2D point.
The four match-feature quantities of Sec. 3.2.2 are all closed form. The epipolar vector is computed from the directly predicted relative pose. Its translation magnitude is irrelevant here, since the epipolar geometry does not depend on it and is already a unit direction. The essential matrix , with , defines the epipolar line for in image (in normalized camera coordinates) as . Because the simple pinhole model has isotropic focal length , mapping the point-to-line offset back to image-plane coordinates collapses to a single scalar, and the shortest image-plane vector from to the line is
where is the first two components of .
The remaining reprojection and 3D consistency residuals use the absolute-pose-derived relative pose , whose rotation and translation are and in the sense of Sec. A.1, so that . Per-pixel depths are sampled from the current log-depth maps,
with bilinear sampling of the log-depth map at image coordinate ; unprojection gives the per-camera 3D points
and the residuals are
where is the perspective division.
Depth Head
For each match on pair , the depth head scatters onto camera ’s grid at (and onto camera ’s grid at ) using the adjoint of grid_sample. The scatter also accumulates a per-cell count , the bilinear weight of the matches landing in that cell, which divides the summed features into a per-cell mean. That mean, channel-wise layer-normalized and concatenated with , is decoded by three convolutions into the log-depth delta .
Metric Definitions
Pose Accuracy
All pose metrics are computed over the complete pair set of each scene. Let denote the evaluation cameras of a scene and its unordered pairs. For a pair , is the geodesic angle between the predicted and ground-truth relative rotations and the angle between the predicted and ground-truth relative translation directions. Both are measured in degrees, as is every threshold in this subsection. A method that produces no pose for camera (unregistered, dropped, or non-finite) is assigned for every pair containing (zero credit). With the indicator function (1 if the bracketed condition holds, else 0):
Structure Accuracy
Protocol for Table 3. Each reconstruction is aligned to the DTU world frame (millimeters) by a single similarity transform from its predicted camera centers. Both prediction and reference are restricted to the object region by DTU’s per-view masks. Writing for the nearest-neighbor distance and for the predicted and reference point sets,
Benchmark Composition
We detail the composition of the six evaluation benchmarks introduced in § 4.
CO3Dv2 [38]: object-centric turntable-style videos with wide-baseline pairs, evaluated on 202 held-out scenes across 41 categories, disjoint from our training split. Following the 10-frame protocol of VGGT [57], we sample 10 frames per scene from a fixed per-scene frame list shared across all methods, and report relative pose accuracy.
RealEstate10K [63]: indoor and outdoor video clips with smooth camera motion. We follow the same 10-frame VGGT protocol [57] over 108 clips, drawing the frames from fixed per-clip lists shared across all methods.
ETH3D [41]: the 13 high-resolution scenes of the multi-view training split, with laser-scanned ground truth. No ETH3D image enters our training mixture.
DTU [21]: 15 of the 128 laboratory-captured object scans (49 views each) with structured-light ground truth, the subset used by SPARF [53].
Mip-NeRF 360 [1]: 7 publicly released indoor and outdoor scenes (125–311 cameras each) captured in 360° sweeps around a central area.
IMC2021 [22]: all 9 in-the-wild phototourism scenes of the 2021 Image Matching Challenge, evaluated under the official bag protocol of 175 bags per scene (100 of 5 views, 50 of 10 and 25 of 25, 1,575 bags in total).
Baselines on eLoFTR. The eLoFTR rows of COLMAP/GLOMAP/ResFM/GAsfM receive DeepBA’s cached, RANSAC-verified eLoFTR correspondences. COLMAP/GLOMAP/ResFM/GAsfM need keypoints shared across pairs. Each image’s match endpoints are therefore snapped to square cells, with one keypoint per cell at the mean of its endpoints. Each mapper then runs its own geometric verification and track building. Union-find on a grid cell of 1/400 of the image width (4 px at 1,600 px) chains the matches into tracks, which are kept when seen in at least 3 views.
Training and Implementation Details
These expand the headline configuration of § 4.1.
Gradient-iteration sampling. Each step runs a -iteration Markov trajectory but takes gradients on only the first 12 iterations plus another 12 iterations sampled uniformly from (24 iterations in total with backpropagation of Eq. (17); the remaining 76 run under no_grad). We denote this set of 24 iterations by . We backpropagate per refinement iteration and detach the state at each gradient boundary, giving memory in the number of refinement iterations.
Scene and graph sampling. At each training step, we sample a connected set of cameras from a randomly chosen scene, primarily with a Sampson-gated graph walk over wide-baseline pairs. We add random edges of the view graph so that every camera shares a view with at least one other. Robust kernel. The smooth- (Huber) kernel of § 3.4 is
Convergence against the Refinement Iteration
Figure 6 profiles accuracy against the number of refinement iterations. We measure it on Mip-NeRF 360, which is absent from the training mixture. Its scenes carry 125–311 views, 2.6× to 6.5× the largest (48-camera) view graphs DeepBA trains on. From a cold start, accuracy climbs steeply over the first tens of iterations and then saturates, at a scene-dependent rate, with garden and stump settling within ∼30 iterations and the remaining scenes flattening out well before the end of the profile. We train DeepBA with . The mean RRA@10° keeps rising until ∼150 iterations and is flat beyond. Extending inference to raises it from 73.5 to 82.2, mainly through counter, but lowers the mean RTA@10° from 86.5 to 84.5, mainly through bonsai. With no consistent gain from more iterations, we evaluate every benchmark at .

Figure 6. Pose accuracy vs. refinement iteration (RRA@10° left, RTA@10° right). Light curves are per-scene; the solid black line is the across-scene mean, the dotted line the mean excluding the view-graph-bottlenecked room scene.
Failure Analysis
ETH3D
ETH3D is DeepBA’s weakest multi-view benchmark (Table 2). Three causes explain most of the losses (Figure 7, Figure 8).

Figure 7. Causes of the ETH3D losses. (a) courtyard and (b) facade: eLoFTR matches between two different walls with identical windows (red: ground-truth Sampson error above 20 px, green: at most 4 px, yellow: in between). (c) relief: the eLoFTR view graph, with an image per node and an edge per matched pair, splits into the rotated close-ups (left) and the upright views (right), with no matched pair between them.

Figure 8. The focal–depth ambiguity on ETH3D terrains. Top: two views facing the wall (blue, orange) and one corridor view (green). Below: DeepBA in red against ground truth in grey, with the estimated (upper) and ground-truth (lower) focal length. The three example views keep their colors, solid for DeepBA and dashed for ground truth.
Repeated structure. courtyard and facade show building walls with identical window grids. eLoFTR matches the windows of one wall to those of another, so many pairs consist almost entirely of wrong matches that are mutually consistent (Figure 7a,Figure 7b). DeepBA fits these pairs and places the cameras in a wrong configuration.
Disconnected views. Several scenes mix images rotated by 90° in the image plane with upright ones (Figure 7c), or include views of the sky or a ceiling. eLoFTR is not rotation invariant and finds no reliable matches between such groups. The view graph then splits (Figure 7c).
Focal–depth ambiguity. Of the 42 views of terrains, 18 face a flat wall head-on. Their rotations are right, but
DeepBA overestimates their focal length and places them too far from the wall (Figure 8), the pattern of Sec. F.2. Fixing the focal length to its ground-truth value repairs most of terrains.
IMC2021
IMC2021 is the only benchmark in the evaluation set where each camera has different intrinsics. A scene with largely planar facades constrains only the ratio of focal length to depth rather than either quantity alone. A mis-estimated focal length is therefore absorbed by a compensating depth for that camera, displacing its center along its own optical axis while leaving its viewing directions unchanged (Figure 9). The relative translation directions tilt while the relative rotations stay intact, which is the pattern of Table 2.

Figure 9. The focal–depth ambiguity on one IMC2021 bag (milan_cathedral, 5 views). DeepBA in red against ground truth in grey; the circled camera, shown with its ground truth, is displaced by the focal–depth ambiguity.
Mip-NeRF 360
The room failure. We attribute the room failure to the view graph rather than to scene size. Of room’s 311 cameras and 5,829 verified pairs, a spectral bipartition separates 136 cameras from the other 175, and only 128 pairs cross between the two. As a result, the two clusters are joined too weakly to fix their relative pose, so each is reconstructed well internally while the two are placed independently of one another (Figure 10a).

Figure 10. The room and bonsai failures on Mip-NeRF 360. DeepBA’s reconstructions, with the estimated cameras in green and the ground-truth poses in grey (every third camera in (b)). Only points with a reprojection error below 10 px are shown.
The bonsai failure. Unlike room, bonsai has no view-graph bottleneck and almost no wrong matches. Instead, every camera is pitched up by about the same angle (, Figure 10b), so cameras on opposite sides of the orbit are rotated against each other and most pairs miss the threshold.
The tilt is a drift of DeepBA’s learned update. Even starting from the ground truth (zero error), DeepBA tilts the
cameras to a median pitch error of in 100 iterations, and starting from its own output, it keeps tilting them further (Figure 11). Fewer cameras slow the drift but do not stop it. On 32 evenly spread cameras, fewer than the 48 cameras of the largest training view graphs, DeepBA started from the ground truth still reaches . This example shows a limitation of DeepBA.
Figure 11. The bonsai pitch drift. Median camera pitch error over DeepBA’s iterations on bonsai with 32 to 292 evenly spread cameras, started from DeepBA’s output (left) and from the ground truth (right).
Camera Localization
DeepBA also performs camera localization with no retraining and no architectural change. We build a map from a set of reference views, freeze its cameras and per-view depths, and register each query by rerunning the same refinement iteration with the reference updates zeroed, so no PnP is involved (Figure 12). This is possible because DeepBA’s intermediate state is explicit geometry, absolute poses, per-view depths and intrinsics, so any subset of it can simply be held fixed, whereas a feed-forward regressor carrying only latent features would need an external solver. Figure 13 compares query trajectories against PnP+RANSAC, PnP+RANSAC and HLoc on the same frozen map, and Table 6 reports 7-Scenes under the protocol of HLoc [39], mapping from the official TrainSplit, querying the TestSplit, and reporting the median translation and rotation error per scene. Except for the published HLoc row, every method in that comparison receives the same 200-image DeepBA map rather than the full TrainSplit images, and both PnP+RANSAC and HLoc additionally receive ground-truth query intrinsics. Localization is a by-product of the reconstruction model rather than a main contribution of this paper.
| Method | Intrinsics required? | Chess | Fire | Heads | Office | Pumpkin | Kitchen | Stairs |
| HLoc [39], ∼4,000-image map | ✓ | 2/0.79 | 2/0.87 | 2/0.92 | 3/0.91 | 5/1.12 | 4/1.25 | 6/1.62 |
| Same map, same queries – only the registration step differs | ||||||||
| HLoc [39], our map | ✓ | 11/1.88 | 10/1.89 | 6/1.90 | 9/4.75 | 16/3.85 | 10/4.37 | 37/17.86 |
| PnP+RANSAC+GT K | ✓ | 12/1.91 | 11/1.67 | 6/2.04 | 10/4.68 | 15/3.71 | 10/4.21 | 37/17.39 |
| PnPf+RANSAC | ✗ | 6/1.66 | 5/1.67 | 3/2.01 | 11/4.84 | 10/3.75 | 9/3.96 | 50/17.35 |
| DeepBA | ✗ | 3/1.66 | 3/1.67 | 2/1.66 | 8/4.80 | 8/3.86 | 6/4.30 | 30/18.30 |
Table 6. Camera localization on 7-Scenes [42]. Median translation/rotation error (cm/). The first row is HLoc [39] as published. The other rows share one frozen DeepBA map built from 200 TrainSplit images. Bold/underline: best/second best among them.

Figure 12. Localization reruns the same refinement iteration in place. With the reference map frozen (its 3D points in their image colors, reference cameras in blue), two example query views (rows) are registered across refinement iterations 0, 1, 2, 10 (columns). From the mean pose of the reference cameras, the query camera and its lifted 3D points (green) converge onto its ground-truth camera (grey) within iterations.

Figure 13. (a) Object-centric orbit (CO3Dv2). (b) Indoor scene (ScanNet). Localization on a frozen DeepBA map. Query-camera trajectories for four methods registering the same held-out queries against the same frozen DeepBA map, vs. ground truth (grey): DeepBA (green), PnP+RANSAC given ground-truth intrinsics (orange), focal-solving PnPf+RANSAC without known intrinsics (pink), and HLoc [39] (cyan).
References
- [1]J. T. Barron, B. Mildenhall, D. Verbin, P. P. Srinivasan, and P. Hedman. Mip-NeRF 360: Unbounded anti-aliased neural radiance fields. In CVPR, 2022. 1, 6, 7, 8, 12
- [2]L. Brynte, J. P. Iglesias, C. Olsson, and F. Kahl. Learning structure-from-motion with graph attention networks. In CVPR, 2024. 2, 3, 6, 7arxiv.org/abs/2308.15984
- [3]P. H. Chui, H. Andrade-Loarca, and D. Cremers. Track-free dense structure from motion. In WACV, 2027. 2
- [4]R. Clark, M. Bloesch, J. Czarnowski, S. Leutenegger, and A. J. Davison. LS-Net: Learning to solve nonlinear least squares for monocular stereo. In ECCV, 2018. 3
- [5]A. Dai, A. X. Chang, M. Savva, M. Halber, T. Funkhouser, and M. Nießner. ScanNet: Richly-annotated 3D reconstructions of indoor scenes. In CVPR, 2017. 6
- [6]D. DeTone, T. Malisiewicz, and A. Rabinovich. SuperPoint: Self-supervised interest point detection and description. In CVPR Workshops, 2018. 8arxiv.org/abs/1712.07629
- [7]B. Duisterhof, L. Zust, P. Weinzaepfel, V. Leroy, Y. Cabon, and J. Revaud. MASt3R-SfM: A fully-integrated solution for unconstrained structure-from-motion. In 3DV, 2025. 2, 6, 7
- [8]J. Edstedt, I. Athanasiadis, M. Wadenbäck, and M. Felsberg. DKM: Dense kernelized feature matching for geometry estimation. In CVPR, 2023. 2, 8arxiv.org/abs/2202.00667
- [9]J. Edstedt, Q. Sun, G. Böckman, M. Wadenbäck, and M. Felsberg. RoMa: Robust dense feature matching. In CVPR, 2024. 2DOI
- [10]O. Enqvist, F. Kahl, and C. Olsson. Non-sequential structure from motion. In ICCV Workshops, 2011. 6DOI
- [11]M. A. Fischler and R. C. Bolles. Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, 24(6):381–395, 1981. 6
- [12]R. Girshick. Fast R-CNN. In ICCV, 2015. 5
- [13]H. Han and H. Yang. Building Rome with convex optimization. arXiv preprint arXiv:2502.04640, 2025. 2arxiv.org/abs/2502.04640
- [14]R. Hartley and A. Zisserman. Multiple View Geometry in Computer Vision. Cambridge University Press, 2003. 1, 2DOI
- [15]X. He, J. Sun, Y. Wang, S. Peng, Q. Huang, H. Bao, and X. Zhou. Detector-free structure from motion. In CVPR, 2024. 2arxiv.org/abs/2306.15669
- [16]J. Heinly, E. Dunn, and J.-M. Frahm. Correcting for duplicate scene structure in sparse 3D reconstruction. In ECCV, 2014. 2DOI
- [17]J. H. Hong and C. Zach. pOSE: Pseudo object space error for initialization-free bundle adjustment. In CVPR, 2018. 2DOI
- [18]J. H. Hong, C. Zach, A. Fitzgibbon, and R. Cipolla. Projective bundle adjustment from arbitrary initialization using the variable projection method. In ECCV, 2016. 2DOI
- [19]P. J. Huber. Robust estimation of a location parameter. The Annals of Mathematical Statistics, 35(1):73–101, 1964. 5
- [20]J. P. Iglesias, A. Nilsson, and C. Olsson. expOSE: Accurate initialization-free projective factorization using exponential regularization. In CVPR, 2023. 2DOI
- [21]R. Jensen, A. Dahl, G. Vogiatzis, E. Tola, and H. Anaes. Large scale multi-view stereopsis evaluation. In CVPR, 2014. 6, 7, 8, 12DOI
- [22]Y. Jin, D. Mishkin, A. Mishchuk, J. Matas, P. Fua, K. M. Yi, and E. Trulls. Image matching across wide baselines: From paper to practice. IJCV, 129(2):517–547, 2021. 6, 7, 8, 12DOI
- [23]F. Khatib, Y. Kasten, D. Moran, M. Galun, and R. Basri. RESfM: Robust deep equivariant structure from motion. In ICLR, 2025. 3, 6, 7arxiv.org/abs/2404.14280
- [24]V. Leroy, Y. Cabon, and J. Revaud. Grounding image matching in 3D with MASt3R. In ECCV, 2024. 2, 6arxiv.org/abs/2406.09756
- [25]X. Li and H. Ling. PoGO-Net: Pose graph optimization with graph neural networks. In ICCV, 2021. 3
- [26]Z. Li and N. Snavely. MegaDepth: Learning single-view depth prediction from internet photos. In CVPR, 2018. 6arxiv.org/abs/1804.00607
- [27]P. Lindenberger, P.-E. Sarlin, and M. Pollefeys. LightGlue: Local feature matching at light speed. In ICCV, 2023. 8arxiv.org/abs/2306.13643
- [28]L. Ling, Y. Sheng, Z. Tu, W. Zhao, C. Xin, K. Wan, L. Yu, Q. Guo, Z. Yu, Y. Lu, et al. DL3DV-10K: A large-scale scene dataset for deep learning-based 3D vision. In CVPR, 2024. 6
- [29]L. Lipson, Z. Teed, and J. Deng. RAFT-Stereo: Multilevel recurrent field transforms for stereo matching. In 3DV, 2021. 3arxiv.org/abs/2109.07547
- [30]I. Loshchilov and F. Hutter. Decoupled weight decay regularization. In ICLR, 2019. arXiv:1711.05101. 6arxiv.org/abs/1711.05101
- [31]M. I. A. Lourakis and A. A. Argyros. SBA: A software package for generic sparse bundle adjustment. ACM Transactions on Mathematical Software (TOMS), 36(1):1–30, 2009. 2
- [32]D. G. Lowe. Distinctive image features from scale-invariant keypoints. IJCV, 60:91–110, 2004. 2, 8DOI
- [33]D. Moran, H. Koslowsky, Y. Kasten, H. Maron, M. Galun, and R. Basri. Deep permutation equivariant structure from motion. In ICCV, 2021. 3arxiv.org/abs/2104.06703
- [34]R. Mur-Artal, J. M. M. Montiel, and J. D. Tardos. ORB-SLAM: A versatile and accurate monocular SLAM system. IEEE Transactions on Robotics, 31(5):1147–1163, 2015. 1
- [35]D. Nistér. An efficient solution to the five-point relative pose problem. IEEE TPAMI, 26(6):756–770, 2004. 2
- [36]C. Olsson and O. Enqvist. Stable structure from motion for unordered image collections. In Scandinavian Conference on Image Analysis (SCIA), 2011. 6DOI
- [37]L. Pan, D. Baráth, M. Pollefeys, and J. L. Schönberger. Global structure-from-motion revisited. In ECCV, 2024. 2, 6, 7arxiv.org/abs/2407.20219
- [38]J. Reizenstein, R. Shapovalov, P. Henzler, L. Sbordone, P. Labatut, and D. Novotny. Common objects in 3D: Large-scale learning and evaluation of real-life 3D category reconstruction. In ICCV, 2021. 6, 12arxiv.org/abs/2109.00512
- [39]P.-E. Sarlin, C. Cadena, R. Siegwart, and M. Dymczyk. From coarse to fine: Robust hierarchical localization at large scale. In CVPR, 2019. 14, 15arxiv.org/abs/1812.03506
- [40]J. L. Schönberger and J.-M. Frahm. Structure-from-motion revisited. In CVPR, 2016. 1, 2, 6, 7DOI
- [41]T. Schöps, J. L. Schönberger, S. Galliani, T. Sattler, K. Schindler, M. Pollefeys, and A. Geiger. A multi-view stereo benchmark with high-resolution images and multi-camera videos. In CVPR, 2017. 6, 7, 8, 12DOI
- [42]J. Shotton, B. Glocker, C. Zach, S. Izadi, A. Criminisi, and A. Fitzgibbon. Scene coordinated regression forests for camera relocalization in RGB-D images. In CVPR, 2013. 15DOI
- [43]C. Smith, D. Charatan, A. Tewari, and V. Sitzmann. FlowMap: High-quality camera poses, intrinsics, and depth via gradient descent. arXiv preprint arXiv:2404.15259, 2024. 2arxiv.org/abs/2404.15259
- [44]H. Strasdat, J. M. M. Montiel, and A. J. Davison. Scale drift-aware large scale monocular SLAM. In Robotics: Science and Systems (RSS), 2010. 2
- [45]J. Sun, Z. Shen, Y. Wang, H. Bao, and X. Zhou. LoFTR: Detector-free local feature matching with transformers. In CVPR, 2021. 2arxiv.org/abs/2104.00680
- [46]T. Tanaka, Y. Sasagawa, and T. Okatani. Learning to bundle-adjust: A graph network approach to faster optimization of bundle adjustment for vehicular SLAM. In ICCV, 2021. 3DOI
- [47]C. Tang and P. Tan. BA-Net: Dense bundle adjustment networks. In ICLR, 2019. 3
- [48]Z. Teed and J. Deng. RAFT: Recurrent all-pairs field transforms for optical flow. In ECCV, 2020. 3arxiv.org/abs/2003.12039
- [49]Z. Teed and J. Deng. DROID-SLAM: Deep visual SLAM for monocular, stereo, and RGB-D cameras. In NeurIPS, 2021. 3arxiv.org/abs/2108.10869
- [50]Z. Teed and J. Deng. RAFT-3D: Scene flow using rigid-motion embeddings. In CVPR, 2021. 3arxiv.org/abs/2012.00726
- [51]P. H. S. Torr, A. W. Fitzgibbon, and A. Zisserman. The problem of degeneracy in structure and motion recovery from uncalibrated image sequences. IJCV, 32(1):27–44, 1999. 2DOI
- [52]B. Triggs, P. F. McLauchlan, R. I. Hartley, and A. W. Fitzgibbon. Bundle adjustment — a modern synthesis. In International Workshop on Vision Algorithms, 1999. 1, 2DOI
- [53]P. Truong, M.-J. Rakotosaona, F. Manhardt, and F. Tombari. SPARF: Neural radiance fields from sparse and noisy poses. In CVPR, 2023. 12arxiv.org/abs/2211.11738
- [54]A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin. Attention is all you need. In NeurIPS, 2017. 4
- [55]J. Wang, C. Rupprecht, and D. Novotny. PoseDiffusion: Solving pose estimation via diffusion-aided bundle adjustment. In ICCV, 2023. 2, 6arxiv.org/abs/2306.15667
- [56]J. Wang, N. Karaev, C. Rupprecht, and D. Novotny. VGGSfM: Visual geometry grounded deep structure from motion. In CVPR, 2024. 2, 6DOI
- [57]J. Wang, M. Chen, N. Karaev, A. Vedaldi, C. Rupprecht, and D. Novotny. VGGT: Visual geometry grounded transformer. In CVPR, 2025. 2, 6, 7, 12arxiv.org/abs/2503.11651
- [58]S. Wang, V. Leroy, Y. Cabon, B. Chidlovskii, and J. Revaud. DUSt3R: Geometric 3D vision made easy. In CVPR, 2024. 2, 6arxiv.org/abs/2312.14132
- [59]Y. Wang, X. He, S. Peng, D. Tan, and X. Zhou. Efficient LoFTR: Semi-dense local feature matching with sparse-like speed. In CVPR, 2024. 6, 8arxiv.org/abs/2403.04765
- [60]S. Weber, J. H. Hong, and D. Cremers. Power variable projection for initialization-free large-scale bundle adjustment. In ECCV, 2024. 2arxiv.org/abs/2405.05079
- [61]Y. Yao, Z. Luo, S. Li, J. Zhang, Y. Ren, L. Zhou, T. Fang, and L. Quan. BlendedMVS: A large-scale dataset for generalized multi-view stereo networks. In CVPR, 2020. 6arxiv.org/abs/1911.10127
- [62]X. Zhao, X. Wu, W. Chen, P. C. Y. Chen, Q. Xu, and Z. Li. ALIKED: A lighter keypoint and descriptor extraction network via deformable transformation. IEEE Transactions on Instrumentation and Measurement, 2023. 8arxiv.org/abs/2304.03608
- [63]T. Zhou, R. Tucker, J. Flynn, G. Fyffe, and N. Snavely. Stereo magnification: Learning view synthesis using multiplane images. In SIGGRAPH, 2018. 6, 12arxiv.org/abs/1805.09817


