betaFeedback

WorksComputer Science

Deep Bundle Adjustment

Abstract

We present DeepBA (Deep Bundle Adjustment), a graph neural network (GNN) that recovers camera poses, intrinsics, and scene structure from 2D matches across an arbitrary number of views. Rather than solving a non-linear least-squares problem per scene, DeepBA amortizes bundle adjustment into a learned graph network that iteratively refines its estimate from geometric residuals. Where classical SfM computes its updates analytically and needs careful initialization, DeepBA learns them and converges from a cold start. It is trained with scale-invariant supervision (ground-truth rotation, translation direction, and intrinsics, plus reprojection self-consistency), so datasets with arbitrary metric units can be mixed without per-scene normalization. DeepBA is decoupled from the matcher. At 2.04M parameters, it outperforms prior learned SfM from matches on scene reconstruction and on most pose metrics, is competitive with classical SfM at moderate accuracy thresholds and overtakes it in wall time as the view count grows, and approaches monolithic image-to-geometry transformers such as VGGT (∼\sim1.2B parameters) with greater modularity. Project~page: https://cphjsn.github.io/deep-ba/

Introduction

Recovering camera poses and scene geometry from a set of images is a foundational problem in 3D computer vision, addressed by structure-from-motion (SfM) [40] and simultaneous localization and mapping (SLAM) [34]. Most modern SfM pipelines and keyframe-based SLAM systems rely on the same classical solver, bundle adjustment (BA) [14, 52], which jointly refines cameras and 3D points by minimizing reprojection residuals via non-linear least squares. Bundle adjustment is near-optimal given a good initialization, but its non-convex landscape is fragile. Incremental SfM obtains such an initialization through heuristics and per-scene tuning [40], yet these heuristics break down on weak baselines or planar structure [51]. Meanwhile, learning-based semi-dense [45] and dense [8, 9] matchers have made matches far more reliable than SIFT-style keypoints [32], shifting the bottleneck from matching to the solver. However, feeding them into a COLMAP-style pipeline does not automatically help, as its rigid track building is overwhelmed by the volume and its initialization stays brittle. This mismatch motivated a line of feed-forward 3D networks (DUSt3R [58], MASt3R [24], and VGGT [57]) that regress geometry directly from images, trading the classical pipeline’s modularity for robustness. There is a missing middle: a learned solver that amortizes bundle adjustment itself, while leaving the matcher decoupled and solving any number of views at once. Classical bundle adjustment already operates on a graph of cameras and their 2D matches, so a graph neural network is the natural architecture for that structure.

We introduce DeepBA (Deep Bundle Adjustment, Figure 1), a GNN that takes 2D matches from an off-the-shelf matcher and jointly infers absolute camera poses, intrinsics, and 3D points. DeepBA replaces the per-scene non-linear least-squares solver with a learned Markovian iterative refinement. Each iteration recomputes geometric features from the current state and predicts an updated state via shared GNN weights. The number of iterations is a free choice at test time, independent of the training schedule: accuracy rises with the iteration count until it saturates, mirroring an iterative classical solver. It operates directly on pairwise matches, with no separate track-building stage. Our contributions are:

DeepBA runs the bundle-adjustment loop with a learned update, on a track-free state

Figure 1. DeepBA runs the bundle-adjustment loop with a learned update, on a track-free state. From a view graph G\mathcal{G} built by any off-the-shelf matcher (Input), a weight-shared GNN iteratively refines camera poses, intrinsics, and depth from a trivial cold start s0s_0. The example is the full 240-camera Mip-NeRF 360 [1] counter scene, with the estimated cameras in green.

  • Amortized cold-start bundle adjustment via a GNN. We formulate bundle adjustment as a learned iterative refinement over the view graph of any number of views, with a Markovian update that lets the user trade runtime for accuracy at test time. It requires no initial poses or intrinsics and converges from a cold start, where classical BA fails.

  • Scale-invariant, anchor-free, multi-dataset supervision. Chordal rotation, direction-only translation, intrinsics, self-consistent reprojection and 3D losses train DeepBA on datasets with arbitrary metric units without per-scene normalization. Because DeepBA predicts state updates rather than geometry, it needs no anchor camera, unlike VGGT [57], which uses the first camera as its reference frame.

  • Robustness across matcher families. DeepBA sees only 2D matches, so a trained network can run from sparse to dense matching without retraining. We demonstrate this extensively, across every matcher family.

  • Competitive accuracy at a fraction of the size. With 2.04M parameters (~71×71 \times fewer than the strongest prior match-based network, GASfM [2] at 145M), DeepBA outperforms learned SfM from matches on most metrics

and is competitive with classical SfM at moderate thresholds, approaching image-to-geometry transformers such as VGGT (~1.2B parameters).

Related Work

Classical structure-from-motion and bundle adjustment. The classical pipeline for multi-view reconstruction proceeds by feature extraction, pairwise matching, geometric verification, incremental pose estimation, and a final non-linear refinement via BA [14, 52]. COLMAP [40] is its standard implementation and the baseline most work compares against. GLOMAP [37] replaces its incremental stage with global rotation averaging followed by a positioning step that solves for all cameras and points at once. That drops the seed-and-order heuristics, but leans on the pairwise rotations instead. Bundle adjustment’s accuracy hinges on initialization [31] and hand-tuned heuristics, making it brittle on weak baselines, planar or low-texture scenes, and repetitive structure [15, 16, 51]. A long line of work has widened bundle adjustment’s basin of attraction with alternative error parameterizations and convex relaxations [13, 17, 18, 20, 60], which improves cold-start convergence. These remain per-scene optimizations and still face the mirror ambiguity and the global-scale gauge freedom of unconstrained multi-view geometry [35, 44]. Recent neural pipelines inherit the same structure, still running an optimization once per scene at test time. For example, Detector-Free SfM [15] adapts track building to dense matchers, and FlowMap [43] fits poses, intrinsics, and depth by gradient descent through a dense-flow objective. TFD-SfM [3] removes track building altogether. It lifts both pixels of every correspondence to their own 3D landmarks and optimizes the reconstruction from a cold start. Similar to TFD-SfM, DeepBA is track-free but amortizes the per-scene optimization into a learned refinement from a cold start.

Feed-forward 3D reconstruction. Recent works remove the classical pipeline, training neural networks to regress geometry directly from images. PoseDiffusion [55] diffuses camera poses alone. DUSt3R [58] and MASt3R [24] produce per-camera pointmaps in a shared frame, with MASt3R adding an explicit matching head for downstream localization. Both are pair-only: extending them to many views needs a global-alignment step (e.g. MASt3R-SfM [7]) that re-introduces a per-scene non-linear optimization. VG-GSfM [56] combines deep pose and triangulator modules with a differentiable bundle adjustment back-end, while VGGT [57] pushes furthest, absorbing the matcher, visual encoder, and geometry solver into a single transformer that ingests dozens of views in one forward pass, at the cost of model size, training-data scale, and tight perception–geometry coupling. DeepBA instead operates on matches rather than image features and reasons jointly over all views, needing no fusion of pairwise reconstructions. Keeping the matcher external rather than learning one, it is ∼590×\sim590\times smaller than VGGT and trainable on substantially less data.

Iterative learned refinement. RAFT [48] introduced iterative refinement with a fixed update network applied for a variable number of iterations, recomputing features from the current estimate at each one. The template has been extended to stereo, scene flow, and visual SLAM [29, 49, 50]. What makes the template work is supervising the intermediate iterations. The network cannot then rely on one large correction, and instead learns a contraction, an update that improves the estimate every time it is applied. The number of iterations therefore becomes a test-time trade between accuracy and runtime. DeepBA carries this template into the multi-view geometry setting with a weight-shared GNN (a Markovian update, without a recurrent gate). The closest learned refiner is DROID-SLAM [49], which targets calibrated, temporally-ordered video with a coupled dense feature network, whereas DeepBA handles unordered, wide-baseline collections and recovers intrinsics from a trivial initialization.

Learning-based SfM from matches. The work closest to ours also learns SfM directly from 2D matches rather than from pixels. Most of it regresses the reconstruction with a network and ends in a classical bundle adjustment. Deep Permutation Equivariant SfM [33] and its successor RESfM [23] regress cameras and sparse structure by minimizing reprojection error. Both fine-tune the network on each test scene before the final bundle adjustment. GASfM [2] instead re-places the equivariant layers of [33] with graph attention over the same point tracks. It is feed-forward, restricted to a calibrated or purely projective regime, and refined by a final classical bundle adjustment. LS-Net [4] learns a non-linear least-squares solver for dense, calibrated two-view stereo. Learning-to-Bundle-Adjust [46] predicts BA updates with a graph network, refining the local map of an initialized vehicular SLAM system. BA-Net [47] starts its poses at the identity but runs a dense feature-metric BA on calibrated short clips, with a learned Levenberg–Marquardt damping and an image-dependent depth basis. PoGO-Net [25] is a graph network for rotation/motion averaging that consumes relative poses, not raw matches, and recovers only orientations. Unlike these methods, DeepBA learns the update of poses, intrinsics and depth itself. It recomputes geometric features of the matches from its current estimate at every refinement iteration. It starts every scene from the same initial state.

Method

We introduce DeepBA, a graph neural network that performs amortized bundle adjustment via iterative refinement over a graph of cameras and their pairwise matches (Figure 2).

DeepBA architecture

Figure 2. DeepBA architecture. Left: one refinement iteration: the scene state sks_k and view graph G\mathcal{G} are encoded into (hpair,hmatch)(\mathbf{h}^{\mathrm{pair}},\mathbf{h}^{\mathrm{match}}), re-initialized every iteration (Markovian), mixed by LL MPC blocks, and decoded into the state delta Δsk\Delta s_k composed via ⊞\boxplus (Eq. (3)); the block is weight-shared across the KK iterations. Right: one MPC block; its three message-passing steps are specified in § 3.3.

Problem Setting and Notation

Given NN unordered images of a static scene, DeepBA maps pairwise 2D matches to per-image estimates of absolute pose, intrinsics, and a depth map (3D points are recovered by unprojection). The matches are image-point pairs from an off-the-shelf matcher. The inputs form a view graph G=(V,Epair,Ematch)\mathcal{G} = (\mathcal{V}, \mathcal{E}_{\mathrm{pair}}, \mathcal{E}_{\mathrm{match}}) in which V\mathcal{V} is the set of NN cameras, Epair\mathcal{E}_{\mathrm{pair}} is the set of PP pairs of cameras connected by at least 50 matches, and Ematch\mathcal{E}_{\mathrm{match}} is the set of all MM matches over all pairs of the view graph. Each camera i∈Vi \in\mathcal{V} carries an absolute pose TiabsT_i^{\mathrm{abs}}, a log field of view (log-FOV) ϕi\phi_i (our default camera model, more detail in Sec. A.2), and a log-depth map Di∈R1×Hd×WdD_i \in\mathbb{R}^{1 \times H_d \times W_d} on a fixed grid with Hd=Wd=32H_d = W_d = 32. Each pair (i,j)∈Epair(i,j) \in\mathcal{E}_{\mathrm{pair}} carries a relative pose Tijrel=(t^ijrel,ωijrel)T_{ij}^{\mathrm{rel}} = (\hat{\mathbf{t}}_{ij}^{\mathrm{rel}}, \boldsymbol{\omega}_{ij}^{\mathrm{rel}}) in a decoupled parameterization: a unit translation direction t^ijrel∈S2\hat{\mathbf{t}}_{ij}^{\mathrm{rel}} \in S^2 and a rotation vector ωijrel∈R3\boldsymbol{\omega}_{ij}^{\mathrm{rel}} \in\mathbb{R}^3. Together, these form the scene state

\mathbf{s} := \left(\{T_i^{\mathrm{abs}}\}_{i\in\mathcal{V}},\ \{T_{ij}^{\mathrm{rel}}\}_{(i,j)\in\mathcal{E}_{\mathrm{pair}},\ \{\phi_i\}_{i\in\mathcal{V}},\ \{D_i\}_{i\in\mathcal{V}}\right).} \tag*{(1)}

The absolute poses, FOVs and depth maps already define the reconstruction. We still keep a relative pose per pair, which the network refines at every iteration under ground-truth (GT) supervision, while the absolute poses receive no GT supervision. The motivation of this design comes from pose-graph optimization. The absolute poses are updated to agree with the current relative poses instead of being pulled directly toward the GT, so they follow the gradual refinement of the relative poses. A relative pose is only a rotation and a translation direction, so its GT supervision needs no metric scale, while the reprojection error sets the translation lengths of the absolute poses. At every refinement iteration, the network predicts a delta

\Delta\mathbf{s} = \left(\{\Delta T_i^{\mathrm{abs}}\}_{i\in\mathcal{V}},\ \{\Delta T_{ij}^{\mathrm{rel}}\}_{(i,j)\in\mathcal{E}_{\mathrm{pair}},\ \{\Delta\phi_i\}_{i\in\mathcal{V}},\ \{\Delta D_i\}_{i\in\mathcal{V}}\right),} \tag*{(2)}

with ΔTiabs∈R6\Delta T_i^{\mathrm{abs}} \in\mathbb{R}^6 in Lie-algebra coordinates, and composes it onto the current state,

sk+1=sk⊞Δsk,(3)\mathbf{s}_{k+1} = \mathbf{s}_k \boxplus\Delta\mathbf{s}_k, \tag*{(3)}

where ⊞\boxplus acts component-wise (exact update rules and conventions in Sec. A.1).

Per-Iteration Geometric Features

At every refinement iteration, DeepBA computes three hidden states from scratch (Markovian: no hidden state crosses iterations): the per-pair hpair\mathbf{h}^{\mathrm{pair}} and per-match hmatch\mathbf{h}^{\mathrm{match}} encode closed-form geometric features of the current scene state, while the per-camera hcam\mathbf{h}^{\mathrm{cam}} takes no geometric input and starts from a single learned embedding ecam∈Rdh\mathbf{e}^{\mathrm{cam}} \in\mathbb{R}^{d_h} shared by every camera. All three hidden states have dimension dh=128d_h = 128.

Pair Geometric Features

The pair geometric features give the network pairwise information. Although a pair stores a single relative pose, we featurize it in both directions so that message passing in the GNN reaches both of its cameras: we construct a 14-dim feature per direction, fijpair\mathbf{f}^{\mathrm{pair}}_{ij} for the forward edge (i→j)(i \to j) and fjipair\mathbf{f}^{\mathrm{pair}}_{ji} for the inverse (j→i)(j \to i), and encode each into its own per-pair hidden state, hijpair\mathbf{h}^{\mathrm{pair}}_{ij} and hjipair\mathbf{h}^{\mathrm{pair}}_{ji}. The forward feature is

fijpair=[t^ijrel,ωijrel⏟rel-pose state (6),ϕi,ϕj⏟intrinsics (2),δijt,δijR⏟pose-graph residual (6)](4)\mathbf{f}^{\mathrm{pair}}_{ij} = \left[ \underbrace{\hat{\mathbf{t}}^{\mathrm{rel}}_{ij},\boldsymbol{\omega}^{\mathrm{rel}}_{ij}}_{\text{rel-pose state (6)}}, \underbrace{\phi_i,\phi_j}_{\text{intrinsics (2)}}, \underbrace{\delta^{\mathrm{t}}_{ij},\delta^{\mathrm{R}}_{ij}}_{\text{pose-graph residual (6)}} \right] \tag*{(4)}

where the pose-graph residual measures the disagreement between the directly predicted relative pose TijrelT^{\mathrm{rel}}_{ij} and the counterpart TijabsT^{\mathrm{abs}}_{ij} derived from the two absolute poses (exact form in Sec. A.3).

Match Geometric Features

The match geometric features tell the network how well the current state explains each match. Each match eij∈Ematche_{ij} \in\mathcal{E}_{\mathrm{match}}, with endpoints usrceij,udsteij∈R2\mathbf{u}^{e_{ij}}_{\mathrm{src}},\mathbf{u}^{e_{ij}}_{\mathrm{dst}} \in\mathbb{R}^2 in images ii and jj, contributes a 12-dim feature per direction—feijmatch\mathbf{f}^{\mathrm{match}}_{e_{ij}} for the forward edge (i→j)(i \to j) and fejimatch\mathbf{f}^{\mathrm{match}}_{e_{ji}} for the inverse—and is encoded into its own per-match hidden state heijmatch\mathbf{h}^{\mathrm{match}}_{e_{ij}} and hejimatch\mathbf{h}^{\mathrm{match}}_{e_{ji}}. For brevity, we drop the edge subscript and write e:=eije := e_{ij} throughout the remainder of this subsection. The forward feature is

fematch=[usrce,udste⏟4,ϵe⏟2,ρe⏟2,log⁡∥Pdste∥⏟1,δe3D⏟3](5)\mathbf{f}^{\mathrm{match}}_e = \left[ \underbrace{\mathbf{u}^{e}_{\mathrm{src}},\mathbf{u}^{e}_{\mathrm{dst}}}_{4}, \underbrace{\boldsymbol{\epsilon}_e}_{2}, \underbrace{\boldsymbol{\rho}_e}_{2}, \underbrace{\log\lVert\mathbf{P}^{e}_{\mathrm{dst}}\rVert}_{1}, \underbrace{\delta^{3\mathrm{D}}_e}_{3} \right] \tag*{(5)}

Beyond (usrce,udste)(\mathbf{u}^{e}_{\mathrm{src}},\mathbf{u}^{e}_{\mathrm{dst}}), the remaining entries are closed-form functions of the current state (full formulas in Sec. A.4). The epipolar vector ϵe\boldsymbol{\epsilon}_e is the point-to-epipolar-line offset under the directly predicted relative pose. The others are the 2D reprojection residual ρe\boldsymbol{\rho}_e, the 3D consistency residual δe3D\delta^{3\mathrm{D}}_e, and the log-magnitude log⁡∥Pdste∥\log\lVert\mathbf{P}^{e}_{\mathrm{dst}}\rVert of the destination point.

Iterative Refinement Network

State refinement block. At iteration k=0,…,K−1k = 0,\ldots,K-1, the state refinement block (Figure 2, left) runs three steps.

  • Step 1 (encode). The features fpair,k,fmatch,k\mathbf{f}^{\mathrm{pair},k},\mathbf{f}^{\mathrm{match},k} enter per-type hidden states through a linear map,

h∙,0=W∙f∙,k,∙∈{pair,match},(6)\mathbf{h}^{\bullet,0}=W^{\bullet}\mathbf{f}^{\bullet,k},\qquad\bullet\in\{\mathrm{pair},\mathrm{match}\}, \tag*{(6)}

while hcam,0\mathbf{h}^{\mathrm{cam},0} is set to the learned embedding ecam\mathbf{e}^{\mathrm{cam}}.

  • Step 2 (mix). A stack of L=4L=4 Match–Pair–Camera (MPC) blocks mixes the three tiers over the graph,

(hcam,hpair,hmatch)←MPC⁡ℓ(hcam,hpair,hmatch,G),(7)(\mathbf{h}^{\mathrm{cam}},\mathbf{h}^{\mathrm{pair}},\mathbf{h}^{\mathrm{match}}) \leftarrow\operatorname{MPC}_{\ell}(\mathbf{h}^{\mathrm{cam}},\mathbf{h}^{\mathrm{pair}},\mathbf{h}^{\mathrm{match}},\mathcal{G}), \tag*{(7)}

for ℓ=1,…,L\ell=1,\ldots,L. The LL blocks do not share weights.

  • Step 3 (decode). The prediction heads emit the state delta Δsk\Delta s_k.

MPC block composition. Each MPC block (Figure 2, right) updates the three hidden states in three steps.

  • Step 1 (camera→\topair/match broadcast). Injects endpoint camera states into per-pair and per-match channels:

hijpair+=W1pair(hicam∥hjcam),\mathbf{h}^{\mathrm{pair}}_{ij} \mathrel{+}= W^{\mathrm{pair}}_1(\mathbf{h}^{\mathrm{cam}}_i\mathbin\Vert\mathbf{h}^{\mathrm{cam}}_j),
heijmatch+=W1match(hicam∥hjcam),(8)\mathbf{h}^{\mathrm{match}}_{e_{ij}} \mathrel{+}= W^{\mathrm{match}}_1(\mathbf{h}^{\mathrm{cam}}_i\mathbin\Vert\mathbf{h}^{\mathrm{cam}}_j), \tag*{(8)}

where W∙W^{\bullet} represents the learned linear map R2dh→Rdh\mathbb{R}^{2d_h}\to\mathbb{R}^{d_h}.

  • Step 2 (match→\topair aggregation). Matches are conditioned on their parent pair and aggregated into it by attention over the pair’s matches Eij\mathcal{E}_{ij}:

hematch+=W2match(hematch∥hijpair),\mathbf{h}^{\mathrm{match}}_e \mathrel{+}= W^{\mathrm{match}}_2(\mathbf{h}^{\mathrm{match}}_e\mathbin\Vert\mathbf{h}^{\mathrm{pair}}_{ij}),
hijpair+=Attn⁡(hijpair,{hematch}e∈Eij),(9)\mathbf{h}^{\mathrm{pair}}_{ij} \mathrel{+}= \operatorname{Attn}(\mathbf{h}^{\mathrm{pair}}_{ij},\{\mathbf{h}^{\mathrm{match}}_e\}_{e\in\mathcal{E}_{ij}}), \tag*{(9)}

where Attn⁡(q,S)\operatorname{Attn}(\mathbf{q},\mathcal{S}) is 4-head dot-product attention [54] (query q\mathbf{q}, keys/values from S\mathcal{S}).

  • Step 3 (pair→\tocamera attention). Each camera attends over its incident directional pair states. The forward (i→j)(i\to j) state reaches camera jj, the inverse (j→i)(j\to i) state camera ii:

hjcam+=Attn⁡(hjcam,{hijpair}i∈N(j)),(10)\mathbf{h}^{\mathrm{cam}}_j \mathrel{+}= \operatorname{Attn}(\mathbf{h}^{\mathrm{cam}}_j,\{\mathbf{h}^{\mathrm{pair}}_{ij}\}_{i\in\mathcal{N}(j)}), \tag*{(10)}

with N(j)\mathcal{N}(j) the cameras paired with jj.

Prediction heads. Four heads read the hidden states of the final MPC block and emit the change of the state Δsk\Delta\mathbf{s}_k.

  • Absolute-pose head and log-FOV head. Two MLPs on hicam\mathbf{h}^{\mathrm{cam}}_i produce ΔTiabs∈R6\Delta T^{\mathrm{abs}}_i \in\mathbb{R}^{6} and Δϕi∈R\Delta\phi_i \in\mathbb{R}, respectively.

  • Relative-pose head. An MLP on hijpair\mathbf{h}^{\mathrm{pair}}_{ij} produces ΔTij∈R6\Delta T_{ij} \in\mathbb{R}^{6} in the decoupled [t^,ω][\hat{t}, \omega] parameterization. The change of relative pose is obtained by

ΔTijrel=12(ΔTij−ΔTji).(11)\Delta T^{\mathrm{rel}}_{ij} = \frac{1}{2}\left(\Delta T_{ij} - \Delta T_{ji}\right). \tag*{(11)}

Here ΔTji\Delta T_{ji} is the output of the same head on the reversed state hjipair\mathbf{h}^{\mathrm{pair}}_{ji} and predicts the change of Tjirel=(Tijrel)−1T^{\mathrm{rel}}_{ji} = \left(T^{\mathrm{rel}}_{ij}\right)^{-1}. We use −ΔTji-\Delta T_{ji} as a second prediction of the change of TijrelT^{\mathrm{rel}}_{ij}. (11) averages the two predictions, which makes the update antisymmetric by construction, ΔTjirel=−ΔTijrel\Delta T^{\mathrm{rel}}_{ji} = -\Delta T^{\mathrm{rel}}_{ij}.

  • Depth head. A small spatial decoder: each hmatch\mathbf{h}^{\mathrm{match}} is scattered onto its destination camera’s Hd×WdH_d \times W_d grid at the match location, pooled by a bilinear per-cell mean, and decoded into the log-depth delta ΔDi\Delta D_i (details in Sec. A.5).

Training Loss

Scale-invariant pose supervision. We define a scale-invariant pose distance that combines a chordal rotation distance with a translation-direction cosine,

d(Ta,Tb)=(1−cos⁡θR)⏟rotation cosine on SO(3)+(1−cos⁡θt)⏟translation cosine on S2,(12)d(T_a,T_b) = \underbrace{(1-\cos\theta_R)}_{\text{rotation cosine on }SO(3)} + \underbrace{(1-\cos\theta_t)}_{\text{translation cosine on }S^2}, \tag*{(12)}

where θR∈[0,π]\theta_R \in[0,\pi] is the geodesic angle between RaR_a and RbR_b on SO(3)SO(3), computed from the relative rotation RaRb⊤R_aR_b^\top via cos⁡θR=12(tr⁡(RaRb⊤)−1)\cos\theta_R = \frac{1}{2}\left(\operatorname{tr}(R_aR_b^\top)-1\right), and θt∈[0,π]\theta_t \in[0,\pi] is the angle between the unit-direction translations on S2S^2, cos⁡θt=⟨t^a,t^b⟩\cos\theta_t = \langle\hat{t}_a,\hat{t}_b\rangle with t^:=t/∥t∥\hat{t} := t/\lVert t\rVert. Because d(Ta,Tb)d(T_a,T_b) never consumes the magnitude of tt, no metric scale leaks into supervision. This enables us to mix datasets with arbitrary metric units without per-scene normalization. For every pair (i,j)(i,j), we define two different residuals,

rijrel=d(Tijrel,Tijgt),rijabs=d(Tijabs,sg⁡(Tijrel)),(13)r^{\mathrm{rel}}_{ij}=d\left(T^{\mathrm{rel}}_{ij},T^{\mathrm{gt}}_{ij}\right),\qquad r^{\mathrm{abs}}_{ij}=d\left(T^{\mathrm{abs}}_{ij},\operatorname{sg}\left(T^{\mathrm{rel}}_{ij}\right)\right), \tag*{(13)}

where sg⁡(⋅)\operatorname{sg}(\cdot) is the stop-gradient operator. rrelr^{\mathrm{rel}} supervises the relative-pose head against GT. The term rabsr^{\mathrm{abs}} couples the absolute-pose head to the (GT-supervised) relative-pose head, supervising the absolute poses through multi-view consensus, a learned analogue of pose-graph optimization. The stop-gradient makes the absolute poses follow the relative poses, never the reverse. The pose loss averages both over all pairs:

Lpose=1∣Epair∣∑(i,j)∈Epair(rijrel+rijabs).(14)\mathcal{L}_{\mathrm{pose}} = \frac{1}{|\mathcal{E}_{\mathrm{pair}}|} \sum_{(i,j)\in\mathcal{E}_{\mathrm{pair}}} \left(r^{\mathrm{rel}}_{ij}+r^{\mathrm{abs}}_{ij}\right). \tag*{(14)}

Geometric losses

We further enforce self-consistency between the model’s outputs through a geometric reprojection loss that requires no ground-truth scale. Writing smooth⁡L1β(⋅)\operatorname{smooth}^{\beta}_{L_1}(\cdot) for the smooth-L1L_1 kernel [12, 19] of width β\beta (Sec. D), the 2D reprojection and 3D consistency losses average the robustified residual norms over all matches e∈Ematche \in\mathcal{E}_{\mathrm{match}}:

L2d=1∣Ematch∣∑ewesmooth⁡L10.01(∥ρe∥),\mathcal{L}_{2d} = \frac{1}{|\mathcal{E}_{\mathrm{match}}|} \sum_e w_e\operatorname{smooth}^{0.01}_{L_1}\left(\lVert\rho_e\rVert\right),
L3d=1∣Ematch∣∑ewesmooth⁡L10.01(∥δe3D∥),(15)\mathcal{L}_{3d} = \frac{1}{|\mathcal{E}_{\mathrm{match}}|} \sum_e w_e\operatorname{smooth}^{0.01}_{L_1}\left(\lVert\delta^{3D}_e\rVert\right), \tag*{(15)}

where ρe\rho_e and δe3D\delta^{3D}_e are the per-match residuals of § 3.2.2 and we=γ2/(γ2+se2)∈(0,1]w_e=\gamma^2/(\gamma^2+s_e^2)\in(0,1] is a Cauchy weight on match ee, held constant during backpropagation, with ses_e the match’s Sampson distance under the GT epipolar geometry (kernel width γ=5×10−3\gamma=5\times10^{-3}). As a result, L2d/L3d\mathcal{L}_{2d}/\mathcal{L}_{3d} are effectively evaluated on inliers only, encouraging the network to fit the correct matches and to learn to ignore the inaccurate ones.

The FOV loss applies the same kernel to the log-FOV error against GT,

Lfov=1∣V∣∑i∈Vsmooth⁡L1(ϕi−ϕigt).(16)\mathcal{L}_{\mathrm{fov}} = \frac{1}{|\mathcal{V}|} \sum_{i\in\mathcal{V}} \operatorname{smooth}_{L_1}\left(\phi_i-\phi_i^{\mathrm{gt}}\right). \tag*{(16)}

Final loss. We run a Markov trajectory of KtrainK_{\mathrm{train}} iterations per training step and backpropagate through 24 of them (KgradK_{\mathrm{grad}}, Sec. D). In practice, even after the reconstruction has converged, the refiner keeps emitting updates that are pure global rotations and translations of the scene. We therefore add a fixed-point regularizer Lfpk=∥Δsk∥1\mathcal{L}^{k}_{\mathrm{fp}}=\lVert\Delta\mathbf{s}_k\rVert_1 that penalizes the magnitude of these unnecessary, gauge-equivalent updates, pressing late iterations toward true stationarity. The total loss sums over a view graph,

Lfinal=∑k∈Kgrad(Lposek+Lfovk⏟supervised+λ2dL2dk+λ3dL3dk⏟self-consistency+λfpLfpk⏟regularizer).(17)\mathcal{L}_{\mathrm{final}} = \sum_{k\in\mathcal{K}_{\mathrm{grad}}} \left( \underbrace{\mathcal{L}^{k}_{\mathrm{pose}}+\mathcal{L}^{k}_{\mathrm{fov}}}_{\text{supervised}} + \underbrace{\lambda_{2d}\mathcal{L}^{k}_{2d}+\lambda_{3d}\mathcal{L}^{k}_{3d}}_{\text{self-consistency}} + \underbrace{\lambda_{\mathrm{fp}}\mathcal{L}^{k}_{\mathrm{fp}}}_{\text{regularizer}} \right). \tag*{(17)}

Experiments

We evaluate DeepBA on multi-view camera pose across six benchmarks (§ 4.2) and on structure accuracy on DTU (§ 4.3), then ablate and analyze the upstream matcher, network capacity, outlier robustness, generalization beyond the training budget, and runtime scaling (§ 4.4). The appendix adds the accuracy-versus-iteration profile (Sec. E) and testing of camera localization on a frozen DeepBA map (Sec. G). In this paper, we treat all image collections as unordered and without known intrinsics unless explicitly noted.

Setup

Evaluation datasets. Following prior work [24, 56, 57, 58], we evaluate on six benchmarks that span diverse imaging conditions: CO3Dv2 [38], RealEstate10K [63], ETH3D [41], DTU [21], Mip-NeRF 360 [1], and IMC2021 [22]. Per-dataset scene selection and frame sampling are detailed in Sec. C. Five benchmarks (RealEstate10K, ETH3D, DTU, Mip-NeRF 360, and IMC2021) are absent from our training mixture, so they measure cross-dataset generalization; CO3Dv2 is in the mixture but is evaluated only on held-out scenes disjoint from the training set.

Training data. DeepBA is trained on a heterogeneous mixture of 3D datasets: MegaDepth [26], CO3Dv2 [38], ScanNet [5], BlendedMVS [61], DL3DV [28], and Calle [10, 36], spanning indoor/outdoor, object- and scene-centric captures. Their arbitrary and incompatible metric units mix safely under DeepBA’s scale-invariant supervision (§ 3.4), without per-scene normalization.

Baselines. We compare against three families: classical SfM (COLMAP [40], GLOMAP [37]); learned image-to-geometry (PoseDiffusion [55], DUSt3R [58], MASt3R [24], MASt3R-SfM [7], VGGSfM [56], VGGT [57]); and learning-based SfM from matches (GASfM [2], RESfM [23]).

Metrics. For pose estimation, following [24, 57], we report Relative Rotation/Translation Accuracy (RRA/RTA) and AUC@30°, the area under the joint accuracy curve up to 30°. For structure quality, we report Accuracy and Completeness against DTU’s structured-light reference scans at 1/2/5 cm. Both follow the standard definitions, given in Sec. B.

Implementation. DeepBA uses L=4L=4 MPC blocks (2.04M parameters), trained for 50,000 steps with AdamW [30] (peak learning rate 6×10−46 \times10^{-4}) and loss weights λ2d=1\lambda_{2d}=1, λ3d=λfp=0.1\lambda_{3d}=\lambda_{fp}=0.1, on 4 NVIDIA L40S GPUs. Each step unrolls the refiner for all Ktrain=100K_{\mathrm{train}}=100 iterations but backpropagates only through a 24-iteration subset. The scene state is initialized input-independently (identity poses, zero log-FOV and log-depth) and perturbed with Gaussian noise during training. Unless stated otherwise (§ 4.4), we always evaluate on eLoFTR [59] (training mixes eLoFTR and SIFT), per-pair RANSAC-verified [11], with Ktest=100K_{\mathrm{test}}=100 on every benchmark. Further training details are in Sec. D. Code and trained weights will be released upon publication.

Camera Pose Estimation

We evaluate camera pose on CO3Dv2 and RealEstate10K (Table 1) following the 10-frame protocol in VGGT [57].

MethodCO3Dv2 RRA@15°CO3Dv2 RTA@15°CO3Dv2 AUC@30°RealEstate10K AUC@30°ParamsTime
Classical SfM
COLMAP (SIFT) [40]86.379.974.946.5–∼\sim2+3 s
GLOMAP (SIFT) [37]86.676.271.171.7–∼\sim2+2 s
COLMAP (eLoFTR) [40]89.783.577.080.9–∼\sim7+11 s
GLOMAP (eLoFTR) [37]82.172.367.672.7–∼\sim7+5 s
Learned image-to-geometry
PoseDiffusion [55]87.780.071.639.639.0M∼\sim11 s
DUSt3R [58]98.382.172.979.9577.8M∼\sim7 s
MASt3R [24]99.292.184.791.1695.3M∼\sim9 s
VGGSfM [56]99.485.885.089.5281.2M∼\sim10 s
VGGT [57]99.392.185.185.3∼\sim1.2B∼\sim0.2 s
Learning-based SfM from matches
RESfM (SIFT) [23]62.755.551.148.90.7M∼\sim2+13 s
GASfM (SIFT) [2]79.173.968.675.4145.2M∼\sim0.5+5 s
RESfM (eLoFTR) [23]61.154.048.263.70.7M∼\sim7+13 s
GASfM (eLoFTR) [2]79.077.371.282.8145.2M∼\sim7+2 s
DeepBA97.489.278.684.02.04M∼\sim7+3 s

Table 1. Camera pose estimation on CO3Dv2 and RealEstate10K (10 frames per scene). Bold/underline: best/second per column within each group. †: ground-truth intrinsics plus a final classical bundle adjustment. Params: from each checkpoint. Time: seconds per scene on one H100, as matching + solver. ‡: same cached eLoFTR correspondences as DeepBA, given to COLMAP/GLOMAP/GASfM/RESfM as multi-view tracks (Sec. C).

CO3Dv2 covers object-centric, wide-baseline pairs, while RealEstate10K covers indoor and outdoor video motion. We additionally evaluate pose on four further multi-view benchmarks, DTU, Mip-NeRF 360, ETH3D, and IMC2021 (Table 2), each scored at the tighter @3°/@5°/@10° thresholds (per-dataset composition in Sec. C).

MethodDTU [21] (49 img/scene)Mip-NeRF 360 [1] (125–311 img/scene)ETH3D [41] (14–76 img/scene)IMC2021 [22] (5/10/25-view bags)
RRA↑ @3°RRA↑ @5°RRA↑ @10°RTA↑ @3°RTA↑ @5°RTA↑ @10°RRA↑ @3°RRA↑ @5°RRA↑ @10°RTA↑ @3°RTA↑ @5°RTA↑ @10°RRA↑ @3°RRA↑ @5°RRA↑ @10°RTA↑ @3°RTA↑ @5°RTA↑ @10°RRA↑ @3°RRA↑ @5°RRA↑ @10°RTA↑ @3°RTA↑ @5°RTA↑ @10°
Classical SfM
COLMAP (SIFT)[40]87.387.387.365.674.682.6100.0100.0100.099.9100.0100.096.796.796.794.995.896.494.094.995.774.782.589.3
GLOMAP (SIFT)[37]85.986.887.268.773.478.6100.0100.0100.099.9100.0100.094.494.596.686.389.391.592.293.996.073.981.488.2
COLMAP (eLoFTR)‡[40]100.0100.0100.091.897.199.5100.0100.0100.099.699.8100.076.576.676.969.172.875.595.896.797.275.484.391.5
GLOMAP (eLoFTR)‡[37]97.197.197.989.794.896.8100.0100.0100.099.799.9100.058.459.761.650.253.556.995.896.797.874.983.691.2
Learned image-to-geometry
MASt3R-SfM [7]99.6100.0100.092.097.799.568.579.986.960.275.284.067.874.079.063.868.874.694.096.297.562.476.387.7
VGGT [57]100.0100.0100.099.399.9100.097.6100.0100.087.195.398.584.193.496.951.465.079.699.8100.0100.071.182.491.9
Learning-based SfM from matches
RESfM (SIFT)†[23]22.022.022.022.323.326.011.611.812.011.511.712.13.13.54.53.03.23.881.782.684.676.779.081.2
GASfM (SIFT)†[2]38.339.940.839.140.544.132.435.241.131.735.444.38.89.512.18.39.010.774.780.486.356.163.975.3
RESfM (eLoFTR)†,‡[23]36.636.736.836.837.239.310.711.111.310.510.812.019.119.319.618.318.819.481.082.284.875.577.979.8
GASfM (eLoFTR)†,‡[2]36.937.638.837.438.642.449.651.954.750.853.557.621.623.927.221.023.929.489.691.092.283.187.390.8
DeepBA73.695.6100.069.590.899.653.366.773.558.668.286.533.853.469.118.428.849.186.995.699.049.468.987.3

Table 2. Multi-view pose accuracy (RRA/RTA) on DTU, Mip-NeRF 360, ETH3D, and IMC2021. Bold/underline: best/second per column within each group. Grey: image-to-geometry methods, for reference. †, ‡: as in Tab. 1.

CO3Dv2 and RealEstate10K. On both benchmarks (Table 1), DeepBA leads the learned methods from matches. On RealEstate10K, GASfM with the same eLoFTR correspondences comes closest (AUC@30° 82.8 vs. 84.0), while GASfM (SIFT) scores 75.4. Against classical SfM, DeepBA outperforms COLMAP and GLOMAP on every metric of both benchmarks, with both SIFT and the shared eLoFTR input.

DTU, Mip-NeRF 360, ETH3D and IMC2021. On these four benchmarks (Table 2), DeepBA leads the learned methods from matches on most metrics. GASfM and RESfM, given ground-truth intrinsics, are ahead only in a few cells, mostly on IMC2021 translation. Against classical SfM on the shared eLoFTR input, DeepBA comes close to both mappers at 10° on DTU and IMC2021 but falls behind at the tighter thresholds. On Mip-NeRF 360 the mappers are near the ceiling and DeepBA trails them (RRA@10° 73.5, RTA@10° 86.5). On ETH3D, COLMAP is well ahead (RTA@10° 75.5 against 49.1, see Sec. F.1). We also provide a qualitative comparison in Figure 3 and a failure analysis in Sec. F.

Qualitative comparison of camera pose estimation across datasets

Figure 3. Qualitative comparison (rows: scan82, bicycle, delivery_area, and london_bridge; the number after each dataset is the number of input images); 3D points, estimated cameras (green), GT cameras (grey).

Camera localization. DeepBA’s robustness also enables camera localization (Sec. G). Reusing the same model with the reference cameras held fixed, DeepBA registers query views against a frozen map from the mean pose of the map cameras, with no PnP or external solver.

Structure Accuracy

Beyond poses, Table 3 evaluates the recovered structure on DTU against the official structured-light reference scans. Among the learning-based SfM methods from matches, DeepBA is the strongest in every column and the only one to produce a non-empty object reconstruction on all 15 scans—even though GASfM and RESfM use ground-truth intrinsics. The learned image-to-geometry methods are near the ceiling on this object-centric benchmark.

MethodAccuracy (%)† @1 cmAccuracy (%)† @2 cmAccuracy (%)† @5 cmCompleteness (%)† @1 cmCompleteness (%)† @2 cmCompleteness (%)† @5 cmSucc.
Classical SfM
COLMAP (SIFT) [40]94.297.699.480.695.299.615/15
GLOMAP (SIFT) [37]43.049.365.030.140.751.611/15
COLMAP (eLoFTR)‡ [40]91.798.299.896.699.7100.015/15
GLOMAP (eLoFTR)‡ [37]64.671.373.168.172.573.312/15
Learned image-to-geometry
MASt3R-SfM [7]88.997.499.685.594.198.415/15
VGGT [57]95.498.099.297.298.899.915/15
Learning-based SfM from matches
RESfM (SIFT)†[23]9.321.738.00.93.917.410/15
GASfM (SIFT)†[2]33.039.346.425.435.347.313/15
RESfM (eLoFTR)†,‡[23]20.628.152.519.527.546.511/15
GASfM (eLoFTR)†,‡[2]30.934.542.535.641.847.114/15
DeepBA34.162.398.537.159.596.115/15

Table 3. Structure accuracy and completeness on DTU (15 test scans). Succ.: scans with a non-empty object reconstruction. Bold/underline: best/second on accuracy and completeness metrics within each group. †, ‡: as in Tab. 1. Grey: image-to-geometry methods, for reference.

Ablations and Analysis

Configuration ablations. Table 4 (left) ablates the configuration on CO3Dv2. Hidden width dhd_h matters most: halving it to dh=64d_h=64 drops AUC@30° from 78.6 to 58.9, so we keep dh=128d_h=128. Weakening the 3D consistency loss to λ3d=0.01\lambda_{3d}=0.01 is also costly (78.6 → 64.2), indicating the 3D term contributes beyond the pose losses. Enabling the per-iteration gauge strip leaves accuracy almost unchanged, so the gauge freedom needs no explicit fixing as expected. The last two rows change only the evaluation. Ground-truth focal lengths leave RRA almost unchanged but raise RTA, as they remove the focal-depth ambiguity (Sec. F.2). A classical BA, started from DeepBA’s output, raises all three metrics. This gain shows that DeepBA behaves similarly to BA but not identi-

RRA @15°↑RTA @15°↑AUC @30°↑
Default97.489.278.6
dhd_h: 128 → 6479.272.658.9
λ3d\lambda_{3d}: 0.1 → 0.0181.776.664.2
gauge strip: off → on97.188.878.3
GT focal length97.590.981.5
post classical BA98.091.084.6

Table 4. Ablations (left) and outlier robustness (right) on CO3Dv2 (protocol of Tab. 1). Left: each row changes one factor vs. the default, green/red marking gain/loss; the gauge strip removes the global rigid motion and rescales each iteration. Right: a fraction of matches is slid along the GT epipolar line (Sampson≈0).

Corrupted fractionRRA @15°↑RTA @15°↑AUC @30°↑
Default97.489.278.6
5%95.586.874.5
10%95.786.473.4
50%86.877.562.4
90%65.258.642.7
100%63.456.340.7

::: cally, which leaves room for improvement.

Outlier robustness. A RANSAC-style geometric filter cannot remove wrong matches that lie on the true epipolar line (Sampson ≈0\approx0, e.g., from repetitive structure). We probe this regime on CO3Dv2 scenes (protocol of Table 1), sampling a growing fraction of the matches uniformly across all pairs of a scene and sliding the destination endpoint of each by 0–50% of the image width along its ground-truth epipolar line. Table 4 (right) shows slow degradation: against the clean run on the same CO3Dv2 scenes (78.6), 10% corruption costs 5.2 AUC@30° points and 50% costs 16.2. Even at 100%, accuracy stays at 40.7 rather than collapsing, because a match slid along its own epipolar line still constrains that pair’s rotation and translation direction.

Upstream matcher. DeepBA consumes only 2D matches, so the same trained network can run on any front-end without retraining. We evaluate it with five front-ends—sparse SIFT [32], the sparse learned SuperPoint+LightGlue (SP+LG) [6, 27] and ALIKED+LightGlue (ALIKED+LG) [27, 62], semi-dense eLoFTR [59], and dense DKM [8]—on the four multi-view benchmarks (Table 5). The four learned front-ends do not rank by match density, so density alone does not drive accuracy. SIFT gives the lowest accuracy on most of the metrics.

MatcherDTU [21] (49 img/scene)Mip-NeRF 360 [1] (125–311 img/scene)ETH3D [41] (14–76 img/scene)IMC2021 [22] (5/10/25-view bags)
RRA↑ @3°RRA↑ @5°RRA↑ @10°RTA↑ @3°RTA↑ @5°RTA↑ @10°RRA↑ @3°RRA↑ @5°RRA↑ @10°RTA↑ @3°RTA↑ @5°RTA↑ @10°RRA↑ @3°RRA↑ @5°RRA↑ @10°RTA↑ @3°RTA↑ @5°RTA↑ @10°RRA↑ @3°RRA↑ @5°RRA↑ @10°RTA↑ @3°RTA↑ @5°RTA↑ @10°
SIFT [32]32.241.350.725.239.554.114.221.429.410.018.729.831.545.261.314.525.844.457.067.673.321.934.352.1
SP+LG [6, 27]69.593.4100.060.387.199.564.072.282.064.377.791.735.255.772.521.535.957.687.196.199.146.766.786.2
ALIKED+LG [27, 62]60.684.193.354.877.792.979.485.086.779.484.788.828.347.169.517.931.857.187.596.299.247.968.086.8
eLoFTR [59]73.695.6100.069.590.899.653.366.773.558.668.286.533.853.469.118.428.849.186.995.699.049.468.987.3
DKM [8]65.592.999.774.695.799.761.368.974.563.071.390.036.152.466.126.341.459.586.695.298.051.270.387.4

Table 5. DeepBA generalizes across upstream matchers (§ 4.4). Bold/underline and protocols as in Table 2.

Generalization beyond the training budget. We train DeepBA with Ktrain=100K_{\mathrm{train}}=100 refinement iterations on view graphs of at most 48 cameras, yet at test time it extends beyond that: to more iterations and to larger scenes. On iterations, Figure 4 runs K=1000K=1000 (10× the training budget) across checkpoints: early extrapolation is metastable and can degrade the estimate, but stabilizes by ∼\sim40k steps. On size, DeepBA handles Mip-NeRF 360 scenes of up to ∼\sim311 cameras (∼\sim6.5\times its largest training graph).

Figure 4. Generalizing beyond the training iteration budget (RRA@10° left, RTA@10° right, on bicycle). Each training checkpoint is evaluated at K=100K=100 (blue) and K=1000K=1000 (red).

Runtime scaling. Figure 5 times DeepBA, COLMAP and GLOMAP on the same eLoFTR matches of the first NN images of bonsai, so the methods differ only in the solver (matching: 279 s at N=292N=292). From N=10N=10 on, DeepBA is the fastest of the three. At N=292N=292 (3.4M matches), its K=100K=100 iterations take 56 s with a GPU memory peak of 24 GB, against 988 s (0.9 GB) for COLMAP and 793 s (0.5 GB) for GLOMAP with their GPU solvers. DeepBA’s per-iteration cost is linear in the number of matches, 25 ms + 155 ns per match.

Figure 5. Left: time after matching on the first NN images of Mip-NeRF 360 bonsai, with every method using the same eLoFTR matches on one H100 (matching time dashed). Right: time per iteration vs. number of matches.

Conclusion

We presented DeepBA, a lightweight graph neural network that amortizes bundle adjustment. From the 2D matches of any off-the-shelf matcher and a trivial cold start, it jointly infers camera poses, intrinsics, and scene structure for arbitrarily many views. Across six benchmarks, DeepBA outperforms prior learned SfM from matches on most metrics with ∼\sim71× fewer parameters than G&SM, the strongest of them, and approaches image-to-geometry transformers that consume the images. Four limits remain: the focal–depth ambiguity on planar scenes, a drift of the learned update on some scenes, the accuracy at tight thresholds, and the pinhole camera model. We hope DeepBA encourages further work on learned solvers for multi-view geometry.

Method Details

Lie-Group and Pose Conventions

We use Log⁡:SE⁡(3)→R6\operatorname{Log}:\operatorname{SE}(3)\to\mathbb{R}^{6} and Exp⁡:R6→SE⁡(3)\operatorname{Exp}:\mathbb{R}^{6}\to\operatorname{SE}(3) for the standard Lie-group logarithm and exponential between a 4×44\times4 rigid-body pose matrix and its 66-dim tangent vector. We overload Log⁡\operatorname{Log} for SO⁡(3)\operatorname{SO}(3): Log⁡:SO⁡(3)→R3→SO⁡(3)\operatorname{Log}:\operatorname{SO}(3)\to\mathbb{R}^{3}\to\operatorname{SO}(3). Ti=TiabsT_i=T_i^{\mathrm{abs}} maps world coordinates to camera-ii coordinates, xi=Rixw+ti\mathbf{x}_i=R_i\mathbf{x}_w+\mathbf{t}_i; hence Tijabs=Tiabs(Tjabs)−1T_{ij}^{\mathrm{abs}}=T_i^{\mathrm{abs}}(T_j^{\mathrm{abs}})^{-1} maps camera-jj to camera-ii coordinates.

Component-wise state update (⊞\boxplus, Eq. (3)). The absolute pose composes on the group, Tiabs←Exp⁡(ΔTiabs)TiabsT_i^{\mathrm{abs}}\leftarrow\operatorname{Exp}(\Delta T_i^{\mathrm{abs}})T_i^{\mathrm{abs}}, and the log-FOV and log-depth maps update additively, ϕi←ϕi+Δϕi\phi_i\leftarrow\phi_i+\Delta\phi_i, Di←Di+ΔDiD_i\leftarrow D_i+\Delta D_i. The decoupled per-edge relative pose (t^ijrel∈S2,ωijrel∈R3)(\hat{\mathbf{t}}_{ij}^{\mathrm{rel}}\in S^2,\boldsymbol{\omega}_{ij}^{\mathrm{rel}}\in\mathbb{R}^{3}) of Sec. 3.1 is updated component-wise. Writing ΔTijrel=(Δtij,Δωij)\Delta T_{ij}^{\mathrm{rel}}=(\Delta\mathbf{t}_{ij},\Delta\boldsymbol{\omega}_{ij}), the rotation is composed on the group and the translation direction is added and renormalized onto S2S^2:

ωijrel←Log⁡(R(Δωij)R(ωijrel)),t^ijrel←t^ijrel+Δtij∥t^ijrel+Δtij∥(18)\boldsymbol{\omega}_{ij}^{\mathrm{rel}}\leftarrow\operatorname{Log}\left(R(\Delta\boldsymbol{\omega}_{ij})R(\boldsymbol{\omega}_{ij}^{\mathrm{rel}})\right),\qquad \hat{\mathbf{t}}_{ij}^{\mathrm{rel}}\leftarrow\frac{\hat{\mathbf{t}}_{ij}^{\mathrm{rel}}+\Delta\mathbf{t}_{ij}}{\left\|\hat{\mathbf{t}}_{ij}^{\mathrm{rel}}+\Delta\mathbf{t}_{ij}\right\|} \tag*{(18)}

Focal Length FOV

Writing FOVi\mathrm{FOV}_i for camera ii’s field-of-view angle in radians, the log-FOV is ϕi:=log⁡FOVi\phi_i:=\log\mathrm{FOV}_i; the simple-pinhole intrinsic is

Ki=K(ϕi)=(fi00.50aifi0.5001),fi=12tan⁡(exp⁡(ϕi)/2)(19)K_i=K(\phi_i)= \begin{pmatrix} f_i & 0 & 0.5\\ 0 & a_i f_i & 0.5\\ 0 & 0 & 1 \end{pmatrix}, \qquad f_i=\frac{1}{2\tan(\exp(\phi_i)/2)} \tag*{(19)}

with image coordinates normalized to [0,1]2[0,1]^2 per axis and ai=Wi/Hia_i=W_i/H_i the aspect ratio of image ii, so FOVi\mathrm{FOV}_i is the horizontal field of view and pixels stay square.

Pose-Graph Residual

The per-edge pose-graph residual (Sec. 3.2.1) measures the disagreement between the directly predicted relative pose TjirelT_{ji}^{\mathrm{rel}} and the one derived from the absolute poses, TjiabsT_{ji}^{\mathrm{abs}}. The rotation term δjiR\delta_{ji}^{R} is that disagreement in the tangent space,

δjiR=Log⁡(Rjirel(Rjiabs)T).(20)\delta_{ji}^{R}=\operatorname{Log}\left(R_{ji}^{\mathrm{rel}}(R_{ji}^{\mathrm{abs}})^{\mathsf{T}}\right). \tag*{(20)}

The translation term compares directions only. Write t^jiabs\hat{\mathbf{t}}_{ji}^{\mathrm{abs}} for the unit-normalized translation of TjiabsT_{ji}^{\mathrm{abs}}. Then δjit\delta_{ji}^{t} is the minimal rotation carrying t^jiabs\hat{\mathbf{t}}_{ji}^{\mathrm{abs}} onto t^jirel\hat{\mathbf{t}}_{ji}^{\mathrm{rel}}, written as an axis–angle 33-vector

δjit=θn^,θ=∠(t^jirel,t^jiabs),n^∝t^jiabs×t^jirel.(21)\delta_{ji}^{t}=\theta\hat{\mathbf{n}},\qquad \theta=\angle(\hat{\mathbf{t}}_{ji}^{\mathrm{rel}},\hat{\mathbf{t}}_{ji}^{\mathrm{abs}}),\qquad \hat{\mathbf{n}}\propto\hat{\mathbf{t}}_{ji}^{\mathrm{abs}}\times\hat{\mathbf{t}}_{ji}^{\mathrm{rel}}. \tag*{(21)}

Match-Feature Residuals

Throughout this subsection ee is a match on the pair (i,j)∈Epair(i,j)\in\mathcal{E}_{\mathrm{pair}}, with endpoints uie\mathbf{u}_i^e in image ii and uje\mathbf{u}_j^e in image jj, written usrce\mathbf{u}_{\mathrm{src}}^e and udste\mathbf{u}_{\mathrm{dst}}^e in Sec. 3.2.2. Both are in the normalized image coordinates of Eq. (19), and u~=(uT,1)T\tilde{\mathbf{u}}=(\mathbf{u}^{\mathsf{T}},1)^{\mathsf{T}} denotes the homogeneous lift of a 2D point.

The four match-feature quantities of Sec. 3.2.2 are all closed form. The epipolar vector is computed from the directly predicted relative pose. Its translation magnitude is irrelevant here, since the epipolar geometry does not depend on it and t^jirel\hat{\mathbf{t}}_{ji}^{\mathrm{rel}} is already a unit direction. The essential matrix Eji=[t^jirel]×RjirelE_{ji}=[\hat{\mathbf{t}}_{ji}^{\mathrm{rel}}]_{\times}R_{ji}^{\mathrm{rel}}, with Rjirel:=R(ωjirel)R_{ji}^{\mathrm{rel}}:=R(\boldsymbol{\omega}_{ji}^{\mathrm{rel}}), defines the epipolar line for uie\mathbf{u}_i^e in image jj (in normalized camera coordinates) as lji=EjiKi−1u~ie\mathbf{l}_{ji}=E_{ji}K_i^{-1}\tilde{\mathbf{u}}_i^e. Because the simple pinhole model has isotropic focal length fjf_j, mapping the point-to-line offset back to image-plane coordinates collapses to a single scalar, and the shortest image-plane vector from uje\mathbf{u}_j^e to the line is

ϵe=−fjlji,1:2T(Kj−1u~je)∥lji,1:2∥2.(22)\epsilon_e=-f_j\frac{\mathbf{l}_{ji,1:2}^{\mathsf{T}}(K_j^{-1}\tilde{\mathbf{u}}_j^e)}{\left\|\mathbf{l}_{ji,1:2}\right\|^2}. \tag*{(22)}

where lji,1:2\mathbf{l}_{ji,1:2} is the first two components of lji\mathbf{l}_{ji}.

The remaining reprojection and 3D consistency residuals use the absolute-pose-derived relative pose TjiabsT_{ji}^{\mathrm{abs}}, whose rotation and translation are RjiabsR_{ji}^{\mathrm{abs}} and tjiabs\mathbf{t}_{ji}^{\mathrm{abs}} in the sense of Sec. A.1, so that xj=Rjiabsxi+tjiabs\mathbf{x}_j=R_{ji}^{\mathrm{abs}}\mathbf{x}_i+\mathbf{t}_{ji}^{\mathrm{abs}}. Per-pixel depths are sampled from the current log-depth maps,

die=exp⁡(Di[uie]),dje=exp⁡(Dj[uje]),(23)d_i^e=\exp(D_i[\mathbf{u}_i^e]),\qquad d_j^e=\exp(D_j[\mathbf{u}_j^e]), \tag*{(23)}

with Di[u]D_i[\mathbf{u}] bilinear sampling of the log-depth map at image coordinate u\mathbf{u}; unprojection gives the per-camera 3D points

Pie=dieKi−1u~ie,Pje=djeKj−1u~je;(24)\mathbf{P}_i^e=d_i^eK_i^{-1}\tilde{\mathbf{u}}_i^e,\qquad \mathbf{P}_j^e=d_j^eK_j^{-1}\tilde{\mathbf{u}}_j^e; \tag*{(24)}

and the residuals are

ρe=π(Kj(RjiabsPie+tjiabs))−uje,\rho_e=\pi\left(K_j\left(R_{ji}^{\mathrm{abs}}\mathbf{P}_i^e+\mathbf{t}_{ji}^{\mathrm{abs}}\right)\right)-\mathbf{u}_j^e,
δe3D=Pje−(RjiabsPie+tjiabs).(25)\delta_e^{3\mathrm{D}}=\mathbf{P}_j^e-\left(R_{ji}^{\mathrm{abs}}\mathbf{P}_i^e+\mathbf{t}_{ji}^{\mathrm{abs}}\right). \tag*{(25)}

where π([x,y,z]T)=[x/z,y/z]T\pi([x,y,z]^{\mathsf{T}})=[x/z,y/z]^{\mathsf{T}} is the perspective division.

Depth Head

For each match ee on pair (i,j)(i,j), the depth head scatters heijmatch\mathbf{h}_{e_{ij}}^{\mathrm{match}} onto camera jj’s Hd×WdH_d\times W_d grid at udste\mathbf{u}_{\mathrm{dst}}^e (and hejimatch\mathbf{h}_{e_{ji}}^{\mathrm{match}} onto camera ii’s grid at usrce\mathbf{u}_{\mathrm{src}}^e) using the adjoint of grid_sample. The scatter also accumulates a per-cell count cc, the bilinear weight of the matches landing in that cell, which divides the summed features into a per-cell mean. That mean, channel-wise layer-normalized and concatenated with log⁡(1+c)\log(1+c), is decoded by three 3×33\times3 convolutions into the log-depth delta ΔDi\Delta D_i.

Metric Definitions

Pose Accuracy

All pose metrics are computed over the complete pair set of each scene. Let V\mathcal{V} denote the NN evaluation cameras of a scene and P={(i,j):1≤i<j≤N}\mathcal{P}=\{(i,j):1\le i<j\le N\} its (N2)\binom{N}{2} unordered pairs. For a pair (i,j)(i,j), eRije_{R}^{ij} is the geodesic angle between the predicted and ground-truth relative rotations and etije_{t}^{ij} the angle between the predicted and ground-truth relative translation directions. Both are measured in degrees, as is every threshold τ\tau in this subsection. A method that produces no pose for camera ii (unregistered, dropped, or non-finite) is assigned eRij=etij=∞e_{R}^{ij}=e_{t}^{ij}=\infty for every pair containing ii (zero credit). With I[⋅]\mathbb{I}[\cdot] the indicator function (1 if the bracketed condition holds, else 0):

RRA@τ=100∣P∣∑(i,j)∈PI[eRij<τ],RTA@τ=100∣P∣∑(i,j)∈PI[etij<τ],(26)\begin{aligned} \mathrm{RRA}@\tau&= \frac{100}{|\mathcal{P}|}\sum_{(i,j)\in\mathcal{P}}\mathbb{I}[e_{R}^{ij}<\tau],\\ \mathrm{RTA}@\tau&= \frac{100}{|\mathcal{P}|}\sum_{(i,j)\in\mathcal{P}}\mathbb{I}[e_{t}^{ij}<\tau], \tag*{(26)} \end{aligned}
AUC@30∘=130∫030A(τ) dτ,A(τ)=100∣P∣∣{(i,j)∈P:max⁡(eRij,etij)<τ}∣.(27)\begin{aligned} \mathrm{AUC}@30^\circ&= \frac{1}{30}\int_{0}^{30} A(\tau)\,\mathrm{d}\tau,\\ A(\tau) &= \frac{100}{|\mathcal{P}|}\left|\{(i,j)\in\mathcal{P}:\max(e_{R}^{ij},e_{t}^{ij})<\tau\}\right|. \tag*{(27)} \end{aligned}

Structure Accuracy

Protocol for Table 3. Each reconstruction is aligned to the DTU world frame (millimeters) by a single similarity transform from its predicted camera centers. Both prediction and reference are restricted to the object region by DTU’s per-view masks. Writing d(x,S)=min⁡y∈S∥x−y∥d(x,\mathcal{S})=\min_{y\in\mathcal{S}}\lVert x-y\rVert for the nearest-neighbor distance and X,Y\mathcal{X},\mathcal{Y} for the predicted and reference point sets,

Accuracy@τ=100∣X∣∑x∈XI[d(x,Y)<τ],Completeness@τ=100∣Y∣∑y∈YI[d(y,X)<τ].(28)\begin{aligned} \mathrm{Accuracy}@\tau&= \frac{100}{|\mathcal{X}|}\sum_{x\in\mathcal{X}}\mathbb{I}[d(x,\mathcal{Y})<\tau],\\ \mathrm{Completeness}@\tau&= \frac{100}{|\mathcal{Y}|}\sum_{y\in\mathcal{Y}}\mathbb{I}[d(y,\mathcal{X})<\tau]. \tag*{(28)} \end{aligned}

Benchmark Composition

We detail the composition of the six evaluation benchmarks introduced in § 4.

  • CO3Dv2 [38]: object-centric turntable-style videos with wide-baseline pairs, evaluated on 202 held-out scenes across 41 categories, disjoint from our training split. Following the 10-frame protocol of VGGT [57], we sample 10 frames per scene from a fixed per-scene frame list shared across all methods, and report relative pose accuracy.

  • RealEstate10K [63]: indoor and outdoor video clips with smooth camera motion. We follow the same 10-frame VGGT protocol [57] over 108 clips, drawing the frames from fixed per-clip lists shared across all methods.

  • ETH3D [41]: the 13 high-resolution scenes of the multi-view training split, with laser-scanned ground truth. No ETH3D image enters our training mixture.

  • DTU [21]: 15 of the 128 laboratory-captured object scans (49 views each) with structured-light ground truth, the subset used by SPARF [53].

  • Mip-NeRF 360 [1]: 7 publicly released indoor and outdoor scenes (125–311 cameras each) captured in 360° sweeps around a central area.

  • IMC2021 [22]: all 9 in-the-wild phototourism scenes of the 2021 Image Matching Challenge, evaluated under the official bag protocol of 175 bags per scene (100 of 5 views, 50 of 10 and 25 of 25, 1,575 bags in total).

Baselines on eLoFTR. The eLoFTR rows of COLMAP/GLOMAP/ResFM/GAsfM receive DeepBA’s cached, RANSAC-verified eLoFTR correspondences. COLMAP/GLOMAP/ResFM/GAsfM need keypoints shared across pairs. Each image’s match endpoints are therefore snapped to square cells, with one keypoint per cell at the mean of its endpoints. Each mapper then runs its own geometric verification and track building. Union-find on a grid cell of 1/400 of the image width (4 px at 1,600 px) chains the matches into tracks, which are kept when seen in at least 3 views.

Training and Implementation Details

These expand the headline configuration of § 4.1.

Gradient-iteration sampling. Each step runs a Ktrain=100K_{\mathrm{train}}=100-iteration Markov trajectory but takes gradients on only the first 12 iterations plus another 12 iterations sampled uniformly from [12,100)[12,100) (24 iterations in total with backpropagation of Eq. (17); the remaining 76 run under no_grad). We denote this set of 24 iterations by KgradK_{\mathrm{grad}}. We backpropagate per refinement iteration and detach the state at each gradient boundary, giving memory O(1)O(1) in the number of refinement iterations.

Scene and graph sampling. At each training step, we sample a connected set of cameras from a randomly chosen scene, primarily with a Sampson-gated graph walk over wide-baseline pairs. We add random edges of the view graph so that every camera shares a view with at least one other. Robust kernel. The smooth-L1L_1 (Huber) kernel of § 3.4 is

smooth⁡L1β(r)={12r2/β,∣r∣<β,∣r∣−12β,otherwise.(29)\operatorname{smooth}^{\beta}_{L_1}(r)= \begin{cases} \frac{1}{2}r^2/\beta, & |r|<\beta,\\ |r|-\frac{1}{2}\beta, & \text{otherwise}. \end{cases} \tag*{(29)}

Convergence against the Refinement Iteration

Figure 6 profiles accuracy against the number of refinement iterations. We measure it on Mip-NeRF 360, which is absent from the training mixture. Its scenes carry 125–311 views, 2.6× to 6.5× the largest (48-camera) view graphs DeepBA trains on. From a cold start, accuracy climbs steeply over the first tens of iterations and then saturates, at a scene-dependent rate, with garden and stump settling within ∼30 iterations and the remaining scenes flattening out well before the end of the profile. We train DeepBA with Ktrain=100K_{\mathrm{train}}=100. The mean RRA@10° keeps rising until ∼150 iterations and is flat beyond. Extending inference to K=1000K=1000 raises it from 73.5 to 82.2, mainly through counter, but lowers the mean RTA@10° from 86.5 to 84.5, mainly through bonsai. With no consistent gain from more iterations, we evaluate every benchmark at Ktest=100K_{\mathrm{test}}=100.

Pose accuracy versus refinement iteration

Figure 6. Pose accuracy vs. refinement iteration (RRA@10° left, RTA@10° right). Light curves are per-scene; the solid black line is the across-scene mean, the dotted line the mean excluding the view-graph-bottlenecked room scene.

Failure Analysis

ETH3D

ETH3D is DeepBA’s weakest multi-view benchmark (Table 2). Three causes explain most of the losses (Figure 7, Figure 8).

Causes of the ETH3D losses

Figure 7. Causes of the ETH3D losses. (a) courtyard and (b) facade: eLoFTR matches between two different walls with identical windows (red: ground-truth Sampson error above 20 px, green: at most 4 px, yellow: in between). (c) relief: the eLoFTR view graph, with an image per node and an edge per matched pair, splits into the rotated close-ups (left) and the upright views (right), with no matched pair between them.

Focal-depth ambiguity on ETH3D terrains

Figure 8. The focal–depth ambiguity on ETH3D terrains. Top: two views facing the wall (blue, orange) and one corridor view (green). Below: DeepBA in red against ground truth in grey, with the estimated (upper) and ground-truth (lower) focal length. The three example views keep their colors, solid for DeepBA and dashed for ground truth.

Repeated structure. courtyard and facade show building walls with identical window grids. eLoFTR matches the windows of one wall to those of another, so many pairs consist almost entirely of wrong matches that are mutually consistent (Figure 7a,Figure 7b). DeepBA fits these pairs and places the cameras in a wrong configuration.

Disconnected views. Several scenes mix images rotated by 90° in the image plane with upright ones (Figure 7c), or include views of the sky or a ceiling. eLoFTR is not rotation invariant and finds no reliable matches between such groups. The view graph then splits (Figure 7c).

Focal–depth ambiguity. Of the 42 views of terrains, 18 face a flat wall head-on. Their rotations are right, but

DeepBA overestimates their focal length and places them too far from the wall (Figure 8), the pattern of Sec. F.2. Fixing the focal length to its ground-truth value repairs most of terrains.

IMC2021

IMC2021 is the only benchmark in the evaluation set where each camera has different intrinsics. A scene with largely planar facades constrains only the ratio of focal length to depth rather than either quantity alone. A mis-estimated focal length is therefore absorbed by a compensating depth for that camera, displacing its center along its own optical axis while leaving its viewing directions unchanged (Figure 9). The relative translation directions tilt while the relative rotations stay intact, which is the pattern of Table 2.

Focal-depth ambiguity on one IMC2021 bag

Figure 9. The focal–depth ambiguity on one IMC2021 bag (milan_cathedral, 5 views). DeepBA in red against ground truth in grey; the circled camera, shown with its ground truth, is displaced by the focal–depth ambiguity.

Mip-NeRF 360

The room failure. We attribute the room failure to the view graph rather than to scene size. Of room’s 311 cameras and 5,829 verified pairs, a spectral bipartition separates 136 cameras from the other 175, and only 128 pairs cross between the two. As a result, the two clusters are joined too weakly to fix their relative pose, so each is reconstructed well internally while the two are placed independently of one another (Figure 10a).

The room and bonsai failures on Mip-NeRF 360

Figure 10. The room and bonsai failures on Mip-NeRF 360. DeepBA’s reconstructions, with the estimated cameras in green and the ground-truth poses in grey (every third camera in (b)). Only points with a reprojection error below 10 px are shown.

The bonsai failure. Unlike room, bonsai has no view-graph bottleneck and almost no wrong matches. Instead, every camera is pitched up by about the same angle (∼9∘\sim9^\circ, Figure 10b), so cameras on opposite sides of the orbit are rotated against each other and most pairs miss the 10∘10^\circ threshold.

The tilt is a drift of DeepBA’s learned update. Even starting from the ground truth (zero error), DeepBA tilts the

cameras to a median pitch error of 9.4∘9.4^\circ in 100 iterations, and starting from its own output, it keeps tilting them further (Figure 11). Fewer cameras slow the drift but do not stop it. On 32 evenly spread cameras, fewer than the 48 cameras of the largest training view graphs, DeepBA started from the ground truth still reaches 7.0∘7.0^\circ. This example shows a limitation of DeepBA.

Figure 11. The bonsai pitch drift. Median camera pitch error over DeepBA’s iterations on bonsai with 32 to 292 evenly spread cameras, started from DeepBA’s output (left) and from the ground truth (right).

Camera Localization

DeepBA also performs camera localization with no retraining and no architectural change. We build a map from a set of reference views, freeze its cameras and per-view depths, and register each query by rerunning the same refinement iteration with the reference updates zeroed, so no PnP is involved (Figure 12). This is possible because DeepBA’s intermediate state is explicit geometry, absolute poses, per-view depths and intrinsics, so any subset of it can simply be held fixed, whereas a feed-forward regressor carrying only latent features would need an external solver. Figure 13 compares query trajectories against PnP+RANSAC, PnP+RANSAC and HLoc on the same frozen map, and Table 6 reports 7-Scenes under the protocol of HLoc [39], mapping from the official TrainSplit, querying the TestSplit, and reporting the median translation and rotation error per scene. Except for the published HLoc row, every method in that comparison receives the same 200-image DeepBA map rather than the full ∼4,000\sim4{,}000 TrainSplit images, and both PnP+RANSAC and HLoc additionally receive ground-truth query intrinsics. Localization is a by-product of the reconstruction model rather than a main contribution of this paper.

MethodIntrinsics required?ChessFireHeadsOfficePumpkinKitchenStairs
HLoc [39], ∼4,000-image map✓2/0.792/0.872/0.923/0.915/1.124/1.256/1.62
Same map, same queries – only the registration step differs
HLoc [39], our map✓11/1.8810/1.896/1.909/4.7516/3.8510/4.3737/17.86
PnP+RANSAC+GT K✓12/1.9111/1.676/2.0410/4.6815/3.7110/4.2137/17.39
PnPf+RANSAC✗6/1.665/1.673/2.0111/4.8410/3.759/3.9650/17.35
DeepBA✗3/1.663/1.672/1.668/4.808/3.866/4.3030/18.30

Table 6. Camera localization on 7-Scenes [42]. Median translation/rotation error (cm/∘^\circ). The first row is HLoc [39] as published. The other rows share one frozen DeepBA map built from 200 TrainSplit images. Bold/underline: best/second best among them.

Localization refinement iterations

Figure 12. Localization reruns the same refinement iteration in place. With the reference map frozen (its 3D points in their image colors, reference cameras in blue), two example query views (rows) are registered across refinement iterations 0, 1, 2, 10 (columns). From the mean pose of the reference cameras, the query camera and its lifted 3D points (green) converge onto its ground-truth camera (grey) within ∼10\sim10 iterations.

Localization trajectories on a frozen DeepBA map

Figure 13. (a) Object-centric orbit (CO3Dv2). (b) Indoor scene (ScanNet). Localization on a frozen DeepBA map. Query-camera trajectories for four methods registering the same held-out queries against the same frozen DeepBA map, vs. ground truth (grey): DeepBA (green), PnP+RANSAC given ground-truth intrinsics (orange), focal-solving PnPf+RANSAC without known intrinsics (pink), and HLoc [39] (cyan).

References

  1. [1]J. T. Barron, B. Mildenhall, D. Verbin, P. P. Srinivasan, and P. Hedman. Mip-NeRF 360: Unbounded anti-aliased neural radiance fields. In CVPR, 2022. 1, 6, 7, 8, 12
  2. [2]L. Brynte, J. P. Iglesias, C. Olsson, and F. Kahl. Learning structure-from-motion with graph attention networks. In CVPR, 2024. 2, 3, 6, 7arxiv.org/abs/2308.15984
  3. [3]P. H. Chui, H. Andrade-Loarca, and D. Cremers. Track-free dense structure from motion. In WACV, 2027. 2
  4. [4]R. Clark, M. Bloesch, J. Czarnowski, S. Leutenegger, and A. J. Davison. LS-Net: Learning to solve nonlinear least squares for monocular stereo. In ECCV, 2018. 3
  5. [5]A. Dai, A. X. Chang, M. Savva, M. Halber, T. Funkhouser, and M. Nießner. ScanNet: Richly-annotated 3D reconstructions of indoor scenes. In CVPR, 2017. 6
  6. [6]D. DeTone, T. Malisiewicz, and A. Rabinovich. SuperPoint: Self-supervised interest point detection and description. In CVPR Workshops, 2018. 8arxiv.org/abs/1712.07629
  7. [7]B. Duisterhof, L. Zust, P. Weinzaepfel, V. Leroy, Y. Cabon, and J. Revaud. MASt3R-SfM: A fully-integrated solution for unconstrained structure-from-motion. In 3DV, 2025. 2, 6, 7
  8. [8]J. Edstedt, I. Athanasiadis, M. Wadenbäck, and M. Felsberg. DKM: Dense kernelized feature matching for geometry estimation. In CVPR, 2023. 2, 8arxiv.org/abs/2202.00667
  9. [9]J. Edstedt, Q. Sun, G. Böckman, M. Wadenbäck, and M. Felsberg. RoMa: Robust dense feature matching. In CVPR, 2024. 2DOI
  10. [10]O. Enqvist, F. Kahl, and C. Olsson. Non-sequential structure from motion. In ICCV Workshops, 2011. 6DOI
  11. [11]M. A. Fischler and R. C. Bolles. Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, 24(6):381–395, 1981. 6
  12. [12]R. Girshick. Fast R-CNN. In ICCV, 2015. 5
  13. [13]H. Han and H. Yang. Building Rome with convex optimization. arXiv preprint arXiv:2502.04640, 2025. 2arxiv.org/abs/2502.04640
  14. [14]R. Hartley and A. Zisserman. Multiple View Geometry in Computer Vision. Cambridge University Press, 2003. 1, 2DOI
  15. [15]X. He, J. Sun, Y. Wang, S. Peng, Q. Huang, H. Bao, and X. Zhou. Detector-free structure from motion. In CVPR, 2024. 2arxiv.org/abs/2306.15669
  16. [16]J. Heinly, E. Dunn, and J.-M. Frahm. Correcting for duplicate scene structure in sparse 3D reconstruction. In ECCV, 2014. 2DOI
  17. [17]J. H. Hong and C. Zach. pOSE: Pseudo object space error for initialization-free bundle adjustment. In CVPR, 2018. 2DOI
  18. [18]J. H. Hong, C. Zach, A. Fitzgibbon, and R. Cipolla. Projective bundle adjustment from arbitrary initialization using the variable projection method. In ECCV, 2016. 2DOI
  19. [19]P. J. Huber. Robust estimation of a location parameter. The Annals of Mathematical Statistics, 35(1):73–101, 1964. 5
  20. [20]J. P. Iglesias, A. Nilsson, and C. Olsson. expOSE: Accurate initialization-free projective factorization using exponential regularization. In CVPR, 2023. 2DOI
  21. [21]R. Jensen, A. Dahl, G. Vogiatzis, E. Tola, and H. Anaes. Large scale multi-view stereopsis evaluation. In CVPR, 2014. 6, 7, 8, 12DOI
  22. [22]Y. Jin, D. Mishkin, A. Mishchuk, J. Matas, P. Fua, K. M. Yi, and E. Trulls. Image matching across wide baselines: From paper to practice. IJCV, 129(2):517–547, 2021. 6, 7, 8, 12DOI
  23. [23]F. Khatib, Y. Kasten, D. Moran, M. Galun, and R. Basri. RESfM: Robust deep equivariant structure from motion. In ICLR, 2025. 3, 6, 7arxiv.org/abs/2404.14280
  24. [24]V. Leroy, Y. Cabon, and J. Revaud. Grounding image matching in 3D with MASt3R. In ECCV, 2024. 2, 6arxiv.org/abs/2406.09756
  25. [25]X. Li and H. Ling. PoGO-Net: Pose graph optimization with graph neural networks. In ICCV, 2021. 3
  26. [26]Z. Li and N. Snavely. MegaDepth: Learning single-view depth prediction from internet photos. In CVPR, 2018. 6arxiv.org/abs/1804.00607
  27. [27]P. Lindenberger, P.-E. Sarlin, and M. Pollefeys. LightGlue: Local feature matching at light speed. In ICCV, 2023. 8arxiv.org/abs/2306.13643
  28. [28]L. Ling, Y. Sheng, Z. Tu, W. Zhao, C. Xin, K. Wan, L. Yu, Q. Guo, Z. Yu, Y. Lu, et al. DL3DV-10K: A large-scale scene dataset for deep learning-based 3D vision. In CVPR, 2024. 6
  29. [29]L. Lipson, Z. Teed, and J. Deng. RAFT-Stereo: Multilevel recurrent field transforms for stereo matching. In 3DV, 2021. 3arxiv.org/abs/2109.07547
  30. [30]I. Loshchilov and F. Hutter. Decoupled weight decay regularization. In ICLR, 2019. arXiv:1711.05101. 6arxiv.org/abs/1711.05101
  31. [31]M. I. A. Lourakis and A. A. Argyros. SBA: A software package for generic sparse bundle adjustment. ACM Transactions on Mathematical Software (TOMS), 36(1):1–30, 2009. 2
  32. [32]D. G. Lowe. Distinctive image features from scale-invariant keypoints. IJCV, 60:91–110, 2004. 2, 8DOI
  33. [33]D. Moran, H. Koslowsky, Y. Kasten, H. Maron, M. Galun, and R. Basri. Deep permutation equivariant structure from motion. In ICCV, 2021. 3arxiv.org/abs/2104.06703
  34. [34]R. Mur-Artal, J. M. M. Montiel, and J. D. Tardos. ORB-SLAM: A versatile and accurate monocular SLAM system. IEEE Transactions on Robotics, 31(5):1147–1163, 2015. 1
  35. [35]D. Nistér. An efficient solution to the five-point relative pose problem. IEEE TPAMI, 26(6):756–770, 2004. 2
  36. [36]C. Olsson and O. Enqvist. Stable structure from motion for unordered image collections. In Scandinavian Conference on Image Analysis (SCIA), 2011. 6DOI
  37. [37]L. Pan, D. Baráth, M. Pollefeys, and J. L. Schönberger. Global structure-from-motion revisited. In ECCV, 2024. 2, 6, 7arxiv.org/abs/2407.20219
  38. [38]J. Reizenstein, R. Shapovalov, P. Henzler, L. Sbordone, P. Labatut, and D. Novotny. Common objects in 3D: Large-scale learning and evaluation of real-life 3D category reconstruction. In ICCV, 2021. 6, 12arxiv.org/abs/2109.00512
  39. [39]P.-E. Sarlin, C. Cadena, R. Siegwart, and M. Dymczyk. From coarse to fine: Robust hierarchical localization at large scale. In CVPR, 2019. 14, 15arxiv.org/abs/1812.03506
  40. [40]J. L. Schönberger and J.-M. Frahm. Structure-from-motion revisited. In CVPR, 2016. 1, 2, 6, 7DOI
  41. [41]T. Schöps, J. L. Schönberger, S. Galliani, T. Sattler, K. Schindler, M. Pollefeys, and A. Geiger. A multi-view stereo benchmark with high-resolution images and multi-camera videos. In CVPR, 2017. 6, 7, 8, 12DOI
  42. [42]J. Shotton, B. Glocker, C. Zach, S. Izadi, A. Criminisi, and A. Fitzgibbon. Scene coordinated regression forests for camera relocalization in RGB-D images. In CVPR, 2013. 15DOI
  43. [43]C. Smith, D. Charatan, A. Tewari, and V. Sitzmann. FlowMap: High-quality camera poses, intrinsics, and depth via gradient descent. arXiv preprint arXiv:2404.15259, 2024. 2arxiv.org/abs/2404.15259
  44. [44]H. Strasdat, J. M. M. Montiel, and A. J. Davison. Scale drift-aware large scale monocular SLAM. In Robotics: Science and Systems (RSS), 2010. 2
  45. [45]J. Sun, Z. Shen, Y. Wang, H. Bao, and X. Zhou. LoFTR: Detector-free local feature matching with transformers. In CVPR, 2021. 2arxiv.org/abs/2104.00680
  46. [46]T. Tanaka, Y. Sasagawa, and T. Okatani. Learning to bundle-adjust: A graph network approach to faster optimization of bundle adjustment for vehicular SLAM. In ICCV, 2021. 3DOI
  47. [47]C. Tang and P. Tan. BA-Net: Dense bundle adjustment networks. In ICLR, 2019. 3
  48. [48]Z. Teed and J. Deng. RAFT: Recurrent all-pairs field transforms for optical flow. In ECCV, 2020. 3arxiv.org/abs/2003.12039
  49. [49]Z. Teed and J. Deng. DROID-SLAM: Deep visual SLAM for monocular, stereo, and RGB-D cameras. In NeurIPS, 2021. 3arxiv.org/abs/2108.10869
  50. [50]Z. Teed and J. Deng. RAFT-3D: Scene flow using rigid-motion embeddings. In CVPR, 2021. 3arxiv.org/abs/2012.00726
  51. [51]P. H. S. Torr, A. W. Fitzgibbon, and A. Zisserman. The problem of degeneracy in structure and motion recovery from uncalibrated image sequences. IJCV, 32(1):27–44, 1999. 2DOI
  52. [52]B. Triggs, P. F. McLauchlan, R. I. Hartley, and A. W. Fitzgibbon. Bundle adjustment — a modern synthesis. In International Workshop on Vision Algorithms, 1999. 1, 2DOI
  53. [53]P. Truong, M.-J. Rakotosaona, F. Manhardt, and F. Tombari. SPARF: Neural radiance fields from sparse and noisy poses. In CVPR, 2023. 12arxiv.org/abs/2211.11738
  54. [54]A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin. Attention is all you need. In NeurIPS, 2017. 4
  55. [55]J. Wang, C. Rupprecht, and D. Novotny. PoseDiffusion: Solving pose estimation via diffusion-aided bundle adjustment. In ICCV, 2023. 2, 6arxiv.org/abs/2306.15667
  56. [56]J. Wang, N. Karaev, C. Rupprecht, and D. Novotny. VGGSfM: Visual geometry grounded deep structure from motion. In CVPR, 2024. 2, 6DOI
  57. [57]J. Wang, M. Chen, N. Karaev, A. Vedaldi, C. Rupprecht, and D. Novotny. VGGT: Visual geometry grounded transformer. In CVPR, 2025. 2, 6, 7, 12arxiv.org/abs/2503.11651
  58. [58]S. Wang, V. Leroy, Y. Cabon, B. Chidlovskii, and J. Revaud. DUSt3R: Geometric 3D vision made easy. In CVPR, 2024. 2, 6arxiv.org/abs/2312.14132
  59. [59]Y. Wang, X. He, S. Peng, D. Tan, and X. Zhou. Efficient LoFTR: Semi-dense local feature matching with sparse-like speed. In CVPR, 2024. 6, 8arxiv.org/abs/2403.04765
  60. [60]S. Weber, J. H. Hong, and D. Cremers. Power variable projection for initialization-free large-scale bundle adjustment. In ECCV, 2024. 2arxiv.org/abs/2405.05079
  61. [61]Y. Yao, Z. Luo, S. Li, J. Zhang, Y. Ren, L. Zhou, T. Fang, and L. Quan. BlendedMVS: A large-scale dataset for generalized multi-view stereo networks. In CVPR, 2020. 6arxiv.org/abs/1911.10127
  62. [62]X. Zhao, X. Wu, W. Chen, P. C. Y. Chen, Q. Xu, and Z. Li. ALIKED: A lighter keypoint and descriptor extraction network via deformable transformation. IEEE Transactions on Instrumentation and Measurement, 2023. 8arxiv.org/abs/2304.03608
  63. [63]T. Zhou, R. Tucker, J. Flynn, G. Fyffe, and N. Snavely. Stereo magnification: Learning view synthesis using multiplane images. In SIGGRAPH, 2018. 6, 12arxiv.org/abs/1805.09817

Paper details

vlnc:W_trpp5xvj
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 1

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents