betaFeedback

WorksComputer Science

Polygraph: Large-Scale Multi-Session Trajectory Optimization using Pose Graphs

Abstract

Robots, vehicles, and wearable devices often traverse the same environment many times. Each pass produces its own trajectory from onboard visual or visual-inertial odometry. These trajectories are locally accurate but drift over time and live in unrelated coordinate frames. Turning many such independent trajectories into one globally consistent set is the large-scale multi-session trajectory optimization problem. Existing solutions largely work offline. All sessions must be available up front, one session is fixed as the reference frame, and a single fused map is then optimized. This paper presents Polygraph, a multi-session trajectory optimization pipeline that processes sessions online, one keyframe and one spatial cluster at a time, and never builds a 3D map. Keyframes are grouped into local clusters. When a cluster closes, it is matched against every other session through learned image retrieval. Candidate pairs are verified with sparse feature matching, monocular metric depth, and robust pose estimation. A two-tier consensus gate admits a cross-session edge into the shared pose graph only once independent evidence corroborates it. A cheap incremental solve keeps the graph current, and a periodic robust batch solve prunes outliers. We evaluate Polygraph on stereo outdoor driving, HoloLens and iPad indoor traversals, legged indoor locomotion with a Boston Dynamics Spot, and large-scale outdoor vehicle and handheld routes. Against the closest prior work on the same GPU model, using its reported timings, Polygraph runs faster, commits fewer loop-closure edges, and matches or improves on accuracy.

Introduction

Many robots, vehicles, and wearable devices see the same environment more than once. A delivery robot drives the same warehouse floor every day. A person wearing an AR headset walks through the same building on many occasions. We call each such pass a session. Each session is tracked by the device’s own visual or visual-inertial odometry. This odometry is accurate locally but drifts over time, and every session lives in its own coordinate frame. Two sessions that walk the same corridor have no way of knowing that they did. Recovering that relationship is the multi-session trajectory optimization problem. It means detecting where sessions overlap, using those overlaps to correct drift, and placing every session into one shared frame. Figure 1 shows what this looks like once solved.

Loop-closure visualisation

Figure 1. Loop-closure visualisation. Five independent HYDRO sessions recorded with a Boston Dynamics Spot, each in its own color, placed into one shared frame by Polygraph and overlaid on the site’s 3D scan. Purple lines mark confirmed cross-session loop-closure edges. No two of these trajectories start out knowing that they share the same building.

The classical way to detect overlap is to build and maintain a 3D map of visual landmarks and match new observations against it, as in ORB-SLAM3 Atlas [1], Maplab [2, 3], and COVINS [4]. A map gives strong geometric evidence for a match, but is expensive to store and keep optimized; bundle adjustment grows linearly in landmarks and cubically in poses, so this cost grows with every session and with the size of the environment. A well-built map is also rarely what downstream modules want, since planning and navigation seldom consume a landmark cloud directly.

A recent line of work avoids keeping any map at all. Multi-robot systems such as DOOR-SLAM [5], Swarm-SLAM [6], and FLOCC-SLAM [7] treat closures as pose-graph edges directly but still assume every robot is known up front. 2GO [8] instead represents a single session only as a sparse keyframed pose graph, loop closures found by learned image retrieval, verified with sparse feature matching between two views, and lifted to metric scale with a monocular depth prior or multi-view transformer. However, 2GO handles one session at a time. It cannot relate independent sessions to each other, and its consistency check relies on a live trajectory estimate that exists from the first keyframe onward, which a new session does not have.

This paper presents Polygraph, a multi-session trajectory optimization pipeline that closes this gap. Polygraph processes each session online, one keyframe and one spatial cluster at a time. It does not assume that all sessions are available up front, and it does not fix any session as a reference. When a cluster closes, it is matched with every other session by learned retrieval. Candidate pairs are filtered and verified with sparse matching, monocular depth, and robust pose estimation, and reduced to one confirmed match per cluster pair. A cross-session edge only enters the shared graph once a two-tier consensus gate admits it, which protects the graph from spurious matches. A cheap incremental solver keeps the graph current after every cluster, and a periodic batch solve prunes outlier edges. The algorithm is agnostic to the source of the odometry measurements and simply takes calibrated images and odometry edges as input.

We evaluate Polygraph on stereo outdoor driving on KITTI [9], HoloLens and iPad indoor traversals from LaMAR [10], legged indoor locomotion from CroCoDL [11], vehicle and handheld routes from the large-scale outdoor VBR dataset [12] and small-scale indoor EuRoC dataset [13]. Against the closest prior work 2GO on the same GPU model, using its reported timings, Polygraph runs faster, commits fewer loop-closure edges, and matches or improves on accuracy.

In summary, this paper makes the following contributions.

  • An online, reconstruction-free formulation of multi-session pose graph optimization that needs neither all sessions up front nor a fixed reference session.

  • A cluster-level matching and joint pose estimation pipeline with a two-tier consensus gate that admits cross-session edges only once they are corroborated.

  • A characterization of runtime, accuracy, and loop-closure accuracy across a wide range of devices and scenes, including hardware-matched comparison against 2GO and comparison against other similar methods.

Related Work

Large-Scale and Multi-Session SLAM

ORB-SLAM3 [14] and VINS-Fusion [15, 16] track a camera in real time and reach global consistency through loop closures and bundle adjustment over a landmark map, which scales poorly with map size. Multi-session systems extend this recipe. ORB-SLAM3 Atlas [1] merges several maps on place recognition, Maplab [2, 3] offers an open multi-session visual-inertial framework, and COVINS [4] adds a centralized back-end on an ORB-SLAM3 front-end. All rely on map points, so fusion cost is tied to keeping geometry per session. Polygraph uses such a system, or a device’s own odometry, only as input. Its cost scales with keyframes rather than scene density, and its online solve builds on iSAM2 [17].

Visual Place Recognition

Deciding which keyframes might show the same place is a retrieval problem, with each image embedded into a global descriptor for nearest-neighbor search. NetVLAD [18] introduced the learned aggregation approach, and recent descriptors such as SALAD [19], BoQ [20], and MegaLoc [21] improve robustness to viewpoint and appearance changes. Polygraph uses such a descriptor to cast votes between clusters of keyframes. Unlike single-session loop closure, its index spans every other session and must grow as new sessions arrive …

Learned Two-View Matching and Monocular Priors

A retrieval hit is only a hypothesis and must be verified with correspondences between the two images. Learned matchers such as SuperPoint [22] with SuperGlue [23], DISK [24] with LightGlue [25], the lightweight XFeat [26], and the detector-free LoFTR [27] have made two-view matching robust under large viewpoint change, but two-view geometry from correspondences alone only yields a relative pose up to scale. Dense reconstruction with MAsT3R [28] instead regresses a metric point map directly from the pair, recovering scale without any external depth prior. Monocular metric depth models such as Metric3Dv2 [29] recover that same scale from a single image, independent of any two-view matcher.

Map-Less Multi-Session Pose Graph Optimization

Multi-robot SLAM has long treated inter-robot loop closures as pose graph edges rather than map merges. DOOR-SLAM [5], Kimera-Multi [30], and Swarm-SLAM [6] detect such edges with global descriptors and local feature verification, rejecting outliers with pairwise consistency maximization [31] or graduated non-convexity [32] before a distributed solve. FLOCC-SLAM [7] keeps that formulation but swaps classical feature verification for a 3D foundation model, closer to Polygraph’s own two-view verification. COVINS-G [33] centralizes the idea instead, keeping only 2D keypoints and odometry poses on the server and recovering a metric pose by treating a keyframe and its odometry neighbors as one multi-camera rig. All of these rely on a second view for scale and on every robot being known up front. 2GO [8] instead combines learned matching and a monocular depth prior into a pose graph pipeline that never reconstructs the scene, checking each candidate against the trajectory’s own live pose estimate, always available since the graph is one connected chain from the first keyframe. Polygraph reuses this verification recipe with faster components, XFeat with LightGlue [25, 26] and MegaLoc [21], but a newly arriving session has no live estimate and no known relationship to the rest of the graph. Its two-tier consensus gate bootstraps that trust from scratch, playing the role pairwise consistency plays in multi-robot SLAM, but at the cluster level and without assuming the set of sessions is known. Polygraph’s final solve also uses graduated non-convexity to reject edges that survive gating but remain outliers.

Method

We will start with a brief outlook on our pipeline. Figure 3 gives this overview. The odometry of each session is keyframed and clustered online. A closed cluster is matched against every other session by retrieval, candidate pairs are verified as confirmed matches, and a gate decides whether a cross-session match may enter the shared pose graph. The rest of this section follows this order.

Joint pose estimation

Figure 3. Joint pose estimation. Turns one candidate cluster pair into one confirmed match. A spanning tree over descriptor similarity selects a few keyframe pairs (purple, one wrong pair in red). Each pair yields its own metric pose, which is lifted through the odometry into the common cluster frame. Estimates that agree within τagree\tau_{\mathrm{agree}} form the consensus set, and the outlier is dropped. The chordal mean of the set is the measurement the gate decides on, and the agreeing pairs enter the graph as factors once the match is admitted.

Problem Formulation

Let SS be a set of sessions that grows over time. Each session s∈Ss \in S is tracked by its own odometry, which yields poses Ts,iraw∈SE(3)T^{\mathrm{raw}}_{s,i} \in\mathrm{SE}(3) in a local frame that drifts and has no known relation to any other session. Polygraph thins this stream to keyframes and works only at the keyframe level from then on. We write Ts,jT_{s,j} for the pose of the jj-th keyframe of session ss.

We build a graph G=(V,E)G = (V, E) with one vertex per keyframe. The edge set has three parts. Odometry edges connect consecutive keyframes of one session with the relative pose read directly from the odometry, trusted and never removed. Intra-session edges are loop closures between non-consecutive keyframes of the same session; cross-session edges connect keyframes of two different sessions, and both types come from the same retrieval and verification pipeline. Only cross-session edges pass through the consensus gate, since only they can move one session relative to another. With XvX_v the pose of vertex vv in the shared frame of its connected component, the graph is solved by

min⁡{Xv}∑(u,v)∈Eρuv(r(Xu,Xv,zuv))+∑v∈V0∥Xv⊖pv∥Σ02,(1)\min_{\{X_v\}} \sum_{(u,v)\in E} \rho_{uv}\bigl(r(X_u,X_v,z_{uv})\bigr) + \sum_{v\in V_0} \lVert X_v \ominus p_v \rVert_{\Sigma_0}^{2}, \tag*{(1)}

where r(Xu,Xv,z)=Log⁡(z−1Xu−1Xv)r(X_u,X_v,z) = \operatorname{Log}(z^{-1}X_u^{-1}X_v) is the residual SE(3) of the measurement zuvz_{uv} under its covariance, Xv⊖pv=Log⁡(pv−1Xv)X_v \ominus p_v = \operatorname{Log}(p_v^{-1}X_v) is the same tangent-space difference against the prior pvp_v, and ρuv\rho_{uv} is the identity for the edges of the odometry and a Cauchy loss for loop closures. The set V0V_0 contains one gauge-fix prior per session. Two properties separate this from an offline formulation. Both VV and EE grow one keyframe and edge at a time, and V0V_0 is never fixed in advance. Every session starts as its own component with its own prior, and components merge when a cross-session edge survives the gate.

Keyframing and Online Clustering

A raw frame becomes a keyframe once it has moved more than τt\tau_t or rotated more than τr\tau_r from the previous keyframe. Each keyframe receives a MegaLoc [21] global descriptor only at that time. The keyframes are then grouped into spatially bounded clusters using only the session’s own travel distance, as in the second panel of Figure 2. A cluster closes once it has at least nmin⁡n_{\min} keyframes and has traveled more than dmax⁡d_{\max} from its first keyframe or reached nmax⁡n_{\max} keyframes. The last novn_{\mathrm{ov}} keyframes of a closing cluster seed the next one, so consecutive clusters chain without a coverage gap. Clusters, not keyframes, are the unit that all later stages discuss. A cluster becomes visible only to the rest of the pipeline once it closes.

Polygraph pipeline

Figure 2. Polygraph pipeline. Each session’s odometry is keyframed and grouped online into spatial clusters. A closed cluster is matched against every other session by image retrieval, candidate pairs are verified by joint pose estimation into one confirmed match, which enters the shared pose graph once the consensus gate admits it. A fast incremental solve runs after every closed cluster, and a periodic full solve prunes outliers.

Cluster Matching

We identify loop-closure candidates by matching clusters, by checking potential matches of a closed cluster CC against every other cluster. To reduce computational cost, we first apply a cheap retrieval filter that reduces the field to a short list of candidate pairs. It runs once per session s′s', including C′C'’s own session for intra-session loop closures, and never ranks candidates from different sessions against each other. Every keyframe of CC queries the descriptors of s′s' by cosine similarity, as sketched in the cluster matching panel of Figure 2, and casts one vote for the parent cluster of each of its ktopk_{\mathrm{top}} nearest neighbours whose similarity clears a floor. When s′s' is the session of CC itself, the two clusters adjacent to CC are excluded, since they share keyframes by construction. A session in which fewer than npartn_{\mathrm{part}} keyframes of CC found anything to vote for is dropped. Otherwise, each candidate C′C' is scored by its vote share

share⁡(C,C′)=votes for C′ within s′all votes cast within s′,(2)\operatorname{share}(C,C') = \frac{\text{votes for } C' \text{ within } s'}{\text{all votes cast within } s'}, \tag*{(2)}

Candidates with a share greater than τshare\tau_{\mathrm{share}} are kept, at most kclk_{\mathrm{cl}} per session. The output is a short list of candidates for cluster pairs, not a list of matches. Proximity retrieval, as in [8], adds a second, appearance-free candidate source within the same session, using odometry position and heading, not descriptors. Proximity retrieval is by default turned off, but as we will see later, it can be helpful when dealing with long loop-less trajectories. The candidates for loop-closure cluster pairs are then checked for a robust consensus in estimated relative pose.

Joint Pose Estimation

This stage is the only one that looks at image content, and it turns one candidate pair into one confirmed match or rejects it, as Figure 3 illustrates. Relative pose estimation between each keyframe of CC and every keyframe of C′C' would be redundant, so a maximum spanning tree over the bipartite descriptor-similarity graph selects kpairsk_{\mathrm{pairs}} keyframe pairs that are both similar and spread across both clusters.

Each selected pair (k,k′)(k, k') goes through the same chain that closely follows [8]: the features of XFeat [26] are matched with LighterGlue [25], [26]. Essential-matrix RANSAC yields a scale-free relative rotation and translation direction with an inlier set, and rejects the pair early if too few inliers survive. Inliers are triangulated into scale-free 3D points in the frame of kk. Metric3D v2 [29] predicts a metric depth map for each image independently, and every matched keypoint is back-projected through its own depth map to a metric point. For each correspondence, one scalar is fit to best align its 3D triangulated point with its 3D depth-lifted point, an overdetermined one-parameter fit across x, y, and z, and correspondences whose residual after scaling exceeds τscale\tau_{\mathrm{scale}} are discarded. Finally, the surviving depth-lifted points of k′k' and their pixel observations in kk are handed to PnP RANSAC, which yields the metric pose T^k,k′∈SE(3)\hat{T}_{k,k'} \in\mathrm{SE}(3). A pair is rejected unless enough correspondences survive the depth and scale-consistency checks to run PnP at all, and the resulting (post-RANSAC) inlier ratio and model score clear their own floors.

To summarize transforms between keyframes into a transform between clusters, we propagate each individual estimate along the odometry edges into a relative transform between the two clusters’ reference keyframes. Two estimates agree if their implied translations differ by less than τagree\tau_{\mathrm{agree}}. The estimate with the most agreeing pairs anchors a consensus set, averaged by a chordal SE(3)\mathrm{SE}(3) mean into the measurement z^\hat{z} between the two clusters’ reference keyframes. A candidate becomes a confirmed match only if the consensus set holds at least nvoten_{\mathrm{vote}} estimates; a single uncorroborated estimate is never enough, and the gate and outlier checks act on z^\hat{z} alone. With multi_edge_consensus on, every consensus-set member is also added as its own factor once a match is admitted, so the optimizer sees the spread of evidence rather than only its mean. This proved especially helpful for densifying the graph when revisits are sparse.

Two-Tier Consensus Gate

A confirmed cross-session match is still a single piece of evidence from one retrieval hit, and admitting it directly would let one spurious match merge two sessions. The gate in Figure 4 decides whether and when a match enters the graph as a loop-closure edge. An edge whose clusters both belong to the same session commits immediately, since that session’s position is already fixed by its odometry chain, and the edge only adds corroboration.

The consensus gate

Figure 4. The consensus gate. Same-session matches commit directly. A cross-session edge whose query session has no live cross-session edge yet must find an agreeing, independent peer in the corroboration pool. Once a session is connected, each edge is checked against its live estimate and queued for later re-checks if it fails.

For a cross-session edge between ss and s′s' the gate first picks an anchor and a query. Let n(s)n(s) be the number of live cross-session edges incident on ss. The session with more live edges is the anchor, its estimate fixed for this check, and the other is the query, whose count selects a tier.

Tier 1, corroboration pool. If the query has no live cross-session edge, the candidates that target it are pooled and checked only against each other. Once at least mcorrobm_{\mathrm{corrob}} pooled candidates imply mutually consistent placements, within τpool\tau_{\mathrm{pool}} in translation and rotation, and are independent (anchor-side clusters at least gmin⁡g_{\min} apart), they commit together, so the query session always joins with two or more edges at once, never exactly one.

Tier 2, pending queue. If the query already has a live cross-session edge, the candidate is checked immediately against its current estimate under a translation and rotation tolerance and commits if consistent. If not, it enters a pending queue, re-checked at every full update, until it passes, collects rmax⁡r_{\max} rejections, or ages out, so an edge that arrived before the graph settled can still be admitted later.

The gate is what allows a session to arrive with no known relation to the rest of the graph. It plays the role that pairwise consistency checks play in multi-robot SLAM, but at cluster level, online, and without assuming that the set of sessions is known.

Pose Graph Optimization

Every admitted edge becomes a factor in (1). A fast update runs iSAM2 [17] after every closed cluster, folding in new vertices and edges incrementally, while a full update runs every nfulln_{\mathrm{full}} closed clusters as a batch solve over the whole graph. After that, a loop-closure edge is pruned if its residual exceeds κ\kappa times the median residual of its group; intra-session and cross-session edges are judged against their own medians so one session’s tight loop closures do not raise the bar for everyone else. The full update also drains the pending queue against the fresh estimate.

Once every session has been processed, one last full update runs and two final passes follow. First, each session pair’s edges are cross-checked using the pre-fusion local poses captured when each edge was created, exposing groups of edges that are wrong but mutually consistent, which residual-based pruning misses since least squares simply bends the graph to satisfy them. Second, a graduated non-convex solve with a truncated least-squares loss [32] runs over the complete graph and assigns every loop closure a final outlier weight. The result is one pose graph over every connected session. Two sessions never connect with fewer than two corroborated loop closures at once, and once connected, they are solved jointly, so the correction propagates across each trajectory as a whole rather than staying local to the overlap.

Experiments

We evaluate on the five datasets given in Table II. KITTI [9] and VBR [12] are large outdoor benchmarks that allow us to test scalability to long trajectories and large maps. HYDRO and HGE are indoor sites captured by more than one device and test robustness to viewpoint and hardware. Lastly, EuRoC [13] lets us isolate pose-estimation accuracy against a two-view baseline, used only in the ablation section below. We report the following metrics for our experiments: ATE is the RMSE translation error after a per-session rigid alignment to ground truth. Where sessions share ground truth coverage, we also report reference-anchored consistency: one session per component aligned to ground truth, and that fixed transform applied to the rest. This latter metric exposes a multi-session trajectory that is locally consistent but globally wrong, in a way that ATE alone cannot.

DatasetSettingDevicesSess.Multi-device
KITTIoutdoor, drivingcar, stereo7no
VBRdriving, handheldcar / handheld8no
HYDROindoorSpot, iPad10yes
HGEindoorHoloLens, iPad9yes
EuRoCindoor, MAVMAV, stereo(-IMU)11yes

Table II. Datasets used throughout Experiments. Multi-device marks a site captured by more than one platform. VBR’s multi-session sites include two campus sessions.

Single-Session

We evaluate on KITTI and VBR. Both datasets are outdoor, large-scale, and the setting our method targets. We match the experimental setup of 2GO [8], refining the same raw odometry sources in Table I. ‘single’ treats a sequence alone, ‘multi’ runs a dataset together with cross-session edges, and ‘Total’ is the L2 norm over a dataset’s rows. ORB-SLAM3 lost tracking on diag_train0, excluded from that column. VBR’s VINS-Fusion backbone uses one config per platform (keyframe spacing 5.0 m vehicle, 2.5 m handheld).

SequenceLengthDuration\multicolumn{7}{c}{ORB-SLAM3 odometry}\multicolumn{7}{c}{VINS-Fusion odometry}
odom onlyORB-SLAM3Maplab BINMaplab SP2GOOurs singleOurs multiodom onlyVINS-PGOMaplab BINMaplab SP2GOOurs singleOurs multi
KITTI-003.72 km470 s4.204.8537.5112.691.821.281.2713.094.2011.1311.391.581.341.33
KITTI-025.07 km483 s8.055.6418.0144.788.045.305.3021.0213.0323.9618.2910.587.967.96
KITTI-052.21 km288 s2.050.957.447.980.920.890.896.393.6811.3711.211.701.881.88
KITTI-061.23 km114 s2.050.797.0910.740.620.720.723.541.6813.2714.761.481.361.36
KITTI-070.69 km114 s1.190.505.605.630.410.480.382.110.657.487.700.460.570.55
KITTI-083.22 km423 s3.503.6740.7911.653.783.243.2410.4310.0117.0414.274.038.468.46
KITTI-091.71 km165 s2.873.2210.2311.321.841.711.717.807.7115.9712.314.663.323.32
Total (L2)10.629.0060.3151.399.336.696.6828.9919.0740.0634.9612.5612.3912.39
campus_train02.73 km602 s28.9211.9430.7328.229.329.8410.2747.7443.8449.4646.2812.3011.6511.55
campus_train12.95 km584 s28.567.9930.8827.428.986.436.3146.9423.4044.4334.939.608.757.82
ciampino_train09.01 km1549 s82.8268.1183.8981.7718.2918.2518.25131.3395.95130.20130.9422.8416.7216.72
ciampino_train15.20 km942 s25.9813.8327.6125.0013.2513.2213.2247.2134.6840.9440.7914.2012.9112.91
colosseo_train01.45 km882 s11.099.679.4010.296.804.954.9521.4913.3721.4821.476.004.994.99
diag_train01.04 km1003 s///////27.3533.1027.1727.2326.702.972.97
pincio_train01.28 km1114 s6.302.086.316.912.791.961.9611.297.7511.2211.233.184.754.75
spagna_train01.56 km1414 s11.425.6211.9512.016.487.537.5321.1413.2220.8320.353.733.163.16
Total (L2)97.3671.8899.8495.6927.8127.0427.17160.45119.95157.50154.6441.7126.9226.59

Table I. RMSE ATE on KITTI’s loop sequences and on VBR. Refining ORB-SLAM3 (left) and VINS-Fusion (right) odometry [8]. Best per odometry source shaded and bold, second best underlined.

Its ORB-SLAM3 backbone needed one per location instead, since ORB-SLAM3’s larger rotation drift on VBR’s tight turns is harder for retrieval and pose verification to absorb. On KITTI, each odometry shares a single configuration file across all 7 sessions.

Polygraph’s single-session column is already competitive with, and usually ahead of, 2GO on both odometry sources. Figure 5 shows four VBR routes, single-session only. Odometry drifts by tens to hundreds of meters, and Polygraph pulls every route back close to ground truth from retrieval, two-view matching, and a monocular depth prior alone, using only that session’s own revisits. This evaluation indicates that the additional sparsity from keyframe clustering, compared to the direct loop-closures between keyframes in 2GO or the dense bundle-adjustment in ORB-SLAM3 does not come at a cost of accuracy, and instead yields overall more accurate single-session solutions. We attribute this accuracy gain to the additional filtering and consensus steps for the clusters.

Figure 5. Single-session VBR visualisation. Two handheld and two vehicle. Top shows VINS-Fusion odometry; bottom shows Polygraph, both against ground truth in gray. Shades mark spatial clusters; dots mark clusters joined by a surviving loop closure.

Multi-Session

The multi-session results in Table I on KITTI and VBR overall show improvements compared to the single-session results. Importantly, multi-session never falls behind single-session by more than 0.03 m anywhere. However, trajectories on these datasets in general barely overlap geographically.

Multi-Robot Split. Table III gives KITTI a second, genuinely multi-session test, following DOOR-SLAM [5] and Swarm-SLAM’s [6] own KITTI-00 split protocol, one sequence cut into 2 or 3 equal parts, each its own session on its own odometry, run concurrently. Joint ATE and RMSE after one rigid alignment of every robot together stay well below both baselines on every odometry source we tried. RTF is against the longest session in the split, both means of 5 runs.

MethodSplit (robots)Joint ATE (m)RTF
Swarm-SLAM [6]22.17–
Ours, w/ RTAB-Map odom.21.160.81
Ours, w/ ORB-SLAM3 odom.21.220.74
Ours, w/ VINS-Fusion odom.21.470.73
DOOR-SLAM [5]38.00–
Ours, w/ RTAB-Map odom.31.260.95
Ours, w/ ORB-SLAM3 odom.31.200.91
Ours, w/ VINS-Fusion odom.32.310.97

Table III. KITTI multi-robot experiment. KITTI-00 cut into 2 or 3 equal parts, one robot each, against DOOR-SLAM and Swarm-SLAM on their own protocol, RTAB-Map stereo odometry for both. Best shaded and bold, second underlined, per split. RTF is conservative, run on a shared cluster.

HYDRO [11] and HGE [10] allow for large-scale multi-session evaluations. Both sites are captured across many sessions per device: 5 Spot and 5 iOS sessions at HYDRO, and 5 HoloLens and 4 iOS sessions at HGE, each device already tracked by onboard odometry. We tune one config per site on its primary device, Spot at HYDRO and HoloLens at HGE, then run iOS with the same config. Table IV reports single-session and multi-session ATE per device. Multi-session beats both raw odometry and single-session on every device at both sites, even though each device’s built-in odometry already achieves high accuracy. Polygraph successfully improves the poses from these odometry sources in each case.

DeviceATE (m)Anchoring, mixed
odom onlymulti-sessionmixedtrans (m)trans (%)rot (deg)
HYDROSpot Mean0.2990.1490.1480.1810.200.39
HYDROSpot L20.7270.3360.3360.3620.410.79
HYDROiOS Mean0.4080.3120.3090.4180.470.62
HYDROiOS L20.9900.7220.7210.9631.081.42
HGEHoloLens Mean0.6580.2800.2810.9560.570.84
HGEHoloLens L22.1920.6680.6682.1041.261.73
HGEiOS Mean0.5550.5310.5230.6970.420.94
HGEiOS L21.1901.1311.1211.4450.872.04

Table IV. RMSE ATE on HYDRO [11] (5 Spot and 5 iOS sessions, footprint diagonal 89 m) and HGE [10] (5 HoloLens and 4 iOS sessions, footprint diagonal 167 m). Each device on its own odometry, and reference-anchoring error with every session of a site placed into one shared frame. Pooled per device as mean and L2 norm. Best of multi-session and mixed shaded and bold, second best underlined.

Overall, we observe that our multi-session optimization is effective across 4 datasets and outperforms Swarm-SLAM and DOOR-SLAM on their KITTI-Splits by large margins.

Multi-Device

Beyond combining sessions from the same device, HYDRO and HGE let us mix two entirely different devices into one shared graph. The results for this are shown in Table IV’s mixed column. Mixed usually beats standalone multi-session, too, since a second device adds more chances to close a loop. No external baseline exists for either site. Anchoring translation is in meters and percent of the site’s footprint diagonal, allowing comparison of HYDRO and HGE despite their different scale. Figure 6 shows why the reference-anchoring

Reference anchoring on HGE and HYDRO

Figure 6. Reference anchoring on HGE (top) and HYDRO (bottom). One fixed transform per connected component applied to every session past its reference. Ground truth in green, Polygraph’s estimate on top. On HGE, (a) is HoloLens alone with two reference groups, (b) is iOS alone, (c) is HoloLens and iOS anchored together. On HYDRO, (a) is Spot alone, (b) is Spot and iOS together, all ten sessions in one component.

numbers matter beyond ATE alone. HGE’s HoloLens sessions split into two disconnected reference groups on their own, and adding iOS bridges them into one shared frame.

Ablation

Camera Pairing. Table V shows that sensor rigs with more cameras help most where they add a genuinely new viewpoint. HYDRO’s four cameras (from Spot) more than double cross-session edges and improve single-session ATE, while KITTI’s second camera, the other half of the same stereo rig, changes nothing.

DatasetCamerasSingle (m)Multi (m)Cross edges
HGE1, baseline0.6520.280252
4, best0.6520.278506
HYDRO1, baseline0.2360.1491484
4, best0.1970.1463672
KITTI1, baseline1.9461.93018
2, best1.9461.93018

Table V. Camera-pairing ablation. Baseline is single-camera production; best is the best combination available. The better ATE per pair is shaded and bold.

Pose Estimation. Figure 7 justifies our choice of XFeat+LightGlue for pose estimation over LoFTR and MAsT3R. Raw pair success rate does not predict full-pipeline accuracy. On HGE, MAsT3R succeeds on 91.8% of pairs against XFeat+LightGlue’s 76.8%, yet its mean translation error on successful pairs is almost twice as high, 0.961 m against 0.544 m. The full pipeline flips this ranking. LoFTR and MAsT3R-native each land sessions in badly misaligned components, and switching matcher costs 15 to 25% more wall time for no accuracy gain. Reference anchoring makes the case more clearly than ATE. MASt3R-depth variants look competitive on raw HGE ATE but land 18 to 78 m off once anchored, a locally corroborated but globally wrong edge, the real reason production keeps Metric3D. Averaged across all four datasets, XFeat+LighterGlue with Metric3D ties MASt3R with Metric3D on accuracy, 2.20 m mean ATE each, at 21% less wall time. Swapping only the depth source, XFeat+LighterGlue with MASt3R’s own depth, is faster still but 0.07 m worse, mostly a 0.3 m gap on KITTI. Table VI isolates pose-estimation accuracy directly on EuRoC [13] versus more traditional pose estimation methods. Against COVINS-G [33], a genuinely two-view, IMU-fused system, our committed edges average 0.48 m translation error before graph optimization, compared with its 17-point variant at 0.07 m.

Figure 7. Pose-estimation ablation. Mean wall time against mean ATE across KITTI, HGE, HGE iOS, and HYDRO, both raw values, so KITTI’s larger gaps keep their real weight instead of being normalized down to HGE’s. XFeat+LighterGlue (production), M3R = MASt3R, M3D = Metric3D depth, M3Rd = MASt3R’s own depth. XLG+M3D ties M3R+M3D on accuracy (2.20 m each) at 21% less wall time. XLG+M3Rd is faster still but 0.07 m worse, mostly one 0.3 m KITTI gap. MASt3R-native is off this plot at 4.26 m, nearly twice everyone else’s.

MethodFront-endEdgesTrans. (m)Rot. (deg)
COVINS-G PnPVINS-Mono8520.69 ± 0.603.44 ± 3.5
COVINS-G 17-pointVINS-Mono18230.10 ± 0.171.09 ± 1.25
COVINS-G PnPORB-SLAM35050.23 ± 0.321.56 ± 3.7
COVINS-G 17-pointORB-SLAM34060.07 ± 0.121.01 ± 0.9
Ours, stereo-inertialORB-SLAM3430.48 ± 0.291.61 ± 0.96
Ours, stereoORB-SLAM3620.57 ± 0.412.31 ± 1.71

Table VI. EuRoC [13]. Accuracy of committed cross-session edges on MH01–05 before graph optimization, mean and standard deviation against ground truth, and COVINS-G values from their paper [33].

Retrieval

Retrieval is generally robust to substitution; swapping MegaLoc for BoQ, NetVLAD, or SALAD changes ATE only by a few percent.

Timing

Figure 8 compares wall time against 2GO on the same RTX 4090. Polygraph is 1.9× faster on KITTI-00 and 1.3× faster on ciampino_train0 in production, and 2.85× at speed, with far fewer edges: 26 and 51 (14 at speed) against 517 and 3403. Real-time factor, wall time divided by the recording’s own length, is 0.44 on KITTI-00 and 0.47 at speed on ciampino, both faster than the sensor itself, and 1.06 in production. This speedup can mostly be attributed to the consensus gate rejecting single, uncorroborated matches.

Figure 8. Runtime and committed edges against 2GO. Ran on the same RTX 4090, VINS-Fusion odometry, log scale. 2GO values are from their paper; ours is the mean of five runs. ciampino’s speed config trims top-k clusters/min-kf pairs vs. production.

Figure 7’s pose-estimation ablation gives the same picture, but differently. Each variant runs once per dataset, over every session it has, timed end-to-end. Dividing that wall time by the dataset’s longest single session gives a real-time factor against just watching its longest recording once. Averaged across KITTI, HGE, HGE iOS, and HYDRO, this is 0.61 for production, 0.70 for MASt3R with Metric3D depth, 0.65 for LoFTR with Metric3D depth, and 0.49 for MASt3R-native, which skips the depth-lifting stage entirely and is fastest here for exactly that reason.

Odometry Consistency

Our pipeline is deterministic; however, odometry might not be. Therefore, we run our method end-to-end on five different realizations of ORB-SLAM3 odometry, yielding Total L2L2 between 5.97 and 6.63 m, close to Table I’s single realization, confirming the robustness of our approach.

Gate. The two-tier gate is worth ablating directly. In Table VII, we ablate admitting every confirmed match immediately with no corroboration or consistency check. Gate-off runs 5.7 to 10.3% slower, since committing far more edges costs more downstream cleanup than the gate itself saves. As discussed above, the gate not only achieves speedup but also increases accuracy, with per-edge translation error 2.5×2.5 \times higher on average and rotation error almost 6×6 \times higher. In short, the gate guards against single bad edges that are hard to filter out in the optimization and therefore increases robustness and accuracy.

DatasetSubmitted OnSubmitted OffTrans err (m) OnTrans err (m) OffRot err (deg) OnRot err (deg) OffExec Δ\Delta
HGE2503730.3992.5740.8712.41+7.0%
HYDRO7758390.7261.2230.901.78+8.6%
Total102512120.6461.6390.895.05+8.2%

Table VII. Two-tier consensus gate ablation. Six HGE/HYDRO multi-session conditions pooled by dataset. Submitted are distinct cross-session candidates the gate admitted; trans/rot err is per committed edge before graph optimization. Exec Δ\Delta is gate-off vs. on wall time, run on an RTX 2080, not comparable in absolute terms to the pinned-4090 numbers elsewhere. Gate-on shaded and bold.

Conclusion

Polygraph merges sessions online, at cluster level, using only image retrieval, two-view matching, and a monocular depth prior, without ever building or maintaining a map. Across outdoor driving, indoor multi-device capture, and a micro-aerial benchmark, it stays competitive with existing multi-session baselines while committing far fewer edges and running faster than the closest prior work. The same production config transfers across KITTI’s cars, HYDRO’s legged robot and phone, and HGE’s headset and phone, without retuning per device. Future work includes closing the edge-count gap without giving up the accuracy it buys, and testing beyond visual and visual-inertial odometry.

Acknowledgements

The author(s) used generative AI to assist with drafting and editing the manuscript, cleaning up figures and coding. All content was reviewed by the authors, who take full responsibility for it.

References

  1. [1]R. Elvira, J. D. Tardós, and J. M. M. Montiel, “ORBSLAM-Atlas: A robust and accurate multi-map system,” arXiv:1908.11585, 2019.arxiv.org/abs/1908.11585
  2. [2]T. Schneider, M. Dmyt­czyk, M. Fehr, K. Egger, S. Lynen, I. Gilitschenski et al., “maplab: An open framework for research in visual-inertial mapping and localization,” IEEE RA-L, 2018.DOI
  3. [3]A. Cramaric, L. Bernreiter, F. Tschopp, M. Fehr, V. Reijgwart, J. Nieto et al., “maplab 2.0 – a modular and multi-modal mapping framework,” IEEE RA-L, 2023.
  4. [4]P. Schmuck, T. Ziegler, M. Karrer, J. Perraudin, and M. Chli, “COVINS: Visual-inertial SLAM for centralized collaboration,” arXiv:2108.05756, 2021.arxiv.org/abs/2108.05756
  5. [5]P.-Y. Lajoie, B. Ram­ toula, Y. Chang, L. Carlone, and G. Beltrame, “DOOR-SLAM: Distributed, online, and outlier resilient SLAM for robotic teams,” IEEE RA-L, 2020.
  6. [6]P.-Y. Lajoie and G. Beltrame, “Swarm-SLAM: Sparse decentralized collaborative simultaneous localization and mapping framework for multi-robot systems,” IEEE RA-L, 2024.DOI
  7. [7]P.-Y. Lajoie, B. Ram­toula, D. De Martini, and G. Beltrame, “FLOCC-SLAM: 3D foundation model-based loop closing for decentralized collaborative SLAM,” IEEE RA-L, 2025.DOI
  8. [8]T. Y. Lim, B. Sun, M. Pollefeys, and H. Blum, “Loop closure from two views: Revisiting PGO for scalable trajectory estimation through monocular priors,” arXiv:2503.16275, 2025.arxiv.org/abs/2503.16275
  9. [9]A. Geiger, P. Lenz, and R. Urtasun, “Are we ready for autonomous driving? The KITTI vision benchmark suite,” in CVPR, 2012.DOI
  10. [10]P.-E. Sarlin et al., “LaMAR: Benchmarking localization and mapping for augmented reality,” in ECCV, 2022.arxiv.org/abs/2210.10770
  11. [11]H. Blum, A. Mercurio, J. O’Reilly, T. Engelbracht, M. Dusmanu, M. Pollefeys, and Z. Bauer, “CroCoDOL: Cross-device collaborative dataset for localization,” in CVPR, 2025.DOI
  12. [12]L. Brizi, G. Giacomini, L. Di Giammarino, S. Ferrari, O. Salem, L. De Re­bot­ti et al., “VBR: A vision benchmark in rome,” arXiv:2404.11322, 2024.arxiv.org/abs/2404.11322
  13. [13]M. Burri, J. Nikolic, P. Gohl, T. Schneider, J. Rehder, S. Omari et al., “The EuRoC micro aerial vehicle datasets,” IJRR, 2016.DOI
  14. [14]C. Campos, R. Elvira, J. J. G. Rodríguez, J. M. M. Montiel, and J. D. Tardós, “ORB-SLAM3: An accurate open-source library for visual, visual-inertial, and multi-map SLAM,” IEEE T-RO, 2021.DOI
  15. [15]T. Qin, J. Pan, S. Cao, and S. Shen, “A general optimization-based framework for local odometry estimation with multiple sensors,” arXiv:1901.03638, 2019.arxiv.org/abs/1901.03638
  16. [16]T. Qin, S. Cao, J. Pan, and S. Shen, “A general optimization-based framework for global pose estimation with multiple sensors,” arXiv:1901.03642, 2019.arxiv.org/abs/1901.03642
  17. [17]M. Kaess et al., “iSAM2: Incremental smoothing and mapping using the Bayes tree,” IJRR, 2012.DOI
  18. [18]R. Arandjelović, P. Gronat, A. Torii, T. Pajdla, and J. Sivic, “NetVLAD: CNN architecture for weakly supervised place recognition,” in CVPR, 2016.
  19. [19]S. Izquierdo and J. Civera, “Optimal transport aggregation for visual place recognition,” in CVPR, 2024.arxiv.org/abs/2311.15937
  20. [20]A. Ali-bey, B. Chaib-draa, and P. Giguère, “BoQ: A place is worth a bag of learnable queries,” in CVPR, 2024.
  21. [21]G. Berton and C. Masone, “MegaLoc: One retrieval to place them all,” in CVPRW, 2025.arxiv.org/abs/2502.17237
  22. [22]D. DeTone et al., “SuperPoint: Self-supervised interest point detection and description,” in CVPRW, 2018.arxiv.org/abs/1712.07629
  23. [23]P.-E. Sarlin, D. DeTone, T. Malisiewicz, and A. Rabi­novich, “SuperGlue: Learning feature matching with graph neural networks,” in CVPR, 2020.arxiv.org/abs/1911.11763
  24. [24]M. J. Tyszkiewicz, P. Fua, and E. Trulls, “DISK: Learning local features with policy gradient,” in NeurIPS, 2020.arxiv.org/abs/2006.13566
  25. [25]P. Lindenberger et al., “LightGlue: Local feature matching at light speed,” in ICCV, 2023.arxiv.org/abs/2306.13643
  26. [26]G. Potje et al., “XFeat: Accelerated features for lightweight image matching,” in CVPR, 2024.
  27. [27]J. Sun, Z. Shen, Y. Wang, H. Bao, and X. Zhou, “LoFTR: Detector-free local feature matching with transformers,” in CVPR, 2021.arxiv.org/abs/2104.00680
  28. [28]V. Leroy, Y. Cabon, and J. Revaud, “Grounding image matching in 3D with MAST3R,” in ECCV, 2024.arxiv.org/abs/2406.09756
  29. [29]M. Hu, W. Yin, C. Zhang, Z. Cai, X. Long, H. Chen et al., “Metric3d v2: A versatile monocular geometric foundation model for zero-shot metric depth and surface normal estimation,” IEEE TPAMI, 2024.DOI
  30. [30]Y. Tian, Y. Chang, E. Herrera Arias, C. Nieto-Granda, J. P. How, and L. Carlone, “Kimera-Multi: Robust, distributed, dense metric-semantic SLAM for multi-robot systems,” IEEE T-RO, 2022.arxiv.org/abs/2106.14386
  31. [31]J. G. M. Domonic, D. Dominic, R. M. Eustice, and R. Vasudevan, “Pairwise consistent measurement set maximization for robust multi-robot map merging,” in ICRA, 2018.DOI
  32. [32]H. Yang, P. Antonante, V. Tzoumas, and L. Carlone, “Graduated non-convexity for robust spatial perception: From non-minimal solvers to global outlier rejection,” IEEE RA-L, 2020.DOI
  33. [33]M. Patel et al., “COVINS-G: A generic back-end for collaborative visual-inertial SLAM,” in ICRA, 2023.

Paper details

vlnc:W_rabdun53
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
  • Computer Science

generative AI

View statement
Looking for related papers…
Contents