alphaFeedback

WorksComputer Science

Sparse Anchors for Visual Multi-Session Registration in Changing Environments

AI contributions
Not recorded for this paper
View details

Abstract

Repeated robotic monitoring requires localization across visits to the same environment, even if the environment changes. Such changes can substantially reduce persistent visual content, and repetitive structures can lead to incorrect cross-session correspondences. In this paper, we address visual multi-session registration of a new data recording (the query session) to a previously reconstructed reference map. Under substantial visual and structural changes, cross-session localization may be possible only for parts of the trajectory. In our approach, we therefore rely on a sparse set of reliable absolute pose constraints and combine them with within-session relative-pose information to register the complete trajectory to the reference map. We first estimate absolute alignment hypotheses through visual place recognition, local feature matching, and geometric verification. We then use sequence consensus to reject hypotheses that are inconsistent with a common query-to-reference alignment and retain the remaining poses as absolute anchors. These anchors are refined through featuremetric alignment against a Gaussian feature field and combined with within-session relative-pose constraints in a robust pose-graph optimization. We evaluate our approach on 15 cross-session experiments from the challenging HILTI26 construction-site dataset, covering substantial scene changes and difficult-to-match environments. Our method achieves 94.4% recall within 1m and 100.0% within 5m, outperforming independent image localization, which peaks at 65.3% and 70.7%, respectively, as well as direct registration of independently reconstructed Structure-from-Motion geometry. Our results show that sparse, reliable absolute anchors can provide accurate and complete trajectory registration even when reliable cross-session correspondences are available only for part of the query session.

Introduction

Most robotic systems operating repeatedly in the same environment must reliably localize in a previously built map despite scene changes. Reliable cross-session localization and registration are therefore important for long-term autonomy, repeated inspection and monitoring, as well as persistent mapping [13], [17], [37]. This becomes particularly challenging when changes are structural rather than purely appearance-level, since parts of the previously mapped geometry may no longer exist, while new structures may alter or occlude previously visible content.

Construction sites are a particularly demanding instance of this problem. Their geometry and appearance evolve as

walls, installations, temporary structures, and surface finishes are added, removed, or modified, while repetitive and low-texture structures create additional perceptual ambiguity [32], further compounded by the frequent lack of GNSS availability in indoor construction environments. Construction progress monitoring requires repeated observations to compare the current as-built state against earlier observations or the planned state [34]. Autonomous mobile robots can acquire such observations, but comparing visits requires reliable localization in a common coordinate frame.

In this work, we address visual multi-session registration in challenging, GNSS-denied, and changing environments. Given a previously reconstructed reference map, together with its posed reference observations, for example computed with any SLAM system, we seek to register the query session, a visual sequence acquired at a different time. The sequence of body poses associated with the noisy query

All authors except Olga Vysotska are with the University of Bonn, Center for Robotics. Olga Vysotska is with ETH Zurich and Cyrill Stachniss is additionally with the Lamarr Institute for Machine Learning and Artificial Intelligence.

This work has partially been funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) under Germany’s Excellence Strategy, EXC-2070 – 390732324 – PhenoRob, by the German Federal Ministry of Research, Technology and Space (BMFTR) under the Robotics Institute Germany (RIG), and by Innosuisse – Swiss Innovation Agency. observations forms the query trajectory, which we seek to recover in the reference map coordinate frame. This is difficult because substantial geometric and appearance changes reduce the persistent visual content available for cross-session correspondences, while repetitive structures can produce plausible correspondences between physically distinct locations, as shown in Figure 1.

Illustration of perceptual aliasing and localization under severe construction-site changes

Figure 1. Repetitive structures create perceptual aliasing between distant locations, while construction progress causes substantial appearance and geometric changes over time (top). Using sparse absolute anchors, featuremetric refinement, and sequence-level registration, our approach recovers poses with accurate image-to-map alignment even under severe scene changes (bottom).

Existing strategies often use image-based localization against an earlier map [27] or direct registration of independently reconstructed 3D maps. Image-based localization becomes unreliable when persistent visual content is limited or perceptually ambiguous, while 3D point-cloud registration degrades as spatial and temporal changes accumulate [32].

The main contribution of this paper is a visual multi-session registration approach for changing environments, where scene changes and perceptual aliasing can make reliable cross-session correspondences available only for parts of the trajectory. Our approach combines conservative cross-session anchor selection, retaining only absolute pose hypotheses that are geometrically verified and consistent with the query trajectory, with featuremetric pose refinement for precise image-to-map alignment. These sparse absolute anchors are then combined with within-session relative-pose constraints to recover the complete query trajectory even when reliable cross-session correspondences are only sparsely available.

We make three key claims: our approach (i) provides more accurate and complete multi-session registration than the baselines; (ii) improves the reliability of cross-session constraints under scene change and perceptual aliasing through conservative filtering and sequence consensus; and (iii) improves fine image-to-map alignment through featuremetric refinement of the verified absolute anchors. We will publicly release our implementation upon publication.

A common approach to visual multi-session registration is to localize observations from a later session against a previously reconstructed reference map. Visual place recognition (VPR) identifies corresponding places across sessions [5], [11], [15], [20]. Modern VPR methods range from learned aggregation approaches such as NetVLAD [2] to scalable and viewpoint-robust descriptors [1], [4] and representations based on large pretrained visual features [3], [14], [15]. Hierarchical Localization (HLoc) by Sarlin et al. [27] combines retrieval, local feature matching, and perspective-n-point (PnP)-based localization and forms our image-based localization baseline. However, such approaches rely on distinctive, persistent cross-session correspondences and often degrade under appearance changes and repetitive structures [29], [33]. In contrast, we use image-based localization to obtain sparse, reliable absolute anchors rather than localizing every query observation.

Leveraging multiple observations can improve robustness when individual cross-session correspondences are unreliable. Multi-sequence maps retain observations acquired under different appearance conditions [35], while sequence-based VPR exploits temporal context to resolve ambiguities

difficult from individual images [5], [20], [36]. Stenborg et al. [31] combine map-based localization with relative motion to bridge observations that cannot be localized independently, but assume that the query approximately follows the reference route. More generally, multi-map and multi-session SLAM systems exploit cross-session constraints across different traversals [7], and HILTI23 benchmarks multi-session SLAM in construction environments [22]. Our approach similarly exploits within-session motion, but uses sparse cross-session anchors to register trajectories without requiring the query path to follow the reference trajectory.

A different strategy is to reconstruct each session independently and register the resulting 3D maps. Global point-cloud registration methods address limited overlap and high outlier rates [21], [38], but most target dense LiDAR or RGB-D geometry rather than sparse and non-uniform Structure-from-Motion (SfM) reconstructions. ColabSfM [10] specifically addresses this domain gap, while construction-site studies show that temporal scene change further reduces persistent geometric overlap [32]. Rather than aligning independently reconstructed geometry, our method uses sparse image-to-map pose constraints together with the query trajectory.

Featuremetric localization refines camera poses by directly aligning learned features [28], and recent approaches extend this idea to differentiable Gaussian Splatting based scene representations [9], [25]. We apply featuremetric optimization only to cross-session poses that have passed geometric verification and sequence-level consistency checks. The refined poses then serve as sparse constraints for complete trajectory registration rather than as independent localization outputs.

Our Approach to Visual Multi-Session Registration

We address registration of a visual query session to a fixed reference map of the same environment built at a different time. In challenging environments such as construction sites, substantial geometric and appearance changes can limit reliable cross-session correspondences to only a subset of query observations. Our goal is to recover the complete query trajectory in the reference map frame without directly localizing every query observation.

Our approach consists of five stages, illustrated in Figure 2. First, we identify reliable cross-session retrieval hypotheses using visual place recognition and distinctiveness filtering (Sec. III-B). Second, we establish local feature correspondences and estimate absolute pose hypotheses using reference-map geometry and generalized PnP (Sec. III-C). Third, we use the query trajectory to retain hypotheses supporting a common query-to-reference alignment, forming sparse absolute anchors (Sec. III-D). Fourth, we refine these anchors through featuremetric alignment against the reference feature field (Sec. III-E). Finally, we combine the refined anchors with within-session relative-pose constraints in a robust pose-graph optimization (PGO) to recover the complete query trajectory (Sec. III-F).

Overview of the visual multi-session registration pipeline

Figure 2. Overview of our visual multi-session registration pipeline. We first select distinctive cross-session retrievals and establish local feature correspondences between query and reference observations. Generalized PnP with RANSAC uses the reference-map geometry to reject correspondences that do not support a geometrically consistent multi-camera pose, producing locally verified absolute pose hypotheses. Since a locally plausible pose may still correspond to an incorrect location, sequence consensus evaluates the hypotheses jointly with the within-session query trajectory and retains only spatially distributed hypotheses that support a common query-to-reference similarity alignment. The resulting sparse absolute anchors are refined by featuremetric alignment against the reference Gaussian feature field and finally combined with within-session relative-pose constraints and consensus-aligned priors in a pose-graph optimization to recover the complete query trajectory.

Preliminaries and Problem Formulation

We consider a fixed reference map and a query session acquired in the same environment at different times. Both comprise synchronized multi-camera observations from a calibrated system with NvN_v cameras. We index reference observations by time index tt and query observations by kk, with NqN_q query observations in total. Each observation has one body pose and one image from each camera vv. We denote reference and query images by It,vMI^M_{t,v} and Ik,vQI^Q_{k,v}, respectively.

Coordinate frames are written as FA\mathcal{F}_A, and TAB∈SE(3)T^{AB} \in SE(3) maps coordinates from FB\mathcal{F}_B to FA\mathcal{F}_A. Let FM\mathcal{F}_M be the reference map coordinate frame and FQ\mathcal{F}_Q the arbitrary query coordinate frame. The body frame at observation ii is FBi\mathcal{F}_{B_i}, and the frame of camera vv at that observation is FCi,v\mathcal{F}_{C_{i,v}}. The fixed camera-to-body extrinsic calibration satisfies TBiCi,v=TBCv∈SE(3)T^{B_iC_{i,v}} = T^{BC_v} \in SE(3) for all observations ii. The reference body poses TMBtT_{MB_t} are known, and the corresponding camera poses are

TMCt,v=TMBtTBCt,v.(1)T_{MC_{t,v}} = T_{MB_t} T_{BC_{t,v}}. \tag*{(1)}

We represent the reference map using Gaussian splatting [12], [16] as a set of Gaussian primitives encoding geometry, opacity, and appearance. We additionally associate each Gaussian with a learned feature vector f∈Rn\mathbf{f} \in\mathbb{R}^n and denote the resulting map by G\mathcal{G}. Given a camera pose TMCT_{MC}, we render depth and feature maps as

D=RD(G,TMC),F^=RF(G,TMC),(2)D = \mathcal{R}_D(\mathcal{G}, T_{MC}), \qquad\hat{F} = \mathcal{R}_F(\mathcal{G}, T_{MC}), \tag*{(2)}

where RD\mathcal{R}_D and RF\mathcal{R}_F are the depth and feature renderers [12], used for absolute pose estimation and featuremetric pose refinement, respectively.

The query session provides body poses TQBkT_{QB_k} in the arbitrary query coordinate frame FQ\mathcal{F}_Q. We assume that their within-session relative geometry is consistent, while their alignment with the reference map and, potentially, their metric scale are unknown. Our objective is to estimate

TQ∗={TMBk∗}k=1Nq,(3)T_Q^* = \left\{T^*_{MB_k}\right\}_{k=1}^{N_q}, \tag*{(3)}

such that the complete query trajectory is expressed in the reference map coordinate frame FM\mathcal{F}_M.

Reliable Cross-Session Retrieval

The first stage seeks to identify query observations for which a reliable cross-session retrieval hypothesis can be established. This is challenging in changing environments such as construction sites, where repetitive structures and low-texture regions can produce perceptually similar observations at physically distinct locations [32], [33]. Since an incorrect retrieval may lead to an erroneous absolute pose constraint, we deliberately favor precision over retrieval density.

For every camera image Ik,vQI^Q_{k,v} of query observation kk, a shared visual place recognition encoder produces a global descriptor dk,vQ∈Rm\mathbf{d}^Q_{k,v} \in\mathbb{R}^m. We aggregate these descriptors into one descriptor for the complete observation,

dkQ=∑v=1Nvdk,vQNv.(4)\mathbf{d}^Q_k = \frac{\sum_{v=1}^{N_v} \mathbf{d}^Q_{k,v}}{N_v}. \tag*{(4)}

We obtain reference descriptors dtM\mathbf{d}^M_t analogously. Given query observation kk and reference observation tt, we measure their visual similarity by

s(k,t)=dkQTdtM.(5)s(k,t) = \mathbf{d}^{Q\mathsf{T}}_k \mathbf{d}^M_t. \tag*{(5)}

The highest-scoring reference observation

t⋆=argmax⁡t s(k,t),(6)t^\star= \underset{t}{\operatorname{argmax}}\ s(k,t), \tag*{(6)}

defines the initial retrieval hypothesis.

The retrieval score s(k,t)s(k,t) alone does not indicate whether the selected location is visually distinctive from other parts of the environment. One approach is to use the margin between the best and second-ranked reference observations. For sequentially acquired data, however, neighboring reference observations often depict essentially the same place and consequently receive similarly high similarity scores. A small top-two margin therefore does not necessarily indicate genuine spatial ambiguity. We instead compare the best retrieval against the strongest spatially distinct alternative.

Let ptp_t denote the known position of reference observation tt in the reference map coordinate frame. For an exclusion radius rfarr_{\mathrm{far}}, we define the spatial margin as

mfar(k)=s(k,t⋆)−max⁡t∈Ds(k,t),D={t∣∥pt−pt⋆∥>rfar}.(7)\begin{aligned} m_{\mathrm{far}}(k) &= s(k,t^\star) - \max_{t \in\mathcal{D}} s(k,t),\\ \mathcal{D} &= \{t \mid\lVert p_t - p_{t^\star} \rVert> r_{\mathrm{far}}\}. \tag*{(7)} \end{aligned}

We accept retrievals with mfar(k)≥τfarm_{\mathrm{far}}(k) \ge\tau_{\mathrm{far}}. The exclusion radius prevents nearby observations of the same place from competing, so a large margin indicates preference over visually similar alternatives from spatially distinct map regions.

For each accepted query-reference pair, we use the per-camera descriptors to identify views likely to depict the same scene content. We compute pairwise descriptor similarities between query and reference views and select the one-to-one assignment maximizing total similarity using the Hungarian algorithm [19]. We then perform local feature matching only between the selected view pairs. We lift the resulting 2D-2D correspondences to 3D using reference map geometry and verify them with generalized PnP.

Geometric Verification with Generalized PnP

The preceding stage provides tentative 2D–2D correspondences that need not be geometrically consistent. We therefore lift the reference matches to 3D using the reference map geometry and estimate one shared multi-camera body pose through generalized PnP with RANSAC.

Using the known reference camera pose TMCt,vrT_{MC_{t,v_r}}, we render its depth and back-project the reference pixel to obtain the 3D point XnX_n in the reference map coordinate frame:

dn=RD(G,TMCt,vr)[unM],Xn=TMCt,vrπvr−1(unM,dn).(8)\begin{aligned} d_n &= \mathcal{R}_D(\mathcal{G}, T_{MC_{t,v_r}})[u_n^{M}],\\ X_n &= T_{MC_{t,v_r}}\pi_{v_r}^{-1}(u_n^{M},d_n). \tag*{(8)} \end{aligned}

Here, [u][u] denotes indexing the rendered depth map at pixel uu. Since rays from different cameras do not share a common optical center, let ovqo_{v_q} and bnb_n denote the camera center and corresponding unit bearing vector expressed in FBkF_{B_k}.

We seek the body pose TMBk=[RMBk∣tMBk]T_{MB_k} = [R_{MB_k} \mid t_{MB_k}] obeying:

Xn=RMBk(ovq+λnbn)+tMBk,(9)X_n = R_{MB_k}(o_{v_q} + \lambda_n b_n) + t_{MB_k}, \tag*{(9)}

where λn\lambda_n is the unknown ray depth.

We solve the generalized PnP problem jointly over all views using RANSAC [18]. RANSAC rejects inconsistent matches and returns an absolute pose hypothesis TMBkPnPT_{MB_k}^{\mathrm{PnP}} from the inliers. These locally verified hypotheses are then evaluated for consistency with the query trajectory.

Sequence Consensus

Generalized PnP locally verifies each absolute pose hypothesis TMBkPnPT_{MB_k}^{\mathrm{PnP}}, but a geometrically plausible pose may still correspond to an incorrect place. We therefore use the consistency of the query trajectory to identify hypotheses that jointly support a common query-to-reference alignment. For each candidate observation kk, we have its pose TQBkT_{QB_k} in the query coordinate frame and its verified pose hypothesis TMBkPnPT_{MB_k}^{\mathrm{PnP}} in the reference map coordinate frame.

We seek a similarity transformation with scale

SMQ=[sRMQ∣tMQ],s>0,(10)S_{MQ} = [sR_{MQ} \mid t_{MQ}], \qquad s > 0, \tag*{(10)}

which aligns the query trajectory with the reference map while remaining consistent with the absolute pose hypotheses.

This transformation predicts the reference-frame pose of query observation kk as

T~MBk=SMQTQBk.(11)\widetilde{T}_{MB_k} = S_{MQ}T_{QB_k}. \tag*{(11)}

A pose hypothesis TMBkPnPT_{MB_k}^{\mathrm{PnP}} supports the transformation when its position and orientation agree with T~MBk\widetilde{T}_{MB_k}.

To prevent dense neighboring observations from dominating the alignment, we partition the query trajectory into spatial regions and favor support across distinct regions. We generate candidate transformations from spatially separated pose pairs. Each proposal estimates rotation from the corresponding query-to-reference orientations, and translation and, when required, scale from the associated body centers. We evaluate each proposal against all pose hypotheses using translation and rotation residuals.

We select the proposal with the strongest spatially distributed support and refit SMQS_{MQ} using its inliers. During refitting, each spatial region receives equal total weight, preventing densely sampled portions from dominating the estimate. We recompute support and require the alignment to remain stable as supported regions are removed. If no sufficiently distributed and stable consensus exists, we reject the session rather than forcing a global registration.

The resulting SMQS_{MQ} defines the consensus alignment between the query trajectory and reference map. Its inliers form the absolute anchor set A\mathcal{A} passed to featuremetric refinement. For each k∈Ak \in\mathcal{A}, TMBkPnPT_{MB_k}^{\mathrm{PnP}} provides the initial anchor pose.

Featuremetric Pose Refinement

Sequence consensus provides sparse absolute anchors supported by local geometric verification and consistency with the query trajectory. Small residual pose errors can still cause noticeable image-to-map misalignment. We therefore refine each absolute anchor by aligning the query feature maps with the reference feature field. Since we refine only anchors retained by sequence consensus, we assume their views contain sufficient persistent scene content for reliable alignment. The learned features additionally provide robustness to appearance and texture changes between sessions.

Methodt-RMSE<5 m_{<5\ \mathrm{m}} [m] ↓\downarrowr-RMSE<5 m_{<5\ \mathrm{m}} [∘^\circ] ↓\downarrowR@0.5m↑[email protected]\mathrm{m}\uparrowR@1m↑R@1\mathrm{m}\uparrowR@5m↑R@5\mathrm{m}\uparrow
FPFH [26] + ICP [6]1.79336.5111.22%34.59%60.95%
Register Any Point [24]2.31469.501.34%6.69%19.38%
ColabSfM [10]2.25442.515.46%18.03%46.47%
HLoc [27]0.94415.2145.5%65.3%70.7%
Ours0.6001.20462.5%94.4%100.0%

Table 1. Aggregate multi-session registration performance over 15 cross-session experiments. Translation and rotation RMSE are computed only for poses with translation error below 5 m, whereas recall is evaluated over all query observations. All metrics are micro-averaged over query observations.

Let ξ∈R6\boldsymbol{\xi} \in\mathbb{R}^{6} denote a local pose update, with (⋅)∧:R6→se(3)(\cdot)^\wedge:\mathbb{R}^{6}\to\mathfrak{se}(3) mapping it to the Lie algebra of SE(3)SE(3). We parameterize the refined body pose as

TMBk(ξ)=Exp⁡(ξ∧)TMBkPnP.(12)T_{MB_k}(\boldsymbol{\xi})=\operatorname{Exp}(\boldsymbol{\xi}^{\wedge})T^{\mathrm{PnP}}_{MB_k}. \tag*{(12)}

For camera vv, the pose and rendered feature map are

TMCk,v(ξ)=TMBk(ξ)TBkCk,v,T_{MC_{k,v}}(\boldsymbol{\xi})=T_{MB_k}(\boldsymbol{\xi})T_{B_kC_{k,v}},
F^k,v(ξ)=RF(G,TMCk,v(ξ)).(13)\widehat{F}_{k,v}(\boldsymbol{\xi})=\mathcal{R}_{F}(\mathcal{G},T_{MC_{k,v}}(\boldsymbol{\xi})). \tag*{(13)}

Let Φ\Phi denote the feature encoder used to construct the reference feature field and extract query features Fk,vQ=Φ(Ik,vQ)F^{Q}_{k,v}=\Phi(I^{Q}_{k,v}). We optimize a shared body pose across cameras by minimizing the query-rendered feature discrepancy:

ξ⋆=arg⁡min⁡ξ∑v=1Nv1∣Ωv∣∑u∈Ωv∥Fk,vQ[u]−F^k,v(ξ)[u]∥1.(14)\boldsymbol{\xi}^{\star}=\underset{\boldsymbol{\xi}}{\arg\min}\sum_{v=1}^{N_v}\frac{1}{\lvert\Omega_v\rvert}\sum_{u\in\Omega_v}\left\lVert F^{Q}_{k,v}[u]-\widehat{F}_{k,v}(\boldsymbol{\xi})[u]\right\rVert_{1}. \tag*{(14)}

Here, Ωv\Omega_v denotes the valid rendered pixels for camera vv. Since the Gaussian renderer is differentiable with respect to pose, we minimize Eq. (14) with G\mathcal{G} fixed, optimizing only the six-degree-of-freedom body pose. The resulting TMBk(ξ⋆)T_{MB_k}(\boldsymbol{\xi}^{\star}) defines the refined anchor for k∈Ak\in\mathcal{A} and is used in the subsequent pose-graph optimization.

Pose-Graph Optimization

The preceding stages provide refined absolute anchors TMBk(ξ⋆)T_{MB_k}(\boldsymbol{\xi}^{\star}) and the consensus alignment SMQS_{MQ}. We initialize the query poses with the consensus-aligned poses T~MBk\widetilde{T}_{MB_k} from Eq. (11) and refine them using PGO.

Let E\mathcal{E} denote the available within-session relative-pose constraints, with measurements

Zkl=TQBk−1TQBl,(k,l)∈E.(15)Z_{kl}=T^{-1}_{QB_k}T_{QB_l},\qquad(k,l)\in\mathcal{E}. \tag*{(15)}

For trajectories with unknown metric scale, we apply the scale estimated during sequence consensus to the relative-pose translations and keep it fixed during optimization.

For each anchor k∈Ak\in\mathcal{A}, let Ak=TMBk(ξ⋆)A_k=T_{MB_k}(\boldsymbol{\xi}^{\star}) denote its refined absolute pose. We optimize the reference-frame body poses {TMBk}\{T_{MB_k}\} using relative-pose measurements ZklZ_{kl}, refined anchors AkA_k, and consensus-aligned pose priors T~MBk\widetilde{T}_{MB_k}:

{TMBk⋆}=arg⁡min⁡{TMBk} ∑k∈Aρ(∥Log⁡(Ak−1TMBk)∥ΛA2)+∑(k,l)∈Eρ(∥Log⁡(Zkl−1TMBk−1TMBl)∥ΛR2)+∑kρ(∥Log⁡(T~MBk−1TMBk)∥ΛP2).(16)\begin{aligned} \{T^{\star}_{MB_k}\}&=\underset{\{T_{MB_k}\}}{\arg\min}\ \sum_{k\in\mathcal{A}}\rho\left(\left\lVert\operatorname{Log}\left(A_k^{-1}T_{MB_k}\right)\right\rVert_{\Lambda_A}^{2}\right)\\ &\quad+\sum_{(k,l)\in\mathcal{E}}\rho\left(\left\lVert\operatorname{Log}\left(Z_{kl}^{-1}T_{MB_k}^{-1}T_{MB_l}\right)\right\rVert_{\Lambda_R}^{2}\right)\\ &\quad+\sum_k\rho\left(\left\lVert\operatorname{Log}\left(\widetilde{T}_{MB_k}^{-1}T_{MB_k}\right)\right\rVert_{\Lambda_P}^{2}\right). \tag*{(16)} \end{aligned}

Here, ρ\rho is a robust loss, while the information matrices ΛA\Lambda_A, ΛR\Lambda_R, and ΛP\Lambda_P weight the anchor, relative-pose, and prior residuals. The anchor and relative-pose factors constrain the trajectory to the refined poses while preserving local motion, and the priors prevent poorly constrained regions from drifting from the consensus alignment. The optimized poses define TQ⋆T^{\star}_{Q} from Eq. (3).

Implementation Details

We use MegaLoc [3] for visual place recognition, LoMa-B [23] for local feature matching, and DINOv3 ViT-S/16 [30] for featuremetric refinement. MegaLoc descriptors are extracted from 322×322322\times322 images. For the spatial far-margin, we use rfar=10 mr_{\mathrm{far}}=10\,\mathrm{m} and accept retrievals with mfar≥0.01m_{\mathrm{far}}\geq0.01. For each accepted retrieval, up to eight one-to-one query-reference view pairs are matched with LoMa-B using at most 4096 keypoints per image. Generalized PnP with RANSAC uses a 12 px reprojection threshold, confidence 0.999, at most 1000 iterations, and requires at least 20 inliers.

For sequence consensus, spatial cells have width 5% of the query trajectory bounding-box diagonal. Translation and rotation inlier thresholds are 15% in normalized trajectory coordinates and 15∘15^\circ, respectively, and a valid consensus requires at least six anchors across three spatial cells. Featuremetric refinement optimizes one shared SE(3)SE(3) body-pose update over all valid camera views using Adam for 32 steps with learning rate 10−310^{-3}. Pose corrections are limited to 0.03 in normalized translation and 3∘3^\circ in rotation.

The final pose graph uses a soft-L1L_1 loss. Translation standard deviations for relative-pose, absolute-anchor, and trajectory-prior factors are 0.005, 0.03, and 0.05, with corresponding rotation standard deviations of 0.5∘0.5^\circ, 2∘2^\circ, and 3∘3^\circ. Scale recovered by sequence consensus is fixed before PGO.

Experimental Evaluation

We evaluate our approach for registering visual sessions acquired at different times to an earlier reference map under substantial scene change and perceptual ambiguity. Specifically, we evaluate three claims: (i) our approach provides more accurate and complete multi-session registration than the baselines; (ii) conservative cross-session filtering and anchor selection improve the reliability of absolute pose constraints under scene change and perceptual aliasing; and (iii) featuremetric refinement improves the fine image-to-map alignment of verified absolute anchors.

Experimental Setup

We use the HILTI26 [8] construction-site dataset, where data were collected by human operators and ground truth was generated using LiDAR-Inertial SLAM. We construct 15 cross-session registration experiments using seven reference sessions. Unlike HILTI23 [22], whose multi-session recordings span only several days, HILTI26 follows an active construction project over eight months and therefore captures substantially larger structural changes between sessions. We exclude floor 5 because it has only one recording, and UG2, floor 7, and the second runs of floor 2 and UG1 due to SfM failures to generate even a single consistent map. For each remaining environment, we use the earliest recording as the reference session and register all later recordings against it, evaluating registration against an earlier map as the construction site evolves.

Configurationt-RMSE [m]r-RMSE [°][email protected]
Raw PnP hypotheses12.39868.14425.7%
PnP + PGO0.8622.29745.2%
PnP + Consensus alignment0.6811.33449.9%
PnP + Cons. + PGO0.6081.21860.7%
PnP + Cons. + Featuremetric + PGO0.6001.20462.5%

Table 2. Ablation of sequence consensus, pose-graph optimization, and featuremetric refinement. Unlike Tab. I, RMSE is computed without an error gate.

For each reference session, we reconstruct the scene using SfM and train a 3D Gaussian Splatting map [16] from the resulting images and poses. Within-session relative-pose constraints for each query session are obtained from an independent SfM reconstruction. Only its relative geometry is provided; its alignment and metric scale with respect to the reference map must be recovered during registration. All experiments run on an NVIDIA RTX A5000 GPU, Intel Xeon W-1290P CPU, and 64 GiB of RAM.

Multi-Session Registration

The first experiment evaluates the accuracy and completeness of our multi-session registration and compares its performance against several baselines. Its results directly support our first claim that our approach provides more accurate and complete registration than existing alternatives.

We compare against HLoc [27], using the same MegaLoc retrieval, LoMa-B matching, and generalized multi-camera PnP configuration as our localization front end to ensure a fair comparison. HLoc localizes each query observation independently against the reference map and does not exploit within-session relative-pose constraints.

As a complementary geometry-based baseline, we independently reconstruct the reference and query sessions and register their sparse SfM point clouds using FPFH-based [26] RANSAC followed by ICP [6]. Since their relative scale is unknown, we provide this baseline with the ground-truth scale, removing scale estimation as a source of failure. We additionally evaluate recent learned registration methods, CoLabSfM [10] and Register Any Point [24], on the same cross-session SfM reconstructions.

Tab. I reports the aggregate registration performance. Our approach achieves the highest recall at all thresholds, with R@0.5 m=62.5%[email protected]\ \mathrm{m}=62.5\%, R@1 m=94.4%R@1\ \mathrm{m}=94.4\%, and R@5 m=100.0%R@5\ \mathrm{m}=100.0\%. Among poses with translation error below 5 m, it also achieves the lowest translation and rotation RMSE, with 0.600 m and 1.204∘1.204^\circ, respectively.

HLoc performs well for observations that can be localized independently, but suffers from catastrophic failures when reliable cross-session correspondences are unavailable or ambiguous. Direct registration of independently reconstructed SfM point clouds is considerably less reliable. Register Any Point is designed as a general-purpose point-cloud registration method rather than specifically for sparse SfM geometry or temporally changing scenes [24], while CoLabSfM is trained on trajectories observing the same underlying scene [10]. Both therefore face a substantial domain shift under the structural changes present in our setting. FPFH-based RANSAC followed by ICP is more robust than the learned registration methods, but still remains well below our image-based registration approach.

These results support our first claim: sparse, reliable cross-session constraints combined with within-session trajectory information provide more complete and accurate registration than the evaluated alternatives.

Reliability of Cross-Session Constraints

The second experiment evaluates the reliability of the cross-session pose constraints produced by our retrieval and geometric verification pipeline. We first isolate the VPR distinctiveness criterion, then test whether sequence consensus rejects locally plausible but globally inconsistent hypotheses.

a) Visual Distinctiveness: We compare the proposed spatial far-margin against the top-1 retrieval score and conventional top-1–top-2 margin. Figure 3 shows the resulting precision-recall trade-off, where a retrieval is correct if the retrieved reference observation lies within τVPR=2 m\tau_{\mathrm{VPR}}=2\ \mathrm{m} of the ground-truth query position.

Precision–recall trade-off of VPR confidence criteria

Figure 3. Precision–recall trade-off of VPR confidence criteria. Retrievals are considered correct if the retrieved reference observation lies within 2 m of the ground-truth position. Our spatial far-margin achieves substantially higher precision across a broad range of recall than the top-1 retrieval score and the top-1–top-2 margin.

The spatial far-margin achieves higher precision over a broad range of recall. Unlike the conventional top-two margin, it does not penalize neighboring reference observations depicting the same physical location. Instead, it measures whether the selected location remains distinctive from spatially separated alternatives. This matters in repetitive construction environments, where perceptually similar observations may originate from distant site regions.

b) Sequence Consensus: We then evaluate whether sequence consensus rejects absolute pose hypotheses that are locally plausible but inconsistent with the query trajectory. Tab. II compares raw generalized PnP hypotheses, PGO without sequence consensus, consensus alignment, and the subsequent refinement stages. Raw PnP hypotheses are available only for directly localized observations, whereas the remaining configurations estimate the complete query trajectory. Recall is evaluated over all query observations, and RMSE is computed without the 5 m gate used in Tab. I.

Raw generalized-PnP hypotheses contain severe outliers, with a translation RMSE of 12.4 m, rotation RMSE of 68.1∘68.1^\circ, and R@0.5 m=25.7%[email protected]\,\mathrm{m}=25.7\%. Directly combining these hypotheses with within-session relative-pose constraints using PGO substantially reduces the error, but remains less reliable, with 0.862 m translation RMSE, 2.3∘2.3^\circ rotation RMSE, and R@0.5 m=45.2%[email protected]\,\mathrm{m}=45.2\%. Thus, PGO alone cannot fully compensate for incorrect absolute constraints.

Sequence consensus further improves registration reliability. Consensus alignment alone reduces translation and rotation RMSE to 0.681 m and 1.3∘1.3^\circ, while increasing R@0.5 m[email protected]\,\mathrm{m} to 49.9%. Adding PGO further improves R@0.5 m[email protected]\,\mathrm{m} to 60.7% and translation RMSE to 0.608 m.

These results support our second claim that filtering absolute pose hypotheses before trajectory optimization produces more reliable cross-session constraints than jointly optimizing all locally verified hypotheses.

Featuremetric Pose Refinement

Finally, the last experiment evaluates whether featuremetric refinement improves the fine image-to-map alignment of the verified absolute anchors. Unlike the preceding filtering stages, featuremetric optimization is not intended to recover from incorrect place hypotheses or large pose errors. Instead, it operates only on the absolute anchors retained by sequence consensus and targets the remaining small image-to-map misalignments.

As shown in Tab. II, adding featuremetric refinement before PGO reduces the translation RMSE from 0.608 m to 0.600 m and also slightly reduces the rotation RMSE. Registration recall at the stricter threshold, R@0.5 m[email protected]\,\mathrm{m}, increases from 60.7% to 62.5%. This modest trajectory-level improvement is expected because refinement is applied only to already verified sparse anchors, whose pose corrections are subsequently propagated through the trajectory by PGO.

The effect of featuremetric refinement is more apparent at the image level. Figure 4 shows representative absolute anchors before and after optimization. The generalized-PnP poses already provide geometrically valid initializations, while featuremetric refinement improves the alignment of persistent structures between the real query observations and the reference Gaussian rendering despite scene changes. This is a clear support of our third claim.

Qualitative effect of featuremetric pose refinement

Figure 4. Qualitative effect of featuremetric pose refinement. Top row shows alignment after sequence consensus and PGO. Bottom row additionally uses featuremetric refinement, which corrects residual misalignment when present while preserving already well-aligned poses.

Conclusion

In this paper, we presented a novel, complete visual multi-session registration pipeline designed for challenging and changing environments such as construction sites, without requiring reliable cross-session correspondences along the entire query trajectory. Our method identifies a sparse set of strongly supported absolute anchors using visual distinctiveness, geometric verification, and sequence-level consistency, refines these anchors against a reference Gaussian feature field, and propagates their information through the query trajectory using within-session relative-pose constraints.

Our experimental evaluation on evolving construction environments shows that this strategy provides more complete and accurate trajectory registration than independent image localization and direct registration of independently reconstructed geometry. If no sufficiently distributed consensus is found, our method rejects the session rather than forcing an incorrect global alignment. The results further demonstrate the importance of conservative anchor selection and show that featuremetric refinement improves the fine image-to-map alignment of verified absolute anchors.

These properties make the proposed approach suitable for persistent mapping and repeated robotic operation under substantial structural change.

References

References

  1. [1]A. Ali-bey, B. Chaib-draa, and P. Giguère. MixVPR: Feature Mixing for Visual Place Recognition. In Proc. of the IEEE Winter Conf. on Applications of Computer Vision (WACV), 2023.
  2. [2]R. Arandjelovic, P. Gronat, A. Torii, T. Pajdla, and J. Sivic. Netvlad: Cnn architecture for weakly supervised place recognition. In Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2016.
  3. [3]G. Berton and C. Masone. MegaLoc: One Retrieval to Place Them All. In Proc. of the IEEE/CVF Conf. on Computer Vision and Pattern Recognition Workshops, 2025.arxiv.org/abs/2502.17237
  4. [4]G. Berton, C. Masone, and B. Caputo. Rethinking Visual Geo-Localization for Large-Scale Applications. In Proc. of the IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), 2022.arxiv.org/abs/2204.02287
  5. [5]G. Berton, G. Trivigno, B. Caputo, and C. Masone. JIST: Joint Image and Sequence Training for Sequential Visual Place Recognition. IEEE Robotics and Automation Letters (RA-L), 9(2):1310–1317, 2024.arxiv.org/abs/2403.19787
  6. [6]P.J. Besl and N.D. McKay. A Method for Registration of 3-D Shapes. IEEE Trans. on Pattern Analysis and Machine Intelligence (TPAMI), 14(2):239–256, 1992.DOI
  7. [7]C. Campos, R. Elvira, J.J.G. Rodríguez, J.M. Montiel, and J.D. Tardós. Orb-slam3: An accurate open-source library for visual, visual–inertial, and multimtap slam. IEEE Trans. on Robotics (TRO), 37(6):1874–1890, 2021.DOI
  8. [8]S. Centtany, Y. Zhang, Y. Tao, J. Kindle, F. Neuhaus, T. Koß, A. Patel, M. Helmberger, E. Szymańska, T. Gräber, and M. Fallon. Hilti-Trimble-Oxford Dataset: 360 Visual-Inertial Benchmark with Floor Plan Priors for SLAM and Localization. arXiv preprint, arXiv:2607.06464, 2026.
  9. [9]Y. Cheng, J. Jiao, Y. Wang, and D. Kanoulas. LoGS: Visual Localization via Gaussian Splatting with Fewer Training Images. In Proc. of the IEEE Intl. Conf. on Robotics & Automation (ICRA), 2025.arxiv.org/abs/2410.11505
  10. [10]J. Edstedt, A. Mateus, and A. Jaenal. Colabsfm: Collaborative structure-from-motion by point cloud registration. In Proc. of the IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), 2025.arxiv.org/abs/2503.17093
  11. [11]S. Hausler and P. Moghadam. Pair-VPR: Place-Aware Pre-Training and Contrastive Pair Classification for Visual Place Recognition With Vision Transformers. IEEE Robotics and Automation Letters (RA-L), 10(4):4013–4020, 2025.DOI
  12. [12]B. Huang, Z. Yu, A. Chen, A. Geiger, and S. Gao. 2D Gaussian Splatting for Geometrically Accurate Radiance Fields. In Proc. of the Intl. Conf. on Computer Graphics and Interactive Techniques (SIGGRAPH), 2024.DOI
  13. [13]M. Hutter, R. Diethelm, S. Bachmann, P. Fankhauser, C. Gehring, V. Tsounis, A. Lauber, F. Guenther, M. Bjelonic, L. Isler, H. Kolvenbach, K. Meyer, and M.A. Hoepflinger. Towards a Generic Solution for Inspection of Industrial Sites. In Field and Service Robotics, 2018.DOI
  14. [14]S. Izquierdo and J. Civera. Optimal Transport Aggregation for Visual Place Recognition. In Proc. of the IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), 2024.arxiv.org/abs/2311.15937
  15. [15]N.V. Keetha, A. Mishra, J. Karhade, K.M. Jatavallabhula, S. Scherer, M. Krishna, and S. Garg. AnyLoc: Towards Universal Visual Place Recognition. IEEE Robotics and Automation Letters (RA-L), 9(2):1286–1293, 2024.arxiv.org/abs/2308.00688
  16. [16]B. Kerbl, G. Kopanas, T. Leimkühler, and G. Drettakis. 3D Gaussian Splatting for Real-Time Radiance Field Rendering. ACM Trans. on Graphics (TOG), 42(4):1–14, 2023.arxiv.org/abs/2308.04079
  17. [17]A. Kim and R.M. Eustice. Real-Time Visual SLAM for Autonomous Underwater Hull Inspection Using Visual Saliency. IEEE Trans. on Robotics (TRO), 29(3):719–733, 2013.DOI
  18. [18]L. Kneip, P. Furgale, and R. Siegwart. Using multi-camera systems in robotics: Efficient solutions to the NPnP problem. In Proc. of the IEEE Intl. Conf. on Robotics & Automation (ICRA), 2013.DOI
  19. [19]H.W. Kuhn. The Hungarian Method for the Assignment Problem. Naval Research Logistics Quarterly, 2(1–2), 1955.
  20. [20]H. Li, G. Peng, J. Zhang, M. Wen, Y. Ma, and D. Wang. CaseVPR: Correlation-Aware Sequential Embedding for Sequence-to-Frame Visual Place Recognition. IEEE Robotics and Automation Letters (RA-L), 10(4):3430–3437, 2025.DOI
  21. [21]H. Lim, D. Kim, G. Shin, J. Shi, I. Vizzo, H. Myung, J. Park, and L. Carlone. KISS-Matcher: Fast and Robust Point Cloud Registration Revisited. In Proc. of the IEEE Intl. Conf. on Robotics & Automation (ICRA), 2025.arxiv.org/abs/2409.15615
  22. [22]A.D. Nair, J. Kindle, P. Levchev, and D. Scaramuzza. Hilti SLAM Challenge 2023: Benchmarking Single + Multi-Session SLAM Across Sensor Constellations in Construction. IEEE Robotics and Automation Letters (RA-L), 9(8):7286–7293, 2024.
  23. [23]D. Nordström, J. Edstedt, G. Böckman, J. Astermark, A. Heyden, V. Larsson, M. Wadenbäck, M. Felsberg, and F. Kahl. Loma: Local feature matching revisited. In Proc. of the Europ. Conf. on Computer Vision (ECCV), 2026.arxiv.org/abs/2604.04931
  24. [24]Y. Pan, T. Sun, L. Zhu, L. Nunes, I. Armeni, J. Behley, and C. Stachniss. Register Any Point: Scaling 3D Point Cloud Registration by Flow Matching. In Proc. of the Europ. Conf. on Computer Vision (ECCV), 2026.arxiv.org/abs/2512.01850
  25. [25]M. Pietrantoni, G. Csuka, and T. Sattler. Gaussian Splatting Feature Fields for (Privacy-Preserving) Visual Localization. In Proc. of the IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), 2025.
  26. [26]R. Rusn, N. Blodow, and M. Beetz. Fast point feature histograms (fpfh) for 3d registration. In Proc. of the IEEE Intl. Conf. on Robotics & Automation (ICRA), 2009.
  27. [27]P.E. Sarlin, C. Cadena, R. Siegwart, and M. Dymczyk. From Coarse to Fine: Robust Hierarchical Localization at Large Scale. In Proc. of the IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), 2019.arxiv.org/abs/1812.03506
  28. [28]P.E. Sarlin, A. Unagar, M. Larsson, H. Germain, C. Toft, V. Larsson, M. Pollefeys, V. Lepetit, L. Hammarstrand, F. Kahl, and T. Sattler. Back to the Feature: Learning Robust Camera Localization from Pixels to Pose. In Proc. of the IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), 2021.arxiv.org/abs/2103.09213
  29. [29]T. Sattler, W. Maddern, C. Toft, A. Torii, L. Hammarstrand, E. Stenborg, D. Safari, M. Okutomi, M. Pollefeys, J. Sivic, F. Kahl, and T. Pajdla. Benchmarking 6DOF Outdoor Visual Localization in Changing Conditions. In Proc. of the IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), 2018.arxiv.org/abs/1707.09092
  30. [30]O. Siméoni, H.V. Vo, M. Seitzer, F. Baldassarre, M. Oquab, C. Jose, V. Khalidov, M. Szafraniec, S. Yi, M. Ramamonjisoa, F. Massa, D. Haziza, L. Wehrstedt, J. Wang, T. Darcet, T. Moutakanni, L. Sentana, C. Roberts, A. Vedaldi, J. Tolan, J. Brandt, C. Couprie, J. Mairal, H. Jégou, P. Labatut, and P. Bojanowski. Dinov3. Trans. on Machine Learning Research (TMLR), 2026.
  31. [31]E. Stenborg, T. Sattler, and L. Hammarstrand. Using Image Sequences for Long-Term Visual Localization. In Proc. of the Intl. Conf. on 3D Vision (3DV), 2020.DOI
  32. [32]T. Sun, Y. Hao, S. Huang, S. Savarese, K. Schindler, M. Pollefeys, and I. Armeni. Nothing Stands Still: A Spatiotemporal Benchmark on 3D Point Cloud Registration Under Large Geometric and Temporal Change. ISPRS Journal of Photogrammetry and Remote Sensing, 220:799–823, 2025.arxiv.org/abs/2311.09346
  33. [33]A. Torii, J. Sivic, M. Okutomi, and T. Pajdla. Visual place recognition with repetitive structures. IEEE Trans. on Pattern Analysis and Machine Intelligence (TPAMI), 37(11):2346–2359, 2015.
  34. [34]S. Tuttas, A. Braun, A. Borrmann, and U. Stilla. Acquisition and Consecutive Registration of Photogrammetric Point Clouds for Construction Progress Monitoring Using a 4D BIM. In ISPRS Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2014.
  35. [35]O. Vytsotska and C. Stachniss. Effective Visual Place Recognition Using Multi-Sequence Maps. IEEE Robotics and Automation Letters (RA-L), 4(2):1730–1736, 2019.
  36. [36]O. Vytsotska, I. Bogoslavskyi, M. Hutter, and C. Stachniss. Adaptive Thresholding for Sequence-Based Place Recognition. In Proc. of the IEEE Intl. Conf. on Robotics & Automation (ICRA), 2025.
  37. [37]Y. Wang, M. Ramezani, and M. Fallon. Actively Mapping Industrial Structures with Information Gain-Based Planning on a Quadruped Robot. In Proc. of the IEEE Intl. Conf. on Robotics & Automation (ICRA), 2020.arxiv.org/abs/2002.09710
  38. [38]H. Yang, J. Shi, and L. Carlone. TEASER: Fast and Certifiable Point Cloud Registration. IEEE Trans. on Robotics (TRO), 37(2):314–333, 2021.arxiv.org/abs/2001.07715

Paper details

vlnc:W_zmmxay99
v1 · latestHistory (1)

published · Fresh — added to the Hub within the last 30 days

added ·

Usage

views · 7

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

CC BY 4.0

Original source

  • Computer Science

Not recorded for this paper · View statement →

Looking for related papers…
Contents