WorksComputer Science
Kolibri: Color Finds the Place, Refinement Sets the Pose for RGB-D Localization in Colored Point Clouds
Abstract
An agent or robot in a scanned building must locate its camera in rooms that no model saw during training. Point cloud registration methods can align each RGB-D frame with the building's colored point cloud, accurately but at a high cost. We observe that geometric refinement brings nearby initial poses to the same final pose, so the learned component only needs to find the place. Kolibri does this with a graph network of 55,712 parameters that describes 10 cm cubes of points, called cells, by their color, local shape and neighbors. With the same refinement, it matches GeDi on 96 unseen ScanNet rooms (81.4% against 80.3% within 5 cm and 5°) and on a held-out test of 10 further rooms, in 82 ms per frame on an RTX 2080 Ti. The saved time allows for search without a room label. In one point cloud of 100 rooms, Kolibri loses 1.3 points against knowing the room, while retrieving the room first loses 56.0. In a hospital emergency department scene from a real scan, it localizes 86.5% of the frames rendered from the scan in 384 ms each, where GeDi needs 91 minutes per frame on cubes of the scan for a similar recall. The method needs colors that agree between the frame and the point cloud. On gray frames and across cameras with different color responses, point cloud registration methods keep the lead.
Introduction
We scanned the emergency department of a university hospital on foot with a handheld laser scanner (BLK2GO) and obtained a colored point cloud of it. Robots and simulated pedestrian agents need to locate their cameras anywhere in it, without a room label or an initial position. Each view forms a RGB-D frame, with paired color with depth, so point cloud registration methods can align it with the point cloud. The difficulty is cost. On views rendered from the scan with simulated depth noise, GeDi [33] needs 91 minutes per frame to search overlapping cubes of the scan, while Kolibri, the method we present here, reaches a similar recall on the whole scan in 384 ms. The rendered colors agree with the point cloud by construction, and the baseline comparison uses 30 frames, one per walk (§ 4.5). The hospital scan motivated the method but remained untouched while we developed it on public benchmarks.
A descriptor summarizes the shape or color around a point as a vector, so that the same surface seen twice can be matched. Point cloud registration methods spend time computing these descriptors and searching for consistent matches. Even in a single room, BUFFER-X [39] takes a median 1.4 s per frame and GeDi 24 s on an RTX 2080 Ti when assigned to align overlapping fragments. At building scale, searching separate rooms or cubes repeats that work. Image retrieval can narrow the search, but choosing the wrong room loses the frame before pose estimation begins. Among 100 ScanNet rooms, retrieval ranks the true room first for only 28.6% of frames.
Our starting point is that finding the place and refining the pose need different levels of accuracy (Figure 2). Point-to-plane refinement adjusts a pose by pulling frame points onto nearby surfaces of the point cloud. It is a variant of iterative closest point (ICP) [3, 8]. Nearby initial poses often converge to the same result, within a region we call the refinement basin. On real ScanNet frames, Kolibri and BUFFER-X both start in this region for 62% of frames. Refinement gives the same success or failure outcome on 99% of those frames (Figure 4a). The learned component therefore needs to find candidate places close enough for refinement. Color helps: adding color to Fast Point Feature Histograms (FPFH) [36], a geometric descriptor, doubles recall at 5 cm with our pose estimator (Table 4).

Figure 2. The network finds candidate places, and geometry refines their poses. Point-to-plane iterative closest point (ICP) aligns frame points to nearby surfaces. The dashed region illustrates the refinement basin. Its measured boundary is the initial position error at which half the starts succeed within 5 cm and (Figure 4a). A geometric fit score selects the final candidate.
Figure 4. Refinement makes a small descriptor sufficient. (a) Success within 5 cm and on ScanNet-10 as a function of initial position error. The curve fits BUFFER-X and GeDi starts, and the dashed line marks the distance where half succeed. (b) Recall at the same threshold on the ablation rooms and median time as network size changes. Bars show paired intervals, and the band shows the equivalence margin. (c) Recall as the number of training rooms changes.
| Params | Fast ScanNet | Fast 7-Scenes | Precise ScanNet | Precise 7-Scenes | |
| Full network | 55.7k | 78.5 | 93.0 | 80.1 | 93.1 |
| One change to the network, retrained | |||||
| No message passing | 5.5k | −14.6 [−20.9, −8.5] | −24.4 [−32.5, −16.7] | −13.4 [−18.9, −8.4] | −22.7 [−31.1, −15.2] |
| Geometry only | 55.3k | −26.2 [−33.5, −19.3] | −72.7 [−79.4, −65.2] | −26.2 [−33.3, −19.1] | −72.0 [−78.5, −65.2] |
| Color only | 54.7k | −3.3 [−5.5, −1.1] | −0.8 [−1.6, 0.1] | −3.4 [−5.8, −1.4] | −0.9 [−1.6, −0.2] |
| Simple node features | 54.9k | −2.5 [−4.3, −1.1] | −0.5 [−1.1, +0.1] | −2.0 [−3.7, −0.5] | −0.3 [−1.1, +0.4] |
| Two compact layers | 28.4k | +0.2 [−0.9, +1.2] | −0.9 [−2.4, +0.6] | +0.8 [−0.9, +2.5] | −0.6 [−1.9, +0.3] |
| 22 training rooms | 55.7k | −0.2 [−1.3, +0.7] | −0.3 [−0.9, +0.2] | +0.2 [−1.1, +1.1] | −0.1 [−1.9, +0.3] |
| Another descriptor, same estimator and refinement | |||||
| GeDi | 3.11M | +0.9 [−1.5, +3.3] | −0.2 [−1.8, +1.3] | +0.9 [−1.1, +3.4] | −0.3 [−1.6, +1.1] |
| RGB GeoTransformer | 9.83M | +0.2 [−1.8, +2.6] | −2.5 [−5.1, −0.1] | +0.9 [−0.9, +2.5] | −2.8 [−5.4, −0.5] |
| FPFH | 0 | −48.3 [−58.5, −38.4] | −71.6 [−78.9, −64.1] | −48.3 [−57.9, −38.4] | −70.4 [−77.9, −62.7] |
| FPFH + color | 0 | −16.7 [−24.0, −10.3] | −19.6 [−26.4, −13.6] | −14.9 [−22.2, −8.3] | −18.7 [−25.0, −12.8] |
| Ours, untrained | 55.7k | −15.9 [−22.8, −9.0] | −21.9 [−27.6, −16.2] | −15.2 [−22.2, −8.4] | −20.2 [−25.7, −14.8] |
Table 4. Color and information from neighboring cells carry the result. The full network row gives recall (%) at 5 cm and on the ScanNet-10 ablation rooms and 7-Scenes. Other rows give changes in recall points, with paired intervals over walks. For bold, the full network counts as no change. Each network change uses three training seeds.
Kolibri uses a graph network of 55,712 parameters to describe cells, 10 cm cubes of pooled points, from their color, local shape and neighbors (Figure 3). The point cloud is described once and reused. Groups of consistent cell matches propose candidate poses, refinement adjusts them, and a geometric fit score selects the result. Color proposes the place and geometry decides, as color aids coarse scene recognition in humans [27].
Figure 3. Kolibri learns where to start and uses geometry to refine the pose. The frame and point cloud become graphs of colored cells. A shared network assigns each cell a descriptor, or vector for matching. Consistent matches propose candidate poses, point-to-plane refinement adjusts them, and a geometric fit score selects the result.
This division of work needs colors that agree between the frame and point cloud up to a change in brightness and contrast. On gray frames, RGB GeoTransformer, a point cloud registration method with color input, has 43 points higher recall than Kolibri at 5 cm. Point cloud registration methods also achieve higher recall in a laser scan colored by a different camera (§ 4.6).
Our contributions are:
A division of RGB-D localization into place finding and pose refinement, supported by measurements of when different initial poses give the same refinement outcome.
Kolibri, a small graph network that finds the place from color. With matched refinement, it is competitive with point cloud registration methods on unseen rooms and a held-out test, at a small fraction of their cost.
Search without a room label. Kolibri searches 100 rooms within 1.3 points of its room-given recall, and a hospital emergency department in 384 ms per frame.
Related Work
Point cloud registration. Point cloud registration methods combine descriptors with an estimator that rejects inconsistent matches [14, 46]. Descriptors progressed from handcrafted local geometry [36] to learned representations and matchers [1, 18, 33, 34, 48, 49], including methods that transfer across sensors and scales [39]. Colored ICP [31] and ColorPCR [25] use color during alignment. These methods start anew for each pair. Our reference point cloud is known in advance, so its descriptors can be reused. The layer exchanging features between neighboring cells, following EdgeConv [45], only needs to find starting poses for refinement.
Scene-specific relocalization. Scene coordinate regression trains a model for each scene to predict the scene position of each image pixel [4, 6, 40]. It is accurate on 7-Scenes [40] and 12-Scenes [43], where uncertainty in the reference poses limits measurable accuracy [5].
Relocalization in unseen scenes. Other methods predict scene coordinates or relative poses without training on the query scene. They use reference images with known poses or spatial coordinates [11, 35, 41, 47]. MeshLoc [30] matches against rendered mesh views, while general-purpose models of scene geometry match image pairs [22, 44]. Our depth input instead supports direct matching to an existing point cloud. MASt3R and Reloc3r, which relate the query to reference images, have lower recall than Kolibri when those images are rendered from the point cloud (§ 4.2).
Localization at building scale. A common approach retrieves likely places with an image descriptor [2, 28], then estimates the pose within them [37]. Place recognition from laser scans retrieves the matching region of a large point cloud [20, 42]. Image-to-point-cloud methods also use retrieval or restrict the search to regions near known poses [21, 23, 32]. Systems that estimate scene geometry and camera poses together [19, 26] solve a different task from localization in a prior scan. Kolibri searches all rooms together, without an earlier room decision that can discard the correct place.
Method
Task, settings and metric
The inputs are an RGB-D frame with known camera intrinsics and a colored metric point cloud known in advance. The output is the camera pose , a rigid rotation and translation that places the frame in . With the room given, contains only the frame’s room (Table 2 and Table 3). With the room unknown, it joins rooms or covers a hospital emergency department (§ 4.4 and § 4.5). Recall is the share of frames whose position and rotation errors both fall below a threshold pair: 5 cm and 5°, 0.25 m and 2°, or 0.5 m and 5°. A frame without a pose counts as a miss. No evaluation room or scan enters our training.
| ScanNet-96 (7,986 frames) | ScanNet-96-S2 (5,133 frames) | |||
| Fast | Precise | Fast | Precise | |
| Kolibri | 81.4 ±1.8 | 85.3 ±1.6 | 65.1 ±3.4 | 66.5 ±3.4 |
| + color pass | 81.6 ±1.8 | 85.5 ±1.6 | 65.5 ±3.4 | 67.0 ±3.3 |
| BUFFER-X | 69.4 ±2.8▲ | 72.4 ±2.8▲ | 61.5 ±3.7▲ | 62.7 ±3.6▲ |
| BUFFER-X-Lite | 70.4 ±2.7▲ | 73.3 ±2.7▲ | 61.9 ±3.7▲ | 63.0 ±3.6▲ |
| GeDi | 80.3 ±1.9≈▲ | 84.2 ±1.6≈▲ | 65.5 ±3.3≈ | 67.1 ±3.3≈ |
| RGB GeoTransformer | 77.8 ±1.9▲ | 81.4 ±1.8▲ | 64.0 ±3.3≈▲ | 65.7 ±3.4≈▲ |
| ColorPCR | 80.3 ±1.9≈▲ | 84.1 ±1.7≈▲ | 63.2 ±3.4≈▲ | 64.4 ±3.5≈▲ |
| RoITr | 75.2 ±2.2▲ | 78.4 ±2.1▲ | 64.1 ±3.4≈ | 65.7 ±3.3≈ |
| PREDATOR | 49.6 ±2.5▲ | 51.8 ±2.4▲ | 45.6 ±3.4▲ | 46.7 ±3.3▲ |
| FPFH + TEASER++ | 33.2 ±2.4▲ | 34.7 ±2.4▲ | 29.0 ±3.2▲ | 29.8 ±3.2▲ |
Table 2. Room-given test on ScanNet-96 and its later recording sessions. Recall (%) uses 5 cm and 5°. Here and in later tables, gray rows are Kolibri and bold marks the best value per column. Kolibri averages three training seeds. Baselines use one run and refinement matched to Fast or Precise. The ± entries give interval half-widths over walks. Marks compare Kolibri with one color reading against each cell, using the criteria in § 4.1: ≈ equivalent, ▲ Kolibri leads, ≈ ▲ both.
| Method | Parameters | in training | median ms | p95 ms | GB | ScanNet-10 5 cm | ScanNet-10 0.5 m | 7-Scenes 5 cm | 7-Scenes 0.5 m | 12-Scenes 5 cm | 12-Scenes 0.5 m | ScanNet-10-S2 5 cm | ScanNet-10-S2 0.5 m |
| Kolibri Fast | 55.7k | none | 81.7 | 251.1 | 1.44 | 75.7±6.5 | 94.9±2.5 | 93.0±2.5 | 98.5±0.9 | 97.9±3.1 | 99.7±0.6 | 73.3±7.4 | 95.3±3.3 |
| Kolibri Precise | 55.7k | none | 116.4 | 417.4 | 1.44 | 79.2±5.9 | 95.1±2.5 | 93.1±2.7 | 98.6±0.9 | 98.6±2.0 | 99.7±0.6 | 75.9±7.3 | 95.4±3.4 |
| BUFFER-X [39] | 0.91M | 7S, 12S | 1,416 | 1,922 | 6.21 | 66.4±6.7 | 91.3±3.8 | 72.7±7.8 | 96.2±2.5 | 82.1±7.8 | 91.8±5.0 | 66.9±8.6 | 95.2±2.9 |
| + refinement | 1,444 | 1,948 | 6.21 | 74.7±6.9 | 92.9±3.6 | 92.2±3.6 | 97.5±2.3 | 91.8±4.7 | 93.4±4.4 | 73.8±7.3 | 97.1±2.0 | ||
| BUFFER-X-Lite [39] | 0.91M | 7S, 12S | 1,280 | 1,896 | 6.21 | 63.9±6.7 | 90.7±4.2 | 66.6±7.3 | 96.4±2.5 | 81.6±7.9 | 91.7±5.6 | 62.1±9.6 | 95.0±2.9 |
| + refinement | 1,299 | 1,925 | 6.21 | 74.4±6.8 | 92.9±3.5 | 92.3±3.5 | 97.5±2.3 | 91.7±4.7 | 93.4±4.5 | 73.5±7.0 | 96.8±2.2 | ||
| GeDi [33] | 3.11M | 7S, 12S | 24,149 | 43,869 | 5.27 | 78.8±5.8 | 92.8±3.5 | 70.8±9.2 | 90.8±3.9 | 98.8±0.9 | 99.7±0.4 | 74.4±6.8 | 95.9±2.5 |
| + refinement | 24,179 | 43,902 | 5.27 | 76.2±7.1 | 95.1±2.6 | 90.9±3.8 | 95.8±2.4 | 98.1±2.9 | 99.7±0.6 | 73.6±7.1 | 97.5±1.7 | ||
| RGB GeoTransformer [34] | 9.83M | 7S, 12S | 2,501 | 6,338 | 2.70 | 76.3±5.5 | 88.2±4.4 | 66.1±9.8 | 92.4±3.2 | 99.1±0.8 | 99.8±0.2 | 72.6±7.2 | 93.0±3.4 |
| + refinement | 2,530 | 6,372 | 2.70 | 73.5±7.4 | 89.4±4.0 | 89.8±4.3 | 94.9±2.9 | 98.1±2.9 | 99.7±0.4 | 72.3±7.1 | 94.3±2.9 | ||
| ColorPCR [25] | 10.03M | 7S | 4,077 | 10,878 | 3.90 | 66.4±6.8 | 89.1±4.5 | 63.5±8.8 | 89.1±4.8 | 95.5±2.3 | 98.4±1.0 | 58.6±9.8 | 81.7±7.6 |
| + refinement | 4,107 | 10,913 | 3.90 | 72.6±7.0 | 91.4±3.6 | 86.3±5.0 | 92.4±4.0 | 97.2±2.9 | 98.8±0.9 | 67.2±8.7 | 86.1±6.7 | ||
| RoITr [48] | 10.10M | 7S, 12S | 6,402 | 22,324 | 2.07 | 60.2±6.9 | 86.3±4.5 | 55.8±13.3 | 81.2±10.6 | 77.2±8.9 | 92.5±5.8 | 61.5±6.9 | 90.6±4.0 |
| + refinement | 6,429 | 22,353 | 2.07 | 72.1±7.3 | 89.2±4.0 | 81.4±9.1 | 86.1±9.0 | 92.9±6.8 | 94.4±4.9 | 71.4±7.3 | 93.6±3.5 | ||
| PREDATOR [18] | 7.43M | 7S, 12S | 728.1 | 1,287 | 3.88 | 13.4±4.3 | 38.2±6.9 | 14.2±8.1 | 33.7±12.2 | 30.4±8.8 | 52.9±10.6 | 20.6±6.6 | 46.9±7.7 |
| + refinement | 754.0 | 1,316 | 3.88 | 47.4±7.6 | 53.4±7.0 | 47.0±13.3 | 51.4±12.8 | 72.5±8.5 | 74.4±9.0 | 53.1±7.9 | 67.6±6.6 | ||
| FPFH + TEASER++ [36, 46] | none | none | 215.0 | 648.5 | – | 11.0±3.9 | 24.9±6.7 | 2.6±1.5 | 10.8±4.5 | 8.0±2.9 | 16.3±5.0 | 9.5±3.4 | 29.6±8.0 |
| + refinement | 236.9 | 701.0 | – | 30.7±7.4 | 35.2±7.2 | 19.5±6.3 | 20.6±6.5 | 31.3±7.3 | 32.3±6.9 | 33.7±8.0 | 45.7±9.1 | ||
| Trained on each test scene | training, per frame | ||||||||||||
| ACE [6] (RGB) | 2.1M + 5.6M | our run | 258 s (291 s†), 29 ms | – | – | 76.8±8.1 | 99.4±0.6 | 99.5±0.6 | 100.0±0.0 | – | – | ||
| DSAC* [4] (RGB) | 28 MB† | from [4, 6] | 15 h†, 75 ms† | – | – | 84.0† | – | 99.2† | – | – | – | ||
| DSAC* [4] (RGB-D) | 28 MB† | from [4, 5] | 2.5 d† | – | – | 94.7† | – | – | – | – | – |
Table 3. Room-given development sets: recall and cost. Recall (%) uses 5 cm and 5° or 0.5 m and 5°. Kolibri averages three training seeds, and each baseline uses one run. The ± entries give interval half-widths over walks. Marks compare Fast with each row under the criteria in § 4.1: ≈ equivalent, ▲ Kolibri leads, ◦ inconclusive, ≈ ▲ both. Results with the second refinement stage are in the supplement. Time and memory are medians over 200 ScanNet-10 frames on an RTX 2080 Ti (for ACE, over its scenes). FPFH + TEASER++ uses the CPU. The third column identifies test scenes present in training data, or the source of a row in the last block. Methods in the last block train on each test scene and are never bold, and † marks values from the cited papers, on their full test splits and the GPUs they name.
Kolibri first describes cells in the frame and point cloud, then groups consistent cell matches into candidate poses, and finally refines and scores those poses (Figure 3). Only the cell descriptor is learned. The point cloud descriptors are computed once and reused.
Cell descriptors
Color distinguishes places even at a coarse scale. We backproject the frame’s depth into space and pool its points into cubic cells of side 10 cm. Each cell stores its points’ mean position and color . Links to its nearest cells form the neighborhood . The cell normal points in the direction of least spread among offsets to these neighbors, given by the smallest-eigenvalue eigenvector of their covariance. The input feature contains eight values: color centered on mid-gray, the mean and variance of log edge lengths, and linearity, planarity and scattering from the same covariance. The edge feature contains its log length and the absolute cosines between the offset and each normal, and
between the normals. Absolute values remove sensitivity to normal orientation.
A message-passing layer [15] lets each cell combine its features with those of its neighbors. It has width 128 and produces the descriptor :
Here and are the cell features before and after the update. and are linear layers, is the Gaussian error linear unit (GELU), LN is layer normalization, and is a two-layer perceptron. Brackets concatenate features, and the maximum is taken separately for each feature. The unit-length descriptor has 32 values, and the network has 55,712 parameters.
The same pooling and network describe the reference point cloud. A graph holds at most 16,384 cells. With the room given, larger rooms use coarser cells until they fit. With the room unknown, the point cloud is described in pieces at the nominal cell size, each within this limit. Search then considers the cells of all pieces together. A frame descriptor and point cloud descriptor have similarity , where the temperature controls the sharpness of the matching weights.
Candidate poses and refinement
Each frame cell is matched to its 5 most similar point cloud cells. A softmax converts over all point cloud cells into match weights. Matches and are consistent if their distances agree: , where and are point cloud cell positions and cm. The 256 matches with the most consistent partners each seed a consensus set, an idea from the spatial-consistency estimator SC-PCR [9]. This set contains the starting match and its most consistent partners, one in 10 of all matches and at least 8. A rigid fit weighted by the match weights gives a candidate pose.
Repeated furniture can produce a large group of consistent matches at the wrong place. The estimator therefore keeps several distinct candidates. It retains the 4 best first-round candidates separated by at least 0.5 m or 10°. It removes matches explained by these poses, with residuals below 0.5 m, then makes 2 further rounds. Each round adds its best candidate if distinct and supported by at least 50% as many agreeing matches, or inliers, as the best first candidate. There are at most 6 candidates.
Point-to-plane ICP refines each candidate using 1,000 frame points sampled at even steps through the frame. It aligns these points to the planes of their nearest point cloud neighbors for 10 iterations. The correspondence radius shrinks from 20 cm to 5 cm. The fit score counts how many sampled points lie within 3 cm of the point cloud, and the highest-scoring candidate wins. Describing the frame takes 3.8 ms of the total 82 ms per frame. Candidate refinement accounts for most of the remaining time.
Operating points
The pipeline above is our standard setting, Fast. Two optional operations trade time for accuracy or tolerance to color changes, without retraining. Precise adds a tighter second refinement to every candidate. It uses point-to-plane ICP with Tukey weights, which suppress large residuals, at scale 5 cm and a radius that shrinks from 10 cm. The color pass adds another reading of the frame. A shared affine transform of its color channels matches its luminance mean and spread to those of the point cloud. Both the original and adjusted colors go through the full pipeline. The fit score selects the adjusted reading only if it fits strictly better. Unless stated otherwise, Kolibri uses Fast with one color reading.
Training
Training uses 170 rooms, disjoint from every evaluation set: 165 ScanNet rooms [10] and 5 spaces in a university building. Color and depth views rendered along walks provide the training frames. No real camera frame enters training. Augmentations include depth noise that grows with squared range, changes in scale, brightness, contrast and white balance, and missing sectors. For each frame cell, the positive match is the point cloud cell nearest its true position, if within 0.5 m. The loss is cross-entropy over the softmax matching weights. Training uses AdamW with learning rate 0.001 for 18,600 steps of 8 views, taking at most 53 minutes on an RTX 4090.
Experiments
Setup
Evaluation sets. Table 1 names the evaluation sets and their roles. Reference camera poses come from RGB-D reconstruction [5, 10]. For ScanNet [10], the point cloud and frames come from the same recording. A second session (S2) is a later recording localized in the first session’s point cloud. The 7-Scenes [40] and 12-Scenes [43] point clouds come from their training recordings, so their queries also cross sessions. Development uses ScanNet-10, ScanNet-10-S2, every tenth 7-Scenes test frame, and 12-Scenes. Five
| Set | Frames | Rooms | Point cloud | Role |
| Room given (Tabs. 2 and 3) and rooms joined into one point cloud (Sec. 4.4) | ||||
| ScanNet-10 | 871 | 10 | same recording | dev.: design, ablation rooms |
| ScanNet-10-S2 | 683 | 4 | first session | development |
| 7-Scenes | 1,700 | 7 | other recordings | development, ablation |
| 12-Scenes | 575 | 12 | other recordings | development |
| ScanNet-96 | 7,986 | 96 | same recording | test |
| ScanNet-96-S2 | 5,133 | 41 | first session | test |
| Held-out test | 679 | 10 | same recording | test, run once |
| rendered | 2,000† | 10 | same recording | test, run once |
| Room unknown, whole scans (Secs. 4.5 and 4.6) | ||||
| Emergency dept. | 440† | many | handheld laser scan | test |
| Bonn | 597 | one | terrestrial laser scan | scope limit |
caption: tbl:1. Evaluation sets and their roles. A second session (S2) is a later recording localized in the first session’s point cloud. Development sets determine the design and settings. Test sets play no part in either. † marks views rendered from the point cloud with simulated depth noise. Other frames come from real sensors.
:::
ScanNet-10 design rooms determine the network and settings. The other five ablation rooms, together with 7-Scenes, support the component comparisons in § 4.3. ScanNet-96 contains the 96 of the next 100 ScanNet rooms whose reference poses fit their point clouds. ScanNet-96-S2 contains 60 later scans of 41 of these rooms. Neither set shares a room with training. A further 10 rooms form the held-out test, evaluated once after the method was frozen.
Baselines. The point cloud registration methods are BUFFER-X [39] and its released light setting BUFFER-X-Lite, GeDi [33], ColorPCR [25], RoITr [48], PREDATOR [18], FPFH with the robust estimator TEASER++ [36, 46], and RGB GeoTransformer. The latter adds color input to GeoTransformer [34] and is fine-tuned on our training rooms. The other learned methods use released weights trained on 3DMatch [49] or its colored version [25]. 3DMatch contains 7-Scenes scenes and 8 of the 12 12-Scenes rooms, which the colored version omits. We report each baseline as released (native) and with our refinement (+ refinement). BUFFER-X retains its own refinement in both cases. Comparisons with Precise give baselines the same second refinement stage. BUFFER-X chooses its patch radius from the larger input point cloud and samples a fixed number of keypoints, which suits similarly sized fragments. It therefore receives single rooms or 3 m cubes, never a point cloud spanning several rooms.
Statistics and timing. A walk is a run of consecutive frames. Recall differences carry 95% intervals from a bootstrap that resamples whole walks, pairs methods on the same frames, and averages over three training seeds. The ± entries in Table 2, Table 3 and Table 5 give interval half-widths. Methods are equivalent when the 90% interval lies within ±3 recall points. A method leads when the 95% interval excludes zero in its favor. Equivalence and a lead can both hold. A comparison meeting neither criterion is inconclusive. Resampling rooms or seeds gives the same verdicts, and the largest standard deviation of Kolibri recall across seeds is 0.6 points. Unless stated otherwise, time and memory are medians over 200 fixed ScanNet-10 frames, processed individually on four CPU threads and an RTX 2080 Ti. Kolibri and GeDi reuse point cloud descriptors. Other baselines run as released.
| One color reading Fast | One color reading Precise | Color pass Fast | Color pass Precise | |
| Time per frame, median (ms) | 81.7 | 116.4 | 187.8 | 278.0 |
| Time per frame, p95 (ms) | 251.1 | 417.4 | 677.3 | 1,030 |
| Peak GPU memory (GB) | 1.44 | 1.44 | 1.55 | 1.55 |
| Recall at 5 cm, clean frames | ||||
| ScanNet-10 | 75.7 ±6.5 | 79.2 ±5.9 | 76.6 ±6.5 | 79.7 ±5.6 |
| 7-Scenes | 93.0 ±2.5 | 93.1 ±2.7 | 93.6 ±2.3 | 93.7 ±2.7 |
| 12-Scenes | 97.9 ±3.1 | 98.6 ±2.0 | 98.0 ±2.9 | 98.7 ±1.9 |
| ScanNet-10-S2 | 73.3 ±7.4 | 75.9 ±7.3 | 73.5 ±7.4 | 76.2 ±7.4 |
| Recall at 0.25 m, mean over 15 color shifts | ||||
| ScanNet-10 | 64.0 ±4.7 | 65.8 ±4.4 | 74.6 ±4.7 | 76.6 ±4.2 |
| 7-Scenes | 71.4 ±5.8 | 72.8 ±5.6 | 83.1 ±4.4 | 84.3 ±4.0 |
Table 5. Optional refinement and color adjustment trade time for accuracy or tolerance to color shifts. Recall (%) averages three training seeds, with interval half-widths over walks. Bold marks the best value per row. Time and memory follow Table 3.
Room given: accuracy and cost
With the room given and refinement matched, Fast localizes 81.4% of ScanNet-96 frames within 5 cm and 5°, against GeDi’s 80.3% (Table 2). The methods are equivalent at this threshold, and Fast leads BUFFER-X. Across both ScanNet-96 sets, all four operating points and all three thresholds, none of the 240 matched-refinement comparisons shows a significant deficit for Kolibri. Development-set comparisons are also equivalent, in favor of Kolibri, or inconclusive (Table 3). Fast takes 82 ms per frame, while BUFFER-X takes 18 times as long. It also uses less GPU memory than BUFFER-X and GeDi. Even after varying their settings, no baseline reaches Fast’s ScanNet-10 recall in less than 15 times its time (Figure 1b).
Figure 1. Kolibri searches a colored point cloud without a room label and matches the accuracy of BUFFER-X given the true room. (a) In the ten-room ScanNet-10 point cloud, image retrieval favors similar-looking views from other rooms (gray). Searching the full point cloud lets Kolibri localize this frame in its true room (orange). (b) With the room given, recall at 5 cm and is plotted against median time per frame on an RTX 2080 Ti. Methods use their released settings and our refinement, with Kolibri in its standard Fast setting. Vertical bars are 95% intervals that resample whole walks of consecutive frames. Thin lines in a method’s color show its other settings (BUFFER-X, GeDi and RGB GeoTransformer).
The held-out test supports the same result on rooms unused in any design choice. On its sensor frames, Precise reaches 73.1% within 5 cm and 5°, against 72.6% for GeDi with the same refinement stages. They are equivalent, as are Fast and GeDi with the shared refinement at this threshold. On rendered views, Precise is equivalent to ColorPCR within 0.5 m and 5° and leads GeDi and BUFFER-X.
For a different tradeoff, the scene-specific method ACE [6] first trains for minutes on each test scene. It reaches 76.8% on our 7-Scenes frames within 5 cm, against 93.0% for Fast (Table 3). The image-based methods MaST3R [22] and Reloc3r [11] instead use 64 rendered views per room.
With the same refinement, they trail Fast on every evaluated set and threshold.
Why a small network is enough
Refinement reduces the accuracy needed from the learned descriptor. The success rate falls to half at an initial position error of 7.7 cm (Figure 4a), which defines the measured basin boundary. Even when Fast’s best initial candidate lies outside this basin, another candidate rescues 51% of frames, against 21% for BUFFER-X. Increasing capacity has little effect over the tested range. From 4,736 to 915,584 parameters, recall stays within 1.0 point of our network on ablation rooms (Figure 4b). Training on 22 rooms also gives recall equivalent to training on all 170 (Figure 4c).
What matters is the information exchanged between cells (Table 4). With Precise on the ablation rooms, removing message passing costs 13.4 points, and removing color costs 26.2. Keeping only color on the same graph loses 3.4 points, so local geometry still matters. GeDi’s descriptor uses 56 times as many parameters and localizes more frames before refinement. That advantage disappears after refinement.
Search without a room label
To separate search scale from changes in scene geometry, we place rooms apart on a grid with pitch 20 m. They share no walls. The ten-room and seven-room point clouds join ScanNet-10 and 7-Scenes respectively. The largest joins all 100 rooms from which ScanNet-96 is drawn, totaling 15.0 million points. Retrieval references use the image descriptor DINOv2 [28] to rank rooms against 64 rendered views per room. Retrieve-3 + BUFFER-X registers in the top three rooms. Retrieve-1 + Fast runs Kolibri in the top room.
On the ten-room point cloud, Fast reaches 74.3% within 5 cm in 99 ms per frame (Figure 1 and Figure 5a). Its recall is equivalent to BUFFER-X given the true room. Retrieval includes the true room in its top three for only 80.8% of frames. Retrieve-3 + BUFFER-X therefore trails Fast by 14.0 points and takes 47 times as long.

Figure 5. Searching all rooms avoids losses from choosing a room first. (a) Recall at 5 cm and 5° in the ten-room and seven-room point clouds, without a room label. BUFFER-X receives only the true room or retrieved rooms, because its patch radius depends on point cloud size. (b) Median time per frame and peak GPU memory as the point cloud grows, on 288 ScanNet-96 frames and an RTX 2080 Ti.
Searching the 100-room point cloud gives 79.9% recall, only 1.3 points below a paired run given the true room. That reference is separate from the room-given run in Table 2. Keeping several candidates is important: with one candidate, recall falls to 46.2%. Retrieve-1 + Fast reaches only 23.9%, losing 56.0 points relative to searching all rooms. The full search takes 168 ms and 2.15 GB per frame (Figure 5b). RGB GeoTransformer and ColorPCR exhaust the GPU from 10 rooms onward.
A hospital emergency department
The emergency department returns to our motivating setting: many connected rooms captured on foot with a handheld laser scanner and colored by its camera. The colored point cloud spans 49 m by 42 m in plan and played no part in development or tuning. There is no RGB-D recording. The test therefore contains 440 RGB-D frames rendered from the scan along 30 walks, with simulated depth noise. Their colors agree with the point cloud by construction, which favors color-based methods, and their reference poses are exact. Kolibri searches the whole scan without a room label. Baselines take minutes per frame, so they use 30 frames, one from the start of each walk.
Fast localizes 86.5% of all rendered frames within 5 cm and , in 384 ms per frame on an RTX 2080 Ti (Figure 6 and Table 6). GeDi on the whole point cloud reaches 26.7% on the baseline subset. Its random sample consensus (RANSAC) estimator fits poses from sampled matches, but ends with fewer than three agreeing matches on most frames.

Figure 6. Kolibri localizes frames throughout the emergency department, with failures concentrated in a few places. The plotted positions belong to 440 views rendered from the scan along 30 walks, with simulated depth noise and colors that agree with the point cloud. The floor plan comes from the scan geometry. Markers count the training seeds for which Fast succeeds within 5 cm and 5°. GeDi uses the first frame of each walk.
| Method | Search | Frames | 5 cm | 0.5 m | s/frame |
| Kolibri Fast | whole scan | 440 | 86.5 ±0.7 | 90.6 ±0.5 | 0.38 |
| 30 | 84.4 | 90.0 | |||
| Kolibri Precise | whole scan | 440 | 86.4 ±0.5 | 91.1 ±0.5 | 0.62 |
| 30 | 86.7 | 91.1 | |||
| GeDi | whole scan | 30 | 26.7 | 30.0 | 531 |
| GeDi | 350k keypoints, 15 cm | 30 | 43.3 | 46.7 | 121 |
| GeDi | 1,090 cubes of 3 m | 30 | 76.7 | 86.7 | 5,485 |
| BUFFER-X | 1,090 cubes of 3 m | 30 | 60.0 | 63.3 | 932 |
Table 6. In a hospital emergency department, Kolibri localizes a frame in a fraction of a second, where GeDi needs 91 minutes on cubes of the scan for a similar recall. Views are rendered from the scan with simulated depth noise and matching colors. Recall (%) uses 5 cm and 5° or 0.5 m and 5°, without a room label. Baselines use shared refinement on 30 frames, one per walk. For Kolibri on all frames, ± gives the standard deviation over three training seeds. Time is median seconds per frame on an RTX 2080 Ti. The paired comparison between Fast and GeDi on cubes is inconclusive.
Searching 1,090 overlapping 3 m cubes raises GeDi’s recall to 76.7%. The cost is 91 minutes per frame. On these same frames, the difference between Fast and GeDi on cubes is inconclusive: +7.8 points [−7.8, +23.4]. BUFFER-X on the cubes reaches 60.0%.
A sparser point cloud reduces this cost but does not close the gap. Kolibri represents the scan with 350,087 cells for place search. Given a keypoint per cell, 350,000 in all, GeDi reaches 43.3% in 2 minutes per frame with its best of 5 RANSAC inlier distances. This improves both recall and time over GeDi on the full point cloud, but still takes 315 times as long as Fast for half its recall.
Operating points and dependence on color
The second refinement stage in Precise adds 35 ms per frame (Table 5) and gains 4.0 points at 5 cm on ScanNet-96. The color pass takes 2.3 times as long as Fast. Across the tested color changes, including brightness shifts and conversion to gray, it gains an average 10.7 points at 0.25 m and 2° on ScanNet-10 and 11.8 on 7-Scenes.
These options do not remove the dependence on color. At the same threshold, RGB GeoTransformer leads Kolibri by 9.2 points averaged over the color shifts, and by 47.2 on gray frames. The same limitation appears across cameras. Bonn [29] pairs real ASUS Xtion frames with a terrestrial laser scan colored by the scanner. Fast reaches 26.3% within 0.5 m, or 47.2% with the color pass, against 91.5% for GeDi on cubes. GeDi takes 11 minutes per frame. Changes between recording sessions affect Fast and the six strongest refined baselines alike: they lose 16 to 25 points at 5 cm on ScanNet-96-S2 (Table 2).
Conclusion
Kolibri uses color to find candidate places and geometric refinement to set the pose. With matched refinement, a small network reaches the accuracy of point cloud registration methods in single rooms at a fraction of their cost. The saved time makes it practical to search a point cloud spanning 100 rooms or a hospital emergency department without a room label. Agreement in color remains essential, and point cloud registration methods retain the lead when it breaks. Extending search to whole buildings calls for a hierarchy that reduces work without discarding the correct place before refinement.
Author Contributions
Contributions by role, following the CRediT taxonomy: Bartu Tuncay: Conceptualization, Methodology, Software, Data curation, Formal analysis, Investigation, Validation, Visualization, Writing (original draft), Writing (review and editing). Leonel Aguilar: Conceptualization, Methodology, Software, Formal analysis, Investigation, Validation, Visualization, Writing (original draft), Writing (review and editing), Supervision, Project administration. Christoph Hölscher: Supervision, Resources, Funding acquisition, Writing (review and editing).
LLM Usage Statement
OpenAI Codex and Anthropic Claude helped edit the paper and supplement, check reported values. Claude Code was used for programming the computational experiments.
Supplementary Material
This supplement provides the full comparison rows, protocols and controls behind the main paper. The operating points Fast, Precise and the color pass are defined in § 3.4. Statistical terms, including equivalent, lead and inconclusive, follow § 4.1.
What color contributes
Human scene recognition. People recognize scene categories from brief glimpses [16, 38], using coarse visual structure before fine edges [38]. Color helps most at this coarse scale [7, 27], especially when a scene category has characteristic colors [27]. Recognizing a place and orienting within it engage different brain systems [12, 13]. Facing direction is represented relative to the local environment [24], and young children reorient using room shape even when a wall has a distinctive color [17]. This distinction has a parallel in Kolibri: color identifies candidate places, and geometry sets the pose.
Which rooms depend on color? Color diversity differs between rooms. We measure it by the entropy of the point cloud cells’ chromatic coordinates, which describe color independently of lightness. Across 29 rooms, greater diversity accompanies a larger drop in Fast recall when frames become gray (Figure S1). At 0.25 m and 2°, Spearman’s rank correlation is +0.63, with a room-bootstrap interval of [+0.38, +0.79]. The relationship also holds within ScanNet and 7-Scenes, with correlations of +0.66 and +0.75. For the gain from the color pass under color shifts, the corresponding correlation is [−0.66, +0.23], with an interval that includes zero (Figure S1).
Figure S1. Rooms with more diverse colors lose more recall on gray frames. Each point represents a room. Color diversity is the entropy of the chromatic coordinates and in the CIELAB color space. (a) The vertical axis gives the drop in Fast recall from unchanged to gray frames at 0.25 m and 2°. (b) It gives the gain from the color pass, averaged over color shifts on ScanNet and 7-Scenes.
Changes to depth and color. Figure S2 gives both panels of the damage tests from § 4.6. Across 12 levels of point loss, depth noise, reduced resolution, range cuts and holes, Fast has no significant deficit against BUFFER-X or GeDi. At 0.25 m and 2°, its largest leads are 8.6 and 34.3 points respectively (Figure S2). Color changes expose the dependence on color (Figure S2). On gray frames, RGB GeoTransformer leads Kolibri by 47.2 points at the same threshold.

Figure S2. Damaged depth costs Kolibri no more than the baselines, and damaged color is its price. Each comparison gives Fast recall minus baseline recall at 0.25 m and 2°, with intervals over walks. The frame’s depth changes in (a), and its color changes in (b). The point cloud stays unchanged. In (b), filled marks use two color readings selected by the same geometric fit score, corresponding to the color pass for Kolibri. Hollow marks use one reading.
Several candidates against a lookalike place
Repeated structures can give the strongest group of matches to the wrong place. In the washroom example of Figure S3, that group points to a similar stall 1.06 m away, rotated by 177°. Another group proposes the true place. Refinement and the geometric fit score from § 3.3 select it: 0.95 of the frame points fall within 3 cm of the point cloud, compared with 0.79 at the lookalike. The final position error is 1.7 cm.

Figure S3. Several candidates recover the true place when the strongest match group points to a lookalike. The example is an unseen washroom with similar stalls, localized by Fast. The top row shows the frame, its cell graph, the network blocks at their relative sizes, and descriptors colored by a shared projection. The bottom row follows the matches through candidate poses, refinement, fit scoring and the final pose. Dashed boxes mark the optional second refinement and color pass from § 3.4.
Full room-given comparisons
Development sets. Table S1 extends Table 3 with the second refinement stage for every baseline. These rows compare against Precise, so the refinement is matched.
| Method | Parameters | in training | median ms | p95 ms | GB | ScanNet-10 5 cm | ScanNet-10 0.5 m | 7-Scenes 5 cm | 7-Scenes 0.5 m | 12-Scenes 5 cm | 12-Scenes 0.5 m | ScanNet-10-S2 5 cm | ScanNet-10-S2 0.5 m |
| Kolibri Fast | 55.7k | none | 81.7 | 251.1 | 1.44 | 75.7±6.5 | 94.9±2.5 | 93.0±2.5 | 98.5±0.9 | 97.9±3.1 | 99.7±0.6 | 73.3±7.4 | 95.3±3.3 |
| Kolibri Precise | 55.7k | none | 116.4 | 417.4 | 1.44 | 79.2±5.9 | 95.1±2.5 | 93.1±2.7 | 98.6±0.9 | 98.6±2.0 | 99.7±0.6 | 75.9±7.3 | 95.4±3.4 |
| BUFFER-X [39] | 0.91M | 7S, 12S | 1,416 | 1,922 | 6.21 | 66.4±6.7▲ | 91.3±3.8▲ | 72.7±7.8▲ | 96.2±2.5▲ | 82.1±7.8▲ | 91.8±5.0▲ | 66.9±8.6▲ | 95.2±2.9° |
| + refinement | 1,444 | 1,948 | 6.21 | 74.7±6.9≈ | 92.9±3.6° | 92.2±3.6≈ | 97.5±2.3° | 91.8±4.7▲ | 93.4±4.4▲ | 73.8±7.3≈ | 97.1±2.0° | ||
| + second stage | 1,480 | 1,997 | 6.21 | 77.4±6.3▲ | 92.9±3.6▲ | 92.1±4.0° | 97.5±2.3° | 92.5±4.5▲ | 93.4±4.6▲ | 76.6±6.8≈ | 97.1±2.0° | ||
| BUFFER-X-Lite [39] | 0.91M | 7S, 12S | 1,280 | 1,896 | 6.21 | 63.9±6.7▲ | 90.7±4.2▲ | 66.6±7.3▲ | 96.4±2.5▲ | 81.6±7.9▲ | 91.7±5.6▲ | 62.1±9.6▲ | 95.0±2.9° |
| + refinement | 1,299 | 1,925 | 6.21 | 74.4±6.8≈ | 92.9±3.5° | 92.3±3.5≈ | 97.5±2.3° | 91.7±4.7▲ | 93.4±4.5▲ | 73.5±7.0≈ | 96.8±2.2° | ||
| + second stage | 1,317 | 1,949 | 6.21 | 77.4±6.2° | 92.9±3.7° | 92.3±3.8≈ | 97.5±2.2° | 92.3±4.5▲ | 93.4±4.5▲ | 76.3±6.9≈ | 96.8±2.1° | ||
| GeDi [33] | 3.11M | 7S, 12S | 24,149 | 43,869 | 5.27 | 78.8±5.8° | 92.8±3.5° | 70.8±9.2▲ | 90.8±3.9▲ | 98.8±0.9° | 99.7±0.4≈ | 74.4±6.8° | 95.9±2.5° |
| + refinement | 24,179 | 43,902 | 5.27 | 76.2±7.1≈ | 95.1±2.6≈ | 90.9±3.8° | 95.8±2.4▲ | 98.1±2.9≈ | 99.7±0.6≈ | 73.6±7.1≈ | 97.5±1.7° | ||
| + second stage | 24,205 | 43,929 | 5.27 | 79.9±5.8≈ | 95.3±2.5≈ | 90.8±4.1▲ | 95.8±2.5▲ | 99.0±1.6≈ | 99.7±0.6≈ | 76.6±6.9≈ | 97.5±1.7° | ||
| RGB GeoTransformer [34] | 9.83M | 7S, 12S | 2,501 | 6,338 | 2.70 | 76.3±5.5° | 88.2±4.4▲ | 66.1±9.8▲ | 92.4±3.2▲ | 99.1±0.8° | 99.8±0.2≈ | 72.6±7.2° | 93.0±3.4° |
| + refinement | 2,530 | 6,372 | 2.70 | 73.5±7.4▲ | 89.4±4.0▲ | 89.8±4.3▲ | 94.9±2.9▲ | 98.1±2.9≈ | 99.7±0.4≈ | 72.3±7.1≈ | 94.3±2.9° | ||
| + second stage | 2,560 | 6,395 | 2.70 | 76.1±6.4▲ | 89.7±4.2▲ | 89.6±4.3▲ | 94.9±2.7▲ | 98.8±1.7≈ | 99.7±0.4≈ | 75.0±7.3° | 94.3±2.9° | ||
| ColorPCR [25] | 10.03M | 7S | 4,077 | 10,878 | 3.90 | 66.4±6.8▲ | 89.1±4.5▲ | 63.5±8.8▲ | 89.1±4.8▲ | 95.5±2.3° | 98.4±1.0≈▲ | 58.6±9.8▲ | 81.7±7.6▲ |
| + refinement | 4,107 | 10,913 | 3.90 | 72.6±7.0▲ | 91.4±3.6▲ | 86.3±5.0▲ | 92.4±4.0▲ | 97.2±2.9≈ | 98.8±0.9≈ | 67.2±8.7▲ | 86.1±6.7▲ | ||
| + second stage | 4,131 | 10,943 | 3.90 | 76.2±6.0▲ | 91.8±3.6▲ | 86.1±5.1▲ | 92.5±4.0▲ | 97.9±1.9≈ | 98.8±0.9≈ | 69.4±8.3▲ | 86.2±6.5▲ | ||
| RoITr [48] | 10.10M | 7S, 12S | 6,402 | 22,324 | 2.07 | 60.2±6.9▲ | 86.3±4.5▲ | 55.8±13.3▲ | 81.2±10.6▲ | 77.2±8.9▲ | 92.5±5.8▲ | 61.5±6.9▲ | 90.6±4.0° |
| + refinement | 6,429 | 22,353 | 2.07 | 72.1±7.3▲ | 89.2±4.0▲ | 81.4±9.1▲ | 86.1±9.0▲ | 92.9±6.8▲ | 94.4±4.9▲ | 71.4±7.3° | 93.6±3.5° | ||
| + second stage | 6,450 | 22,389 | 2.07 | 75.0±6.7▲ | 89.7±4.1▲ | 81.6±9.2▲ | 86.2±8.4▲ | 93.7±5.6▲ | 94.8±4.8▲ | 73.9±7.0 | 93.6±3.5° | ||
| PREDATOR [18] | 7.43M | 7S, 12S | 728.1 | 1,287 | 3.88 | 13.4±4.3▲ | 38.2±6.9▲ | 14.2±8.1▲ | 33.7±12.2▲ | 30.4±8.8▲ | 52.9±10.6▲ | 20.6±6.6▲ | 46.9±7.7▲ |
| + refinement | 754.0 | 1,316 | 3.88 | 47.4±7.4▲ | 53.4±7.0▲ | 47.0±13.3▲ | 51.4±12.8▲ | 72.5±8.5▲ | 74.4±9.0▲ | 53.1±7.9▲ | 67.6±6.6▲ | ||
| + second stage | 783.7 | 1,347 | 3.88 | 49.1±7.1▲ | 53.4±6.8▲ | 48.6±13.3▲ | 51.9±12.8▲ | 73.4±9.0▲ | 74.6±9.2▲ | 55.3±7.2▲ | 68.7±6.6▲ | ||
| FPFH + TEASER++ [36, 46] | none | none | 215.0 | 648.5 | — | 11.0±3.9▲ | 24.9±6.7▲ | 2.6±1.5▲ | 10.8±4.5▲ | 8.0±2.9▲ | 16.3±5.0▲ | 9.5±3.4▲ | 29.6±8.0▲ |
| + refinement | 236.9 | 701.0 | — | 30.7±7.4▲ | 35.2±7.2▲ | 19.5±6.3▲ | 20.6±6.5▲ | 31.3±7.3▲ | 32.3±6.9▲ | 37.3±8.0▲ | 45.7±9.8▲ | ||
| + second stage | 265.2 | 740.9 | — | 32.7±7.2▲ | 35.5±7.0▲ | 19.9±6.8▲ | 21.1±6.8▲ | 32.2±7.2▲ | 33.0±7.0▲ | 35.7±8.3▲ | 46.3±8.7▲ | ||
| Trained on each test scene | training, per frame | ||||||||||||
| ACE [6] (on RGB) | 2.1M + 5.6M | our run | 258 s (291 s†), 29 ms | — | — | 76.8±8.1 | 99.4±0.6 | 99.5±0.6 | 100.0±0.0 | — | — | ||
| DSAC* [4] (RGB) | 28 MB† | from [4, 6] | 15 h†, 75 ms† | — | — | 84.0† | — | 99.2† | — | — | — | ||
| DSAC* [4] (RGB-D) | 28 MB† | from [4, 5] | 2.5 d† | — | — | 94.7† | — | — | — | — | — |
Table S1. Table S1. Room-given development sets with every refinement stage. Recall (%) uses 5 cm and 5° or 0.5 m and 5°. Gray rows are Kolibri, and bold marks the best value per column among methods that do not train on the test scenes. Kolibri averages three training seeds, while each baseline uses one run. The + entries give half-widths of their intervals over the test walks. Native denotes released output, including the refinement supplied by BUFFER-X and BUFFER-X-Lite. The next rows add our point-to-plane refinement and then the second stage. Marks compare Fast with native and refined rows, and Precise with second-stage rows: ≈ equivalent within ±3 points, ▲ Kolibri leads, ° inconclusive, ≈ ▲ both. Time and memory are medians over 200 ScanNet-10 frames on an RTX 2080 Ti (for ACE, over its scenes). FPFH + TEASER++ uses the CPU. The third column identifies test scenes present in training data, or the subset of a row in the last block. Methods in the last block train on each test scene, and † marks values from the cited papers, on their full test splits and the GPUs they name.
Held-out test. Table S2 gives the full results on 10 rooms evaluated once after the method was frozen. The set contains 679 sensor frames and 2,000 rendered views. With matched refinement, Fast and Precise are equivalent to GeDi on sensor frames at 5 cm and 5° and at 0.25 m and 2°. Comparisons with the other baselines favor Kolibri or are inconclusive. GeDi’s native output outperforms Fast at 5 cm and 5°, but this lead disappears when both use the shared refinement. On rendered views, matched refinement makes Kolibri equivalent to ColorPCR and ahead of the remaining baselines.
| Method | Sensor frames (679) 5 cm | Sensor frames (679) 0.25 m | Sensor frames (679) 0.5 m | Rendered views (2,000) 5 cm | Rendered views (2,000) 0.25 m | Rendered views (2,000) 0.5 m |
| Kolibri Fast | 68.7 | 84.9 | 92.9 | 94.6 | 94.7 | 94.9 |
| Kolibri Precise | 73.1 | 87.1 | 93.3 | 94.8 | 94.9 | 95.0 |
| Kolibri Fast + color pass | 69.1 | 86.2 | 93.9 | 94.5 | 94.7 | 94.8 |
| Kolibri Precise + color pass | 73.4 | 88.0 | 94.4 | 94.8 | 94.9 | 94.9 |
| BUFFER-X [39] | 58.3▼ | 72.9▲ | 86.6▲ | 86.2▲ | 85.7▲ | 89.2▲ |
| + refinement | 66.0◦ | 81.7◦ | 88.1◦ | 90.6▲ | 90.7▲ | 90.8▲ |
| + second stage | 69.2◦ | 83.2◦ | 88.4◦ | 90.8▲ | 90.8▲ | 90.8▲ |
| BUFFER-X-Lite [39] | 54.3▲ | 68.8▲ | 86.2▲ | 83.2▲ | 85.3▲ | 89.4▲ |
| + refinement | 66.4◦ | 81.9◦ | 87.8◦ | 90.8▲ | 90.9▲ | 91.0▲ |
| + second stage | 69.7◦ | 83.4◦ | 88.1◦ | 91.0▲ | 91.0▲ | 91.0▲ |
| GeDi [33] | 73.0▼ | 82.2◦ | 89.8◦ | 88.1▲ | 88.1▲ | 89.6▲ |
| + refinement | 68.6≈ | 84.8≈ | 91.3◦ | 90.9▲ | 91.0▲ | 91.0▲ |
| + second stage | 72.6≈ | 86.7≈ | 91.9◦ | 91.0▲ | 91.0▲ | 91.1▲ |
| RGB GeoTransformer [34] | 64.4◦ | 71.0▲ | 78.1▲ | 86.2▲ | 85.5▲ | 87.4▲ |
| + refinement | 62.7▲ | 74.7▲ | 79.5▲ | 87.8▲ | 87.8▲ | 87.8▲ |
| + second stage | 65.4▲ | 75.7▲ | 80.0▲ | 87.8▲ | 87.8▲ | 87.8▲ |
| ColorPCR [25] | 63.8◦ | 74.1▲ | 84.5▲ | 92.0▲ | 91.4▲ | 93.6≈▲ |
| + refinement | 65.7▲ | 80.9▲ | 87.5▲ | 94.2≈ | 94.2≈ | 94.3≈ |
| + second stage | 69.4▲ | 82.2▲ | 87.5▲ | 94.2≈ | 94.3≈ | 94.4≈ |
| RoITr [48] | 49.5▲ | 58.9▲ | 71.9▲ | 44.8▲ | 46.5▲ | 59.9▲ |
| + refinement | 60.1▲ | 70.8▲ | 73.9▲ | 64.5▲ | 64.7▲ | 65.0▲ |
| + second stage | 62.6▲ | 71.6▲ | 74.2▲ | 64.8▲ | 64.9▲ | 65.1▲ |
| PREDATOR [18] | 15.8▲ | 16.5▲ | 33.1▲ | 25.3▲ | 27.2▲ | 45.7▲ |
| + refinement | 40.2▲ | 45.2▲ | 47.6▲ | 55.2▲ | 55.9▲ | 56.6▲ |
| + second stage | 42.0▲ | 46.2▲ | 48.0▲ | 55.8▲ | 56.2▲ | 56.7▲ |
| FPFH + TEASER++ [36, 46] | 12.1▲ | 14.4▲ | 28.9▲ | 29.9▲ | 37.5▲ | 50.3▲ |
| + refinement | 29.6▲ | 33.9▲ | 36.2▲ | 56.7▲ | 56.8▲ | 57.5▲ |
| + second stage | 30.5▲ | 34.8▲ | 36.4▲ | 57.0▲ | 57.1▲ | 57.6▲ |
Table S2. Every row of the held-out test. Recall (%) uses 5 cm and 5°, 0.25 m and 2°, or 0.5 m and 5°. Gray rows are Kolibri, and bold marks the best value per column. A frame without a valid pose counts as a miss. Kolibri averages three training seeds, while each baseline uses one run. Native denotes released output, including the refinement supplied by BUFFER-X and BUFFER-X-Lite. The next rows add our point-to-plane refinement and then the second stage. Marks compare Fast with native and refined rows, and Precise with second-stage rows. They follow Tab. S1, with ▼ indicating that Kolibri trails.
Search without a room label
The point cloud in § 4.4 joins 100 ScanNet rooms placed apart on a grid, totaling 15.0 million points. Queries are all 7,986 first-session frames of ScanNet-96. Table S3 compares search over all rooms with a paired run on the same frames given the true room. This reference run is separate from the room-given results in Table 2.
| Recall (%) | Median ms | ||||
| Own room | All rooms | Loss (points) | Own room | All rooms | |
| Fast | 81.2 | 79.9 | 1.3 [0.8, 1.8] | 65 | 168 |
| Precise | 85.1 | 83.7 | 1.4 [0.9, 1.9] | – | 230 |
Table S3. Searching all rooms has a small recall cost. Recall uses 5 cm and 5°. The loss compares all-room search with a paired run given the true room, with an interval over walks. Median times use the 288-frame timing subset on an RTX 2080 Ti.
Candidates and room retrieval. Several candidates are important even with refinement fixed. Keeping only one reduces all-room recall to 46.2%. The image descriptor DINOv2 ranks the true room first for 28.6% of frames and in the top three for 44.7%. Running Fast in the top room gives 23.9% recall, 56.0 points below searching all rooms, with interval [53.3, 58.6]. All-room search places 98.0% of frames in the correct room.
Memory. Peak GPU memory is 2.15 GB for Kolibri on the full point cloud. RGB GeoTransformer and ColorPCR exhaust the GPU from 10 rooms onward.
Cell size. Both finer and coarser cells reduce recall on the ScanNet-10 ablation rooms. Relative to the 10 cm default, cells of 5 cm and 20 cm lose 2.3 and 1.9 points at 5 cm and 5°. Median times are 249, 82 and 90 ms for fine, default and coarse cells respectively. Fine cells exhaust the RTX 2080 Ti on 40% of ScanNet-10 frames, so their recall is measured on an RTX 4090.
The hospital emergency department
Scan and frames. The colored point cloud contains 2.1 million points and spans 49 by 42 m in plan. There is no RGB-D recording, so the test frames are rendered from the scan. Each of the 30 walks starts at a random walkable position and takes up to 20 steps of 0.5 m. The camera height ranges from 1.5 to 1.7 m, with a downward tilt of up to 20°. Each point is rendered in its own color as a square of half-size 2 cm. Simulated sensor noise perturbs depth, while colors agree with the point cloud by construction. Frames require depth on at least 30% of pixels and a median depth of at least 0.8 m. This leaves 440 frames, shown in Figure 6. Baselines use the first frame of each walk.
GeDi’s settings on the whole point cloud. GeDi’s starting configuration describes every point and uses an inlier distance of 2 cm, calibrated on room development frames. This distance determines which matches agree with a candidate pose in its random sample consensus (RANSAC) estimator. On the full point cloud, most frames end with fewer than three agreeing matches. Table S4 varies both the number of described points and the inlier distance. Sparser descriptions reduce time, while wider inlier distances allow agreement among more of the remaining matches. Across the 11 tested settings, the best whole-point-cloud recall at 5 cm and 5° is
| Search | Map keypoints | Inlier dist. | 5 cm | 0.5 m | Time |
| GeDi, whole scan | 100,000 | 2 cm* | 10.0 | 10.0 | 0.7 min |
| GeDi, whole scan | 100,000 | 10 cm | 13.3 | 16.7 | 0.6 min |
| GeDi, whole scan | 100,000 | 20 cm | 16.7 | 20.0 | 0.6 min |
| GeDi, whole scan | 350,000 | 2 cm* | 13.3 | 16.7 | 2.0 min |
| GeDi, whole scan | 350,000 | 5 cm | 33.3 | 36.7 | 2.0 min |
| GeDi, whole scan | 350,000 | 10 cm | 40.0 | 43.3 | 2.1 min |
| GeDi, whole scan | 350,000 | 15 cm | 43.3 | 46.7 | 2.0 min |
| GeDi, whole scan | 350,000 | 20 cm | 40.0 | 43.3 | 2.1 min |
| GeDi, whole scan | 1,000,000 | 5 cm | 36.7 | 43.3 | 5.4 min |
| GeDi, whole scan | 1,000,000 | 10 cm | 40.0 | 43.3 | 5.2 min |
| GeDi, whole scan | every point | 2 cm* | 26.7 | 30.0 | 8.9 min |
| GeDi, 1,090 cubes | every point | 5 cm | 76.7 | 86.7 | 91 min |
| BUFFER-X, 1,090 cubes | every point | released | 60.0 | 63.3 | 16 min |
| Kolibri Fast, whole scan | 350,087 cells | – | 84.4 | 90.0 | 0.38 s |
| Kolibri Precise, whole scan | 350,087 cells | – | 86.7 | 91.1 | 0.62 s |
Table S4. Table S4. The tested GeDi settings on the whole scan remain below its cube search and Kolibri. All rows use the first frame of each walk, 30 rendered frames with simulated depth noise and matching colors. Baselines include the shared refinement. Recall (%) uses 5 cm and 5° or 0.5 m and 5°. Gray rows are Kolibri, and bold marks the best value per column. The keypoint column gives how many points GeDi describes, or the cell count for Kolibri. The inlier distance sets the match tolerance, and * marks the room-calibrated value. Time is the median per frame on an RTX 2080 Ti.
43.3%. Searching separate 3 m cubes raises recall to 76.7%, at 91 minutes per frame.
The size of the searched region. Table S5 separates GeDi’s descriptors from the size of its search. Regions of the scan are cut around each of the 30 frames with the true pose, and GeDi runs with its calibrated settings and no refinement. On a 3 m cube around the points a frame sees, GeDi places 50.0% of the frames within 5 cm and 5°. Recall falls steadily as the region grows, to 13.3% on the largest square, and the share of frames for which GeDi returns a pose falls from 100% to 40%. The descriptors work in the emergency department, and the whole-scan result is the cost of matching against a large point cloud. The cube search recovers recall by running many small searches and keeping the cube with the most agreeing matches.
| Region searched | Map points | Frame inside | 5 cm | 0.25 m | 0.5 m |
| 3 m cube around the seen points | 24,570 | 75% | 50.0 | 66.7 | 83.3 |
| 6 m cube around the seen points | 103,354 | 99% | 30.0 | 50.0 | 66.7 |
| 4 m square around the camera | 53,413 | 39% | 23.3 | 30.0 | 40.0 |
| 8 m square around the camera | 174,401 | 89% | 30.0 | 36.7 | 53.3 |
| 16 m square around the camera | 628,505 | 100% | 23.3 | 23.3 | 36.7 |
| 32 m square around the camera | 1,537,817 | 100% | 13.3 | 20.0 | 26.7 |
| Whole scan | 1,917,644 | 100% | 20.0 | 13.3 | 26.7 |
| 1,090 cubes of 3 m, best cube | every point | – | 63.3 | 80.0 | 83.3 |
Table S5. Table S5. GeDi’s recall falls steadily as the searched region grows. GeDi uses its calibrated settings and no refinement, on the first frame of each walk. Regions are cut from the scan with the true pose, so the rows diagnose the search and are not a localization result. Cubes center on the mean of the frame’s points, and squares center on the camera and span every height. Frame inside gives the median share of the frame’s points inside the region, which limits the smallest square. The cube search keeps, per frame, the cube with the most agreeing matches. Recall (%) uses 5 cm and 5°, 0.25 m and 2°, or 0.5 m and 5°. Bold marks the best value per column.
BUFFER-X and the size of the point cloud
BUFFER-X [39] registers two partially overlapping point clouds. It derives its patch radii from the larger input point cloud. Its released code caps them at 5 m and samples a fixed number of keypoints from each input. As the reference point cloud grows, the patches grow and the keypoint budget spreads across more space. On the grid of rooms in § 11, the median largest radius rises from 0.75 m for one room to 1.08 m for two and 2.50 m for five. It reaches the cap from 10 rooms onward. At this scale, patches describe rooms rather than local surfaces. The paper therefore evaluates BUFFER-X only on single rooms and 3 m cubes.
Taking the radii from the frame instead does not improve the room-given result. On 871 ScanNet-10 frames, this reduces the median largest radius from 0.84 to 0.33 m. With shared refinement, recall falls from 74.7% to 72.2% at 5 cm and 5°, and from 92.9% to 90.0% at 0.5 m and 5°. The room-given comparisons retain the released radius rule.
A point cloud colored by another camera
In Bonn [29], the query frames and the point cloud come from different sensors. Its 597 test frames come from the static ASUS Xtion RGB-D sequence, with motion-capture reference poses. The same room is represented by a terrestrial laser scan colored by the scanner’s camera. Kolibri searches the full point cloud with its room-unknown settings. GeDi searches both the full point cloud and overlapping 3 m cubes. BUFFER-X receives only cubes, following Sec. S6.
The cameras record the same walls in different colors, and Kolibri has low recall (Table S6). Fast reaches 26.3% within 0.5 m and 5°, or 47.2% with the color pass. GeDi, which does not use color, reaches 77.1% on the whole point cloud and 91.5% on cubes. Its median times are 2.8 and 11 minutes per frame respectively. This is the boundary of the main paper’s gains: ScanNet, 7-Scenes and 12-Scenes use the same kind of camera for frames and point clouds, while the hospital’s rendered colors agree by construction.
Table S6. Point cloud registration methods retain the lead across cameras. Recall (%) on Bonn uses 5 cm and 5°, 0.25 m and 2°, or 0.5 m and 5°. Gray rows are Kolibri, and bold marks the best value per column. Kolibri averages three training seeds. Baselines include the shared refinement. Baseline times are medians per frame on an RTX 2080 Ti.
Camera and lens properties
A deployed camera can differ from the development cameras in its lens and its calibration. We change one property of the query camera at a time on the 871 real ScanNet-10 frames and leave the colored point cloud unchanged (Table S7). A narrower lens keeps only the points inside a horizontal field of view of 45° or 30°, where the ScanNet depth camera sees 58°. A focal length 5 or 10% too long or too short scales each frame across the optical axis. Uncorrected radial distortion with coefficient moves each point away from the image center for positive (pincushion) or toward it for negative (barrel), in proportion to its squared distance from the center. A color offset pairs each depth pixel with the color 5 or 10 pixels to its right. BUFFER-X and GeDi receive the same frames and the shared refinement.
| Method | Narrower lens | Focal length error | Radial distortion | Color offset | |||||||||
| Unchanged | 45° | 30° | +5% | +10% | −5% | −10% | +0.1 | +0.2 | −0.1 | −0.2 | 5 px | 10 px | |
| Recall at 5 cm and 5° | |||||||||||||
| Kolibri Fast | 75.6 | 71.8 | 60.9 | 56.4 | 21.7 | 53.9 | 16.5 | 73.1 | 63.9 | 73.0 | 62.5 | 74.9 | 75.4 |
| Kolibri Precise | 79.1 | 75.3 | 63.9 | 51.2 | 16.3 | 47.5 | 11.7 | 76.9 | 67.4 | 75.0 | 61.8 | 78.6 | 78.5 |
| BUFFER-X [39] | 74.4 | 69.7 | 52.0 | 56.3 | 21.5 | 53.6 | 16.2 | 72.0 | 62.9 | 71.4 | 61.7 | 74.4 | 74.4 |
| GeDi [33] | 76.2 | 71.6 | 54.9 | 56.8 | 21.9 | 53.6 | 16.5 | 73.2 | 63.8 | 73.4 | 62.3 | 76.2 | 76.2 |
| Recall at 0.25 m and 2° | |||||||||||||
| Kolibri Fast | 86.9 | 80.9 | 68.6 | 75.1 | 30.3 | 74.4 | 17.9 | 85.5 | 79.4 | 85.8 | 78.9 | 86.5 | 86.5 |
| Kolibri Precise | 89.1 | 83.0 | 71.0 | 74.1 | 23.3 | 68.8 | 12.1 | 86.5 | 81.3 | 85.9 | 77.7 | 88.7 | 88.5 |
| BUFFER-X [39] | 86.2 | 78.6 | 59.0 | 74.6 | 30.3 | 74.9 | 17.2 | 84.7 | 77.6 | 85.3 | 77.6 | 86.2 | 86.2 |
| GeDi [33] | 87.3 | 80.7 | 62.5 | 76.0 | 29.5 | 74.4 | 17.2 | 85.6 | 79.7 | 86.8 | 79.7 | 87.3 | 87.3 |
Table S7. Calibration errors cost every method alike, and a narrow lens favors Kolibri. Recall (%) on the 871 real ScanNet-10 frames when one property of the query camera changes. A focal length error of +5% means a focal length 5% too long. Gray rows are Kolibri. Bold marks the best value per column within each block among Fast and the baselines, which share one refinement. Precise adds a second refinement stage. Kolibri averages three training seeds. A color offset leaves the geometry unchanged, so the baselines repeat their unchanged values there.
Calibration errors distort the frame itself, so they cost every method alike. With a focal length 10% too long, Fast localizes 30.3% of the frames within 0.25 m and 2°, GeDi 29.5% and BUFFER-X 30.3%, against 86.9% for Fast on unchanged frames. The place is still found: within 0.5 m and 5°, Fast keeps at least 90.3% at every focal error. The scaled frame cannot fit the point cloud closely, and the second refinement stage of Precise ends below Fast at all 4 focal errors. Distortion costs Fast at most 8.0 points. In 16 of 16 comparisons over the focal and distortion levels, the losses of Fast and of a baseline are equivalent within 3 points (§ 4.1). A color offset of up to 10 pixels changes Fast by at most 0.5 points.
A narrow lens separates the methods. A 30° lens keeps 25% of the frame’s points. Fast then leads GeDi by 6.2 points [+2.7, +10.1] and BUFFER-X by 9.6 [+5.6, +13.8].
References
- [1]Sheng Ao, Qingyong Hu, Hanyun Wang, Kai Xu, and Yulan Guo. BUFFER: balancing accuracy, efficiency, and generalizability in point cloud registration. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023, Vancouver, BC, Canada, June 17-24, 2023, pages 1255–1264. IEEE, 2023. 2DOI
- [2]Relja Arandjelović, Petr Gronat, Akihiko Torii, Tomas Pajdla, and Josef Sivic. NetVLAD: CNN architecture for weakly supervised place recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, June 27-30, 2016, pages 5297–5307. IEEE Computer Society, 2016. 3
- [3]Paul J. Besl and Neil D. McKay. A method for registration of 3-D shapes. IEEE Trans. Pattern Anal. Mach. Intell., 14(2):239–256, 1992. 2DOI
- [4]Eric Brachmann and Carsten Rother. Visual camera relocalization from RGB and RGB-D images using DSAC. IEEE Trans. Pattern Anal. Mach. Intell., 44(9), 2022. 2, 6, 13DOI
- [5]Eric Brachmann, Martin Alzinger, Carsten Rother, and Torsten Sattler. On the limits of pseudo ground truth in visual camera re-localisation. In 2021 IEEE/CVF International Conference on Computer Vision, ICCV 2021, Montreal, QC, Canada, October 10-17, 2021, pages 6198–6208. IEEE, 2021. 2, 4, 6, 13arxiv.org/abs/2109.00524
- [6]Eric Brachmann, Tommaso Cavallari, and Victor Adrian Prisacariu. Accelerated coordinate encoding: Learning to relocalize in minutes using RGB and poses. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023, Vancouver, BC, Canada, June 17-24, 2023, pages 5044–5053. IEEE, 2023. 2, 5, 6, 13arxiv.org/abs/2305.14059
- [7]Monica S. Castelhano and John M. Henderson. The influence of color on the perception of scene gist. Journal of Experimental Psychology: Human Perception and Performance, 34(3):660–675, 2008. 12
- [8]Yang Chen and Gérard G. Medioni. Object modelling by registration of multiple range images. Image Vis. Comput., 10 (3):145–155, 1992. 2
- [9]Zhi Chen, Kun Sun, Fan Yang, and Wenbing Tao. SC²-PCR: a second order spatial compatibility for efficient and robust point cloud registration. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022, New Orleans, LA, USA, June 18-24, 2022, pages 13211–13221. IEEE, 2022. 3
- [10]Angela Dai, Angel X. Chang, Manolis Savva, Maciej Halber, Thomas A. Funkhouser, and Matthias Nießner. ScanNet: Richly-annotated 3D reconstructions of indoor scenes. In 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, July 21-26, 2017, pages 2432–2443. IEEE Computer Society, 2017. 4arxiv.org/abs/1702.04405
- [11]Siyan Dong, Shuzhe Wang, Shaohui Liu, Lulu Cai, Qingnan Fan, Juho Kannala, and Yanchao Yang. Reloc3r: Large-Scale training of relative camera pose regression for generalizable, fast, and accurate visual localization. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2025, Nashville, TN, USA, June 11-15, 2025, pages 16739–16752. Computer Vision Foundation / IEEE, 2025. 2, 5arxiv.org/abs/2412.08376
- [12]Russell A. Epstein. Parahippocampal and retrosplenial contributions to human spatial navigation. Trends in Cognitive Sciences, 12(10):388–396, 2008. 12DOI
- [13]Russell A. Epstein and Lindsay K. Vass. Neural systems for landmark-based wayfinding in humans. Philosophical Transactions of the Royal Society B: Biological Sciences, 369(1635):20120533, 2014. 12DOI
- [14]Martin A. Fischler and Robert C. Bolles. Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM, 24(6):381–395, 1981. 2
- [15]Justin Gilmer, Samuel S. Schoenholz, Patrick F. Riley, Oriol Vinyals, and George E. Dahl. Neural message passing for quantum chemistry. In Proceedings of the 34th International Conference on Machine Learning, ICML 2017, 2017. 3arxiv.org/abs/1704.01212
- [16]Michelle R. Greene and Aude Oliva. The briefest of glances: The time course of natural scene understanding. Psychological Science, 20(4):464–472, 2009. 12pubmed.ncbi.nlm.nih.gov/19399976
- [17]Linda Hermer and Elizabeth S. Spelke. A geometric process for spatial reorientation in young children. Nature, 370(6484):57–59, 1994. 12pubmed.ncbi.nlm.nih.gov/8015605
- [18]Shengyu Huang, Zan Gojcic, Mikhail Usvyatsov, Andreas Wieser, and Konrad Schindler. PREDATOR: registration of 3D point clouds with low overlap. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2021, virtual, June 19-25, 2021, pages 4267–4276. Computer Vision Foundation / IEEE, 2021. 2, 5, 6, 13, 14arxiv.org/abs/2011.13005
- [19]Nikhil Keetha, Norman Müller, Johannes Schönberger, Lorenzo Porzi, Yuchen Zhang, Tobias Fischer, Arno Knaptisch, Duncan Zauss, Ethan Weber, Nelson Antunes, Jonathon Luiten, Manuel Lopez-Antequera, Samuel Rota Bulò, Christian Richardt, Deva Ramanan, Sebastian Scherer, and Peter Kontschieder. MapAnything: Universal feed-forward metric 3D reconstruction. arXiv preprint arXiv:2509.13414, 2025. 3arxiv.org/abs/2509.13414
- [20]Giseop Kim and Ayoung Kim. Scan context: Egocentric spatial descriptor for place recognition within 3D point cloud map. In IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 4802–4809, 2018. 3
- [21]Minjung Kim, Junseo Koo, and Eunhee Kim. EP2P-Loc: End-to-end 3D point to 2D pixel localization for large-scale visual localization. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 21470–21480, 2023. 3arxiv.org/abs/2309.07471
- [22]Vincent Leroy, Yohann Cabon, and Jérôme Revaud. Grounding image matching in 3D with MAST3R. In Computer Vision - ECCV 2024 - 18th European Conference, Milan, Italy, September 29-October 4, 2024, Proceedings, Part LXXII, pages 71–91. Springer, 2024. 3, 5arxiv.org/abs/2406.09756
- [23]Jiaxin Li Lai and Gim Hee Lee. DeepI2P: Image-to-point cloud registration via deep classification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15955–15964, 2021. 3arxiv.org/abs/2104.03501
- [24]Steven A. Marchette, Lindsay K. Vass, Jack Ryan, and Russell A. Epstein. Anchoring the neural compass: Coding of local spatial reference frames in human medial parietal lobe. Nature Neuroscience, 17(11):1598–1606, 2014. 12DOI
- [25]Juncheng Mu, Lin Bie, Shaoyi Du, and Yue Gao. ColorPCR: color point cloud registration with Multi-Stage Geometric-Color fusion. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2024, Seattle, WA, USA, June 16-22, 2024, pages 21061–21070. IEEE, 2024. 2, 5, 6, 13, 14DOI
- [26]Richard A. Newcombe, Shahram Izadi, Otmar Hilliges, David Molyneaux, David Kim, Andrew J. Davison, Pushmeet Kohli, Jamie Shotton, Steve Hodges, and Andrew Fitzgibbon. KinectFusion: Real-time dense surface mapping and tracking. In IEEE International Symposium on Mixed and Augmented Reality, pages 127–136, 2011. 3
- [27]Aude Oliva and Philippe G. Schyns. Diagnostic colors mediate scene recognition. Cognitive Psychology, 41(2):176–210, 2000. 2, 12pubmed.ncbi.nlm.nih.gov/10968925
- [28]Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy V. Vo, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, Mahmoud Assran, Nicolas Ballas, Wojciech Galuba, Russell Howes, Po-Yao Huang, Shang-Wen Li, Ishan Misra, Michael Rabbat, Vasu Sharma, Gabriel Synnaeve, Hu Xu, Hervé Jégou, Julien Mairal, Patrick Labatut, Armand Joulin, and Piotr Bojanowski. DINOv2: learning robust visual features without supervision. Trans. Mach. Learn. Res., 2024, 2024. 3, 6arxiv.org/abs/2304.07193
- [29]Emanuele Palazzolo, Jens Behley, Philipp Lottes, Philippe Giguère, and Cyrill Stachniss. ReFusion: 3D reconstruction in dynamic environments for RGB-D cameras exploiting residuals. In IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 7855–7862, 2019. 8, 15arxiv.org/abs/1905.02082
- [30]Vojtech Panek, Zuzana Kukelova, and Torsten Sattler. MeshLoc: Mesh-Based visual localization. In Computer Vision - ECCV 2022 - 17th European Conference, Tel Aviv, Israel, October 23-27, 2022, Proceedings, Part XXII, pages 589–609. Springer, 2022. 2arxiv.org/abs/2207.10762
- [31]Jaesik Park, Qian-Yi Zhou, and Vladlen Koltun. Colored point cloud registration revisited. In IEEE International Conference on Computer Vision, ICCV 2017, Venice, Italy, October 22-29, 2017, pages 143–152. IEEE Computer Society, 2017. 2DOI
- [32]Quang-Hieu Pham, Mikaela Angelina Uy, Binh-Son Hua, Duc Thanh Nguyen, Gemma Roig, and Sai-Kit Yeung. LCD: Learned cross-domain descriptors for 2D-3D matching. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 11856–11864, 2020. 3arxiv.org/abs/1911.09326
- [33]Fabio Pioesi and Davide Boscaıni. Learning general and distinctive 3D local deep descriptors for point cloud registration. IEEE Trans. Pattern Anal. Mach. Intell., 45(3), 2023. 2, 5, 6, 13, 14, 16, 17
- [34]Zheng Qin, Hao Yu, Changjian Wang, Yulan Guo, Yuxing Peng, and Kai Xu. Geometric transformer for fast and robust point cloud registration. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022, New Orleans, LA, USA, June 18-24, 2022, pages 11133–11142. IEEE, 2022. 2, 5, 6, 13, 14arxiv.org/abs/2202.06688
- [35]Jérôme Revaud, Yohann Cabon, Romain Brégier, Jong-Min Lee, and Philippe Weinzaepfel. SACReg: Scene-Agnostic coordinate regression for visual localization. CoRR, abs/2307.11702, 2023. 2arxiv.org/abs/2307.11702
- [36]Radu Bogdan Rusu, Nico Blodow, and Michael Beetz. Fast point feature histograms (FPFH) for 3D registration. In 2009 IEEE International Conference on Robotics and Automation, ICRA 2009, Kobe, Japan, May 12-17, 2009, pages 3212–3217. IEEE, 2009. 2, 5, 6, 13, 14DOI
- [37]Paul-Edouard Sarlin, Cesar Cadena, Roland Siegwart, and Marcin Dymczyk. From coarse to fine: Robust hierarchical localization at large scale. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019, pages 12708–12717. Computer Vision Foundation / IEEE, 2019. 3
- [38]Philippe G. Schyns and Aude Oliva. From blobs to boundary edges: Evidence for time- and spatial-scale-dependent scene recognition. Psychological Science, 5(4):195–200, 1994. 12DOI
- [39]Minkyun Seo, Hyungtae Lim, Kanghee Lee, Luca Carlone, and Jaesik Park. BUFFER-X: towards Zero-Shot point cloud registration in diverse scenes. In IEEE/CVF International Conference on Computer Vision, ICCV 2025, Honolulu, HI, USA, October 19-25, 2025, pages 3851–3862. IEEE, 2025. 2, 5, 6, 13, 14, 16, 17arxiv.org/abs/2503.07940
- [40]Jamie Shotton, Ben Glocker, Christopher Zach, Shahram Izadi, Antonio Criminisi, and Andrew W. Fitzgibbon. Scene coordinate regression forests for camera relocalization in RGB-D images. In 2013 IEEE Conference on Computer Vision and Pattern Recognition, Portland, OR, USA, June 23-28, 2013, pages 2930–2937. IEEE Computer Society, 2013. 2, 4DOI
- [41]Shitao Tang, Chengzhou Tang, Rui Huang, Siyu Zhu, and Ping Tan. Learning camera localization via dense scene matching. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2021, virtual, June 19-25, 2021, pages 1831–1841. Computer Vision Foundation / IEEE, 2021. 2arxiv.org/abs/2103.16792
- [42]Mikaela Angelina Uy and Gim Hee Lee. PointNetVLAD: Deep point cloud based retrieval for large-scale place recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 4470–4479, 2018. 3arxiv.org/abs/1804.03492
- [43]Julien P. C. Valentin, Angela Dai, Matthias Nießner, Pushmeet Kohli, Philip H. S. Torr, Shahram Izadi, and Cem Keskin. Learning to navigate the energy landscape. In Fourth International Conference on 3D Vision, 3DV 2016, Stanford, CA, USA, October 25-28, 2016, pages 323–332. IEEE Computer Society, 2016. 2, 4
- [44]Shuzhe Wang, Vincent Leroy, Yohann Cabon, Boris Chidlovskii, and Jérôme Revaud. DUSt3R: geometric 3D vision made easy. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2024, Seattle, WA, USA, June 16-22, 2024, pages 20697–20709. IEEE, 2024. 3arxiv.org/abs/2312.14132
- [45]Yue Wang, Yongbin Sun, Ziwei Liu, Sanjay E. Sarma, Michael M. Bronstein, and Justin M. Solomon. Dynamic graph CNN for learning on point clouds. ACM Trans. Graph., 38(5):146:1–146:12, 2019. 2arxiv.org/abs/1801.07829
- [46]Heng Yang, Jingnan Shi, and Luca Carlone. TEASER: fast and certifiable point cloud registration. IEEE Trans. Robotics, 37(2):314–333, 2021. 2, 5, 6, 13, 14arxiv.org/abs/2001.07715
- [47]Luwei Yang, Ziqian Bai, Chengzhou Tang, Honghua Li, Yasutaka Furukawa, and Ping Tan. SANet: scene agnostic network for camera localization. In 2019 IEEE/CVF International Conference on Computer Vision, ICCV 2019, Seoul, Korea (South), October 27 - November 2, 2019, pages 42–51. IEEE, 2019. 2DOI
- [48]Hao Yu, Zheng Qin, Ji Hou, Mahdi Saleh, Dongsheng Li, Benjamin Busam, and Slobodan Ilic. Rotation-Invariant transformer for point cloud matching. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023, Vancouver, BC, Canada, June 17-24, 2023, pages 5384–5393. IEEE, 2023. 2, 5, 6, 13, 14arxiv.org/abs/2303.08231
- [49]Andy Zeng, Shuran Song, Matthias Nießner, Matthew Fisher, Jianxiong Xiao, and Thomas A. Funkhouser. 3DMatch: learning local geometric descriptors from RGB-D reconstructions. In 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, July 21-26, 2017, pages 199–208. IEEE Computer Society, 2017. 2, 5arxiv.org/abs/1603.08182



