WorksComputer Science
Improving Zero-shot 3D Semantic Mapping in Offroad with Geometric Priors
Abstract
Semantic 3D mapping is a powerful representation that enables platform agnostic mobile robot navigation in challenging offroad conditions. Current vision foundation models can accurately predict semantics zero-shot on structured scenes but performance is compromised in natural environments. On the other hand, 3D point clouds encode valuable information for discriminating semantics in scenarios where vision foundation models fail. In this work, we propose fitting a sphericity-elevation distribution for each semantic class to act as geometric anchor. Only the points with the most confident vision foundation model predictions are considered when fitting to produce high quality geometric prior distributions. The geometric prior is fused with the vision foundation model prediction to produce the final semantic mapping. The proposed prior improves SAM3 baseline by +6.3 in mIoU and +8.9 in accuracy on the RELLIS-3D dataset, and by +1.7 in mIoU and +2.1 in accuracy on the WildScenes dataset. The source code will be made public upon paper acceptance.
Introduction
Semantic understanding of the environment is crucial for autonomous navigation in offroad conditions. Most offroad research concentrates on traversability estimation that reduces the rich geometry and semantic information to single dimension [1], [2]. These methods commonly classify any elevated 3D points as non-traversable and cannot accurately reason about the passability of different types of vegetation. Semantic 3D perception would provide more flexible foundation for offroad navigation that can be easily adapted to different scenarios and platforms.
While the current foundation models for general purpose segmentation [3], [4] achieve impressive accuracy on structured environments their performance is significantly lower on unstructured natural scenes. Separating different kinds of vegetation and terrain from images is one of the major challenges as there are no clear visual boundaries. At the same time correct classification of different vegetation and terrain types is critical for succesful navigation decisions as they cover significant portion of natural scenes and have different traversability properties.
Different terrain and vegetation types typically have distinctive geometric features that can help image based foundation models discriminate between them. We propose utilizing elevation and sphericity computed from lidar point clouds for this task. Elevation separates the terrain from vegetation and obstacles above it. Sphericity discriminates between different terrain and vegetation types based on their local shape. We fit an elevation-sphericity histogram for each semantic class
Figure 1. The proposed method improves semantic mapping by fusing the vision foundation model prediction with a geometric prior computed from the pointcloud.
based on the most confident detections from the vision foundation model to act as a geometric anchor. We then place prior for each point by comparing its elevation and sphericity to the fitted histograms. This prior is fused with the vision foundation model prediction to produce the final estimate. The proposed method yields significant improvements over vision foundation model baselines when evaluated with two real-world offroad datasets: Rellis3d [5] and WildScenes [6]. The main contributions are:
Novel elevation-sphericity histogram based prior for 3D semantic segmentation in offroad environments.
Demonstration of significant improvements in both mIoU and accuracy with real-world offroad datasets.
Open-source implementation to support future development
Related work
Vision foundation models for zero-shot semantic segmentation
Historically, segmentation models have been trained on task- and domain-specific datasets to achieve reliable performance within that domain. The current trend is to pretrain large vision models on massive, diverse datasets using self-supervised, weakly supervised, or multimodal objectives, and then adapt them to a variety of downstream tasks and domains with minimal additional training. These models are referred to as vision foundation models. Thanks to extensive pre-training many vision foundation models can be used in zero-shot manner, meaning they require no training in the target domain to perform inference. Variety of vision foundation models have been proposed in recent years. DINO [7] and EVA [8] are popular examples of feature extraction backbones that can be fine tuned for variety of downstream tasks. SAM [9] and Grounded SAM [10] are designed for the segmentation task. Large vision language models like Gemini [11] and Qwen VL [12] support image inputs but they are trained for higher level image understanding and reasoning not pixel level segmentation task.
Another recent trend has been the development of open vocabulary models to allow segmenting images with arbitary prompts. SAM3 [3] added support to query masks with short open vocabulary concepts. DINO.txt [13] aligns dense DINO features with text prompt encodings to to allow open vocabulary segmentation. OpenSeg [4] employs a separate mask proposal network that generates class agnostic masks proposals that are then classified based on the feature similarity with the encoded text prompts.
Zero-shot segmentation in 3D
Traditional supervised 3D segmentation networks are trained to produce semantic labels from the pointcloud with manually labeled dataset. For zero-shot methods a common approach is to predict features or segmentation masks from the image with vision foundation models and lift them to 3D with known calibration [14, 15]. Image based foundation models provide better zero-shot open vocabulary performance thanks to better training data availability, most importantly image-caption pairs. Image foundation models can also serve as supervision to distill 3D prediction models [16, 17] enabling zero-shot open-vocabulary segmentation from 3D inputs alone, while also allowing image and 3D modalities to be fused for improved performance. However, the performance gains are limited as the 3D model relies on the image model for supervision leading to correlated outputs.
Zero-shot segmentation in offroad
Offroad conditions pose spesific challenges to zero-shot segmentation in both 3D and image modalities. Images lack easily identifiable salient objects and most foundation model training data is from structured scenes which leads to decreased performance in offroad environments. In 3D, points don’t necessarily form dense easily identifiable clusters as in the common 3D segmentation benchmarks like ScanNet [18] and Matterport3D [19]. Instead the point density is singificantly lower and shapes more irregular.
Very limited research has been conducted to solve zero-shot semantic segmentation in offroad conditions. Instead, most works concentrate on estimating just the traversability in a self-supervised manner [1, 2]. However, such approaches can’t effectively handle scenarios where the platform needs to navigate over variety of terrain, obstacles and vegetation with different passability. 3D semantic maps are also inherently platform agnostic unlike most self-supervised traversability estimation methods. Traversability is property tied to a specific platform while 3D semantic mapping is an objective description of the environment.
Best of our knowledge OTAS [20] is the only work aiming to specifically address the issue of zero-shot semantic segmentation in offroad conditions. They demonstrate that dense CLIP [21] features commonly used for open vocabulary segmentation are noisy in the offroad scenario. To solve this problem they extract dense DINOv2 features that are visually more robust and cluster them to form mask proposals. These mask proposals are then classified using their average CLIP feature. In the 3D case the point coordinates are concatenated with the features before clustering to encourage spatial coherency.
Geometric features in offroad robotics
Geometric features have been long used for evaluating the environment properties in offroad conditions. Common features are step-height, roughness and slope [22]. These features work well in solid terrain, but not when passing through vegetation is required. High vegetation typically has high roughness and step height leading to classification as obstacle. Utilization of linearity, planarity and sphericity features is one proposed solution to this problem [23, 24]. Vegetation typically has high sphericity, whereas terrain and structures exhibit high planarity, and thin obstacles high linearity.
Research Gap
We improve zero-shot 3D semantic mapping by utilizing geometric priors based on elevation and sphericity extracted from lidar point clouds. Refrence elevation-sphericity histogram is fitted for each class utilizing the points with most confident vision foundation model predictions. The fitting process is lightweight and the histogram describes the geometrical separation of different classes in human understandable form. In existing works, OTAS utilizes 3D coordinates to improve spatial segmentation coherency but ignores other cues available from geometry [20]. On the other hand, distilling 3D networks with image based foundation model supervision requires environment specific neural network training that is time and computation heavy. [16, 17].
Methods
Given a synced lidar scan-image sequence our method generates a semantic pointcloud map of the sequence. The method is divided to four parts. First, confidence score maps are generated for each class using an open vocabulary vision foundation model (Section III-A). Then sphericity and elevation are computed for the 3D point cloud map accumulated from the posed scans (Section III-B). Then sphericity-elevation histogram is fitted using the points with most confident vision foundation model predictions (Section III-C). Finally, the geometric prior is queried from the fitted histograms and fused with the vision foundation model confidence maps to produce the final segmentation output (Section III-D).
Zero-shot open vocabulary image segmentation
We employ an open vocabulary vision foundation model for the image segmentation task. Given an open vocabulary prompt the model outputs a score for each pixel or image patch describing the correspondence with that prompt. To define a class wise score distribution each image is prompted with all the class names in the dataset yielding confidence score map at each timestep , where , is the image size and number of semantic classes. We then project the lidar scan from timestep to the image frame based on known intrinsic and extrinsic calibration and retrieve the semantic information from the corresponding pixels, yielding pointwise confidence score map , where is the number of points in the scan. Points that are outside the camera field of view are discarded. Points are accumulated across the sequence based on known poses to form a confidence score point cloud map , where is total number of scan points in the sequence. A corresponding 3D cloud map is also formed.
Figure 2. Geometric prior computation and utilization illustrated. The most confident image segmentation predictions are used to fit the sphericity-elevation distributions for each class, which are then used to assign prior weights for each point in the fusion phase.
Sphericity and elevation computation
Sphericity and elevation are both computed over the accumulated point cloud . The computations are performed on 0.1 m voxel resolution to save computation and results are upscaled back to the full resolution yielding elevation map and sphericity map . Elevation and sphericity maps are visualized in Figure 3.

Figure 3. Sphericity and elevation visualized. Tree trunks and even ground have low sphericity and tree foliage and vegetation high sphericity.
The relative elevation of a point p is defined as the difference between its height and the minimum height within its local neighborhood
where denotes the local neighborhood comprising all points within a horizontal radius m of p. The orientation of the z-coordinate is aligned with the normal vector of the global point cloud’s principal plane, determined via principal component analysis.
To compute sphericity, we first construct the 3D covariance matrix over the local 3D neighborhood , which includes all points within a sphere of radius m centered at p:
where is the number of points in the neighborhood and is their centroid. Let denote the eigenvalues of . The sphericity of point p is then defined as:
Fitting class histograms
We model the geometric properties of each class with 2D histograms that can accurately capture arbitrary shapes unlike parametric distributions. For elevation, we define non-uniform bins utilizing a 0.1 m bin spacing for values between 0–1 m, and a 1.0 m bin spacing for values exceeding 1 m. For sphericity, we define uniform bins with a 0.05 spacing across the entire range. To fit the distribution for a given class , we utilize only the most confident predictions to avoid incorporating noisy samples. Let denote this high-confidence subset, consisting of the top 5% most confident points among all points where class is the highest-scoring prediction from the vision foundation model. Let be the elevation bin index and be the sphericity bin index. The unnormalized occurrence count for class is computed by accumulating the points in that fall into the respective bins:
where is the indicator function, and is the initial count added to every bin to avoid zero counts that veto out that class. Finally, we normalize the accumulated counts to sum to one across the histogram:
where and iterate over all elevation and sphericity bins, respectively. The fitted histograms for each class in the Rellis3D and WildScenes datasets are presented in Figure 4 and Figure 5.
Figure 4. Elevation-sphericity histograms fitted on the top 5 % SAM3 detections on Rellis3D. Number of samples used for fitting each class in parentheses.
Figure 5. Elevation-sphericity histograms fitted on the top 5 % SAM3 detections on WildScenes. Mud excluded due to insufficient number of points. Number of samples used for fitting each class in parentheses.
Fusion
We can construct a geometric prior matrix for by querying the normalized class histogram at each points and each class :
where and are the bin indeces of the point’s elevation and sphericity respectively and is the points index.
If both distributions and were calibrated likelihoods, their direct product would yield a standard Bayesian fused estimate. Because strict calibration cannot be assumed, we introduce a scalar smoothing exponent to control the weighting between the geometry and vision modalities. The fused score matrix is computed via the element-wise product:
where denotes element-wise multiplication, and denotes element-wise exponentiation. Setting corresponds to standard Bayesian fusion, whereas ignores the geometric prior entirely.
We apply the prior only to the set of well-sampled classes whose histograms exceed 1,000 point samples. While rare classes retain their original scores from , classes in are fused via Equation (7) and rescaled to preserve their original collective score sum in . Consequently, the prior redistributes confidence exclusively among classes in and rest remain unchanged.
The final predicted class for each point is extracted by taking the row-wise argument maximum, yielding a class assignment vector :
Finally, the semantically labeled point cloud is downsampled to a 0.2 m voxel resolution, where each voxel is assigned the class that receives the majority vote of all points in the voxel.
Validation
We evaluate the proposed method using three zero-shot, open-vocabulary image segmentation models: DINOv2.txt, SAM 3, and OpenSeg. Given a text prompt, SAM 3 outputs a set of mask proposals, each paired with a mask score in the range representing the model’s confidence in that mask, as well as a pixel-wise score in indicating the confidence that each pixel belongs to that mask. We define the final per-pixel confidence score as the product of the global mask score and the pixel-wise mask score. DINOv2.txt employs a ViT-L/14 backbone with registers alongside its corresponding vision head and text encoder. OpenSeg uses an EfficientNet-B7 backbone paired with CLIP ViT-L/14@336px text embeddings, following OpenScene [16]. For both DINOv2.txt and OpenSeg, we scale the raw cosine similarities between the text prompt embeddings and visual embeddings by a temperature of before applying a softmax function, consistent with the CLIP training recipe. All models operate at their respective recommended input resolutions: SAM 3 inputs are stretched directly to px, OpenSeg inputs are resized to a 640 px long side while preserving the aspect ratio, with the short side padded to 640 px and DINOv2.txt inputs are resized to a 224 px short side while preserving the aspect ratio.
Additionally, we compare against OTAS, which is specifically designed for zero-shot 3D semantic segmentation in unstructured environments. For each feature patch, we assign the median depth of all LiDAR scan points projecting into that patch. Following the authors’ recommended configuration, we use the spatial variant of OTAS with 24 clusters, 24 components, a DINOv2 ViT-S/14 backbone with registers, and a 224 px input resolution, setting the voxel size to 0.2 m to match our method’s resolution. The reconstructed map is queried with every dataset class prompt, and each 3D point is assigned the class yielding the highest similarity score. To keep GPU memory consumption manageable, the map is processed in chunks of 500 samples.
Data
We evaluate our method using the WildScenes [6] and Rellis3D [5] datasets. Both include image-lidar scan pairs with known calibration and ground truth for pointcloud semantics. We only consider the lidar scan points that are inside the camera field of view as all evaluated methods require image data in addition to the 3D points. Lidar points further than 25 m are filtered out as alignment errors increase and sensor resolution decreases with longer distances.
WildScenes is collected on Australian forest trails surrounded by high tree canopies. We exclude ambiguous classes: other-terrain, other-object and unlabelled. We also exclude sky and water as these labels are not present in the 3D data.
RELLIS3D is collected on rural fields and trails in Texas dominated by grass, bushes and smaller trees. We exclude ambiguous classes void and object. We exclude sky as it is not present in the 3D data and person and vehicle as they are dynamic. We also exclude building and dirt as no points belonging to these classes are retained with the camera fov filtering with 25 m max distance used here. The poses were computed with kiss-icp [25].
Results
Accuracy, mIoU and class wise IoU are reported for each model on Rellis3D dataset (Table I) and WildScenes dataset (Table II). The geometry prior improves performance for all benchmarked zero-shot segmentation models. SAM3 is the best performing baseline model achieving 28.1 mIoU and 73.9 accuracy on Rellis3d and 47.9 mIoU and 80.5 accuracy on WildScenes. With the geometry prior SAM3 achieves +6.3 mIoU and +8.9 accuracy increase on Rellis3d and +1.7 mIoU and +2.1 accuracy increase on WildScenes. On Rellis3d the classes with highest gains are Bush (+44.2), Grass (+8.8), Puddle (+5.2) and Water (+28.7) and on WildScenes Bush (+3.8), Dirt (+7.9) and Grass (+9.2). OTAS often confuses similar classes, like dirt and gravel or tree-foliage and tree-trunk and achieves slightly lower accuracy than the DINOv2.txt and OpenSeg baselines. Qualitative examples for OTAS, SAM3 and SAM3 with geometric priors are presented in Tables III and IV.
| Method | mIoU | Acc | Asphalt | Barrier | Bush | Concrete | Fence | Grass | Log | Mud | Pole | Puddle | Rubble | Tree | Water |
| OTAS | 20.1 | 69.6 | 0.0 | 5.2 | 9.6 | 36.6 | 5.4 | 68.7 | 0.1 | 21.9 | 0.0 | 20.8 | 22.6 | 70.8 | 0.0 |
| SAM 3 | 28.1 | 73.9 | 0.0 | 68.7 | 7.2 | 42.7 | 5.8 | 73.4 | 12.1 | 30.2 | 0.3 | 28.5 | 27.1 | 68.8 | 0.0 |
| + OURS | 34.4 | 82.8 | 0.0 | 68.7 | 49.4 | 42.2 | 4.6 | 82.2 | 12.6 | 30.6 | 0.2 | 33.7 | 26.9 | 67.5 | 28.7 |
| DINOv2.txt | 21.0 | 65.5 | 0.0 | 18.7 | 41.0 | 43.2 | 3.1 | 65.5 | 9.7 | 16.9 | 0.7 | 18.3 | 37.0 | 19.2 | 0.0 |
| + OURS | 22.5 | 68.0 | 0.0 | 18.3 | 46.5 | 44.4 | 3.3 | 67.4 | 9.1 | 17.2 | 0.6 | 18.6 | 37.2 | 30.3 | 0.0 |
| OpenSeg | 21.5 | 75.1 | 0.0 | 41.1 | 41.7 | 0.0 | 22.8 | 77.3 | 0.0 | 14.7 | 9.3 | 22.4 | 25.3 | 24.9 | 0.0 |
| + OURS | 23.1 | 77.4 | 0.0 | 50.1 | 47.4 | 0.0 | 23.1 | 79.3 | 0.0 | 15.5 | 5.9 | 21.7 | 25.6 | 32.3 | 0.0 |
Table I. Semantic mapping performance on Rellis3d dataset. Highest values bolded
| Method | mIoU | Acc | Bush | Dirt | Fence | Grass | Gravel | Log | Mud | Rock | Structure | Tree-Foliage | Tree-Trunk |
| OTAS | 14.5 | 22.1 | 3.4 | 15.3 | 7.5 | 40.9 | 5.5 | 0.0 | 0.0 | 0.1 | 69.0 | 0.4 | 16.9 |
| SAM 3 | 47.9 | 80.5 | 13.4 | 69.8 | 48.3 | 57.0 | 50.7 | 35.8 | 0.0 | 42.7 | 82.3 | 78.5 | 48.1 |
| + OURS | 49.6 | 82.6 | 17.2 | 77.7 | 44.0 | 67.8 | 50.8 | 34.9 | 0.0 | 40.3 | 83.9 | 79.8 | 49.1 |
| DINOv2.txt | 21.3 | 24.0 | 1.2 | 23.3 | 7.3 | 17.7 | 7.5 | 14.3 | 1.5 | 31.1 | 73.8 | 17.0 | 39.1 |
| + OURS | 23.4 | 33.2 | 1.4 | 27.1 | 6.5 | 22.6 | 8.0 | 14.3 | 1.3 | 31.6 | 74.1 | 29.3 | 41.4 |
| OpenSeg | 27.0 | 73.9 | 14.9 | 56.6 | 49.2 | 17.1 | 18.4 | 2.3 | 9.0 | 0.5 | 51.7 | 77.4 | 0.2 |
| + OURS | 27.9 | 74.4 | 14.2 | 58.5 | 48.9 | 19.4 | 18.4 | 1.2 | 8.4 | 0.6 | 59.0 | 78.5 | 0.4 |
Table II. Semantic mapping performance on WildScenes dataset. Highest values bolded
For further analysis, we ablate different smoothing factors and the performance of only sphericity or elevation based prior histograms (Table V). With both datasets the joint elevation-sphericity prior outperforms either component alone. Optimal smoothing factor depends on the quality of the image predictions as it controls how much weight the geometry is given over the image predictions. Smoothing factor 0.25 was found to provide the best compromise across both datasets and is used with all reported results here. In most cases, any choice of the smoothing factor provided significant improvements over the baseline.
| Method | Wildscenes mIoU | Wildscenes Acc | Rellis3d mIoU | Rellis3d Acc |
| SAM 3 | 47.9 | 80.5 | 28.1 | 73.9 |
| + SPHE | 48.9 | 81.5 | 31.4 | 78.3 |
| + SPHE | 49.2 | 81.6 | 32.4 | 81.8 |
| + SPHE | 47.8 | 80.2 | 32.5 | 82.5 |
| + SPHE | 43.5 | 77.1 | 30.8 | 80.3 |
| + ELEV | 49.0 | 81.6 | 30.8 | 77.4 |
| + ELEV | 48.2 | 81.3 | 33.0 | 81.2 |
| + ELEV | 45.8 | 79.9 | 34.5 | 82.6 |
| + ELEV | 38.8 | 76.7 | 34.2 | 82.8 |
| + BOTH | 49.6 | 82.4 | 32.4 | 78.7 |
| + BOTH | 49.6 | 82.6 | 34.4 | 82.8 |
| + BOTH | 47.4 | 81.6 | 35.5 | 83.9 |
| + BOTH | 37.8 | 79.1 | 33.7 | 82.6 |
| DINOv2.txt | 21.3 | 24.0 | 21.0 | 65.5 |
| + BOTH | 22.4 | 28.3 | 21.8 | 66.8 |
| + BOTH | 23.4 | 33.2 | 22.5 | 68.0 |
| + BOTH | 23.0 | 38.1 | 23.1 | 69.0 |
| + BOTH | 19.3 | 42.8 | 22.9 | 68.4 |
| OpenSeg | 27.0 | 73.9 | 21.5 | 75.1 |
| + BOTH | 27.3 | 74.1 | 22.1 | 76.3 |
| + BOTH | 27.9 | 74.4 | 23.1 | 77.4 |
| + BOTH | 28.8 | 74.6 | 24.0 | 78.5 |
| + BOTH | 29.1 | 75.0 | 24.5 | 79.6 |
Table V. Geometry prior ablations with different smoothing values for the geometric prior. ELEV defines the geometric prior based on elevation only, SPHE based on sphericity only and BOTH utilizes their joint distribution as presented in Section III-C. Highest values bolded.
Discussion
This paper introduced a method to improve zero-shot segmentation in offroad scenes with geometric priors computed from the 3D data. The geometric priors increased performance especially on natural classes like bush, grass, dirt, water and puddle while performance on structured classes was very little affected. Our explanation for this behavior is two-fold. First, the zero-shot vision foundation models are more prone to make mistakes on natural classes due to the limited training data from these domains, leaving more room for improvement. Second, the elevation-sphericity histogram has better separation between natural classes than between structured ones. In Figure 4, barrier, fence and pole all share
Figure III (PDF p. 6). Examples of semantic pointcloud map predictions from ego view.
Figure IV (PDF p. 7). Examples of semantic pointcloud map predictions from birds eye view.
quite similar distributions while grass, bush and tree are more clearly separated.
While deep learning models can extract more complex features from 3D pointclouds, the proposed approach has a few main advantages over them. Deep learning models require training on each new environment while the distribution fitting process is much faster and requires less computation. Additionally, the proposed approach is more transparent: the geometric separation between classes can be directly evaluated from the fitted histograms. Finally, for many natural classes elevation and sphericity provide sufficient separation when fused with the vision foundation model predictions as the 3D shapes are simpler compared to cluttered indoor scenes.
The performance of our method is limited by class separability within the histogram; consequently, classes sharing similar elevation and sphericity values cannot be distinguished. Furthermore, our approach relies on the vision foundation model to provide confident detections for each class in order to fit these distributions. Any bias or error on the vision foundation models detections affects the fitted histogram and consequentially the computed priors. In future work, more geometric features could be added to achieve more discriminative prior distributions or alternatively pre-trained point-encoders could be used to extract more complex point features. Another future challenge is calibrating the vision foundation model and geometric prior confidences for improved fusion and uncertainty estimation.
While histograms can capture arbitrary distribution shapes, they also introduce significant limitations. Specifically, a query bin’s prior depends solely on its own value, without accounting for neighboring bins. Consequently, an empty bin adjacent to high-density regions is treated identically to an empty bin in a sparse region. Future work could address this limitation by applying spatial smoothing to the histograms. Parameterized distributions, such as 2D Gaussians are also a potential solution but they also restrict the shape of the distribution.
Acknowledgments
Acknowledgments that can be used to identify the authors have been excluded from this version. Large language models (Claude 5 Sonnet and Opus) were used in the code implementation of the presented methodology, and experimental evaluation (Sections III and IV). The authors reviewed all AI-assisted content and take full responsibility for it.
References
References
- [1]S. Jung, J. Lee, X. Meng, B. Boots, and A. Lambert, “V-strong: Visual self-supervised traversability learning for off-road navigation,” in 2024 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2024, pp. 1766–1773.arxiv.org/abs/2312.16016
- [2]D. Kim, K. Park, J. Choi, J. Park, J. Park, and J. W. Choi, “Geo3tp: Learning 3d off-road traversability from semantic-geometric automatic annotation,” IEEE Robotics and Automation Letters, 2026.DOI
- [3]N. Carion, L. Gustafson, Y.-T. Hu, S. Debnath, R. Hu, D. Suris Coll-Vinent, C. Ryali, K. V. Alwala, H. Khedr, A. Huang et al., “Sam 3: Segment anything with concepts,” in International Conference on Learning Representations, vol. 2026, 2026, pp. 138 846–138 923.arxiv.org/abs/2511.16719
- [4]G. Ghiasi, X. Gu, Y. Cui, and T.-Y. Lin, “Scaling open-vocabulary image segmentation with image-level labels,” in European conference on computer vision. Springer, 2022, pp. 540–557.arxiv.org/abs/2112.12143
- [5]P. Jiang, P. Osteen, M. Wigness, and S. Saripalli, “Rellis-3d dataset: Data, benchmarks and analysis,” in 2021 IEEE international conference on robotics and automation (ICRA). IEEE, 2021, pp. 1110–1116.arxiv.org/abs/2011.12954
- [6]K. Vidanapathirana, J. Knights, S. Hausler, M. Cox, M. Ramezani, J. Jooste, E. Griffiths, S. Mohamed, S. Sridharan, C. Fookes et al., “Wildscenes: A benchmark for 2d and 3d semantic segmentation in large-scale natural environments,” The International Journal of Robotics Research, vol. 44, no. 4, pp. 532–549, 2025.DOI
- [7]M. Oquab, T. Darcet, T. Moutakanni, H. Vo, M. Szafraniec, V. Khalidov, P. Fernandez, D. Haziza, F. Massa, A. El-Nouby et al., “Dinov2: Learning robust visual features without supervision,” arXiv preprint arXiv:2304.07193, 2023.
- [8]Y. Fang, W. Wang, B. Xie, Q. Sun, L. Wu, X. Wang, T. Huang, X. Wang, and Y. Cao, “Eva: Exploring the limits of masked visual representation learning at scale,” in 2023 IEEE/CVF conference on computer vision and pattern recognition (CVPR). IEEE, 2023, pp. 19 358–19 369.arxiv.org/abs/2211.07636
- [9]A. Kirillov, E. Mintun, N. Ravi, H. Mao, C. Rolland, L. Gustafson, T. Xiao, S. Whitehead, A. C. Berg, W.-Y. Lo et al., “Segment anything,” in 2023 IEEE/CVF international conference on computer vision (ICCV). IEEE, 2023, pp. 3992–4003.arxiv.org/abs/2304.02643
- [10]T. Ren, S. Liu, A. Zeng, J. Lin, K. Li, H. Cao, J. Chen, X. Huang, Y. Chen, F. Yan et al., “Grounded sam: Assembling open-world models for diverse visual tasks,” arXiv preprint arXiv:2401.14159, 2024.arxiv.org/abs/2401.14159
- [11]G. Team, R. Anil, S. Borgeaud, J.-B. Alayrac, J. Yu, R. Soricut, J. Schalkwyk, A. M. Dai, A. Hauth, K. Millican et al., “Gemini: a family of highly capable multimodal models,” arXiv preprint arXiv:2312.11805, 2023.
- [12]S. Bai, Y. Cai, R. Chen, K. Chen, X. Chen, Z. Cheng, L. Deng, W. Ding, C. Gao, C. Ge et al., “Qwen3-vl technical report,” arXiv preprint arXiv:2511.21631, 2025.arxiv.org/abs/2511.21631
- [13]C. Jose, T. Moutakanni, D. Kang, F. Baldassarre, T. Darcet, H. Xu, D. Li, M. Szafraniec, M. Ramamonjisoa, M. Oquab et al., “Dinov2 meets text: A unified framework for image-and pixel-level vision-language alignment,” in 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2025, pp. 24 905–24 916.arxiv.org/abs/2412.16334
- [14]Y. Yang, X. Wu, T. He, H. Zhao, and X. Liu, “Sam3d: Segment anything in 3d scenes,” arXiv preprint arXiv:2306.03908, 2023.arxiv.org/abs/2306.03908
- [15]K. M. Jatavallabhula, A. Kuwajerwala, Q. Gu, M. Omama, T. Chen, A. Maalouf, S. Li, G. Iyer, S. Saryazdi, N. Keetha et al., “Conceptfusion: Open-set multimodal 3d mapping,” arXiv preprint arXiv:2302.07241, 2023.
- [16]S. Peng, K. Genova, C. Jiang, A. Tagliacacchi, M. Pollefeys, and T. Funkhouser, “Openscene: 3d scene understanding with open vocabularies,” in 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2023, pp. 815–824.arxiv.org/abs/2211.15654
- [17]X. Zhu, H. Zhou, P. Xing, L. Zhao, H. Xu, J. Liang, A. Hauptmann, T. Liu, and A. Gallagher, “Open-vocabulary 3d semantic segmentation with text-to-image diffusion models,” in European Conference on Computer Vision. Springer, 2024, pp. 357–375.arxiv.org/abs/2407.13642
- [18]A. Dai, A. X. Chang, M. Savva, M. Halber, T. Funkhouser, and M. Nießner, “ScanNet: Richly-annotated 3d reconstructions of indoor scenes,” in 2017 IEEE conference on computer vision and pattern recognition (CVPR). IEEE, 2017, pp. 2432–2443.arxiv.org/abs/1702.04405
- [19]A. Chang, A. Dai, T. Funkhouser, M. Halber, M. Niessner, M. Savva, S. Song, A. Zeng, and Y. Zhang, “Matterport3d: Learning from rgb-d data in indoor environments,” arXiv preprint arXiv:1709.06158, 2017.arxiv.org/abs/1709.06158
- [20]S. Schwaiger, S. Thalhammer, W. Wöber, and G. Steinbauer-Wagner, “Otas: Open-vocabulary token alignment for outdoor segmentation,” arXiv preprint arXiv:2507.08851, 2025.arxiv.org/abs/2507.08851
- [21]A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark et al., “Learning transferable visual models from natural language supervision,” in International conference on machine learning. PmLR, 2021, pp. 8748–8763.arxiv.org/abs/2103.00020
- [22]P. V. Borges, T. Peynot, S. Liang, B. Arain, M. Wildie, M. G. Minareci, S. Lichman, G. Samvedi, I. Sa, N. Hudson et al., “A survey on terrain traversability analysis for autonomous ground vehicles: Methods, sensors, and challenges,” Field Robotics, vol. 2, pp. 1567–1627, 2022.DOI
- [23]F. Ruetz, P. Borges, N. Suenderhauf, E. Hernández, and T. Peynot, “Forest traversability mapping (ftm): Traversability estimation using 3d voxel-based normal distributed transform to enable forest navigation,” in 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2022, pp. 8714–8721.DOI
- [24]M. Bellone, G. Reina, L. Caltagirone, and M. Wahde, “Learning traversability from point clouds in challenging scenarios,” IEEE Transactions on Intelligent Transportation Systems, vol. 19, no. 1, pp. 296–305, 2017.DOI
- [25]I. Vizzo, T. Guadagnino, B. Mersch, L. Wiesmann, J. Behley, and C. Stachniss, “Kiss-icp: In defense of point-to-point icp—simple, accurate, and robust registration if done the right way,” IEEE Robotics and Automation Letters, vol. 8, no. 2, pp. 1029–1036, 2023.DOI





