betaFeedback

WorksComputer Science

Segmenting Geometric Textures of 3D Meshes

Abstract

Recognizing geometric texture patterns on 3D models is critical for applications such as heritage protection, model editing, and 3D printing. Existing 3D model segmentation methods predominantly focus on identifying high-level semantic parts, thereby failing to capture middle-level textures. While recent Vision-Language Models can perform image segmentation with appropriate text prompts, they still struggle to recognize 3D geometric textures. In this work, we present a method that segments 3D mesh models based on middle-level geometric textures. We further introduce a new geometric texture segmentation benchmark and fine-tune SAM3 accordingly. The segmented texture regions are then clustered based on DINO feature similarity and lifted from 2D to 3D. Additionally, we develop an interactive tool to enable both 3D model editing based on our segmentation results and model retrieval from the dataset based on geometric and texture similarity. Experiments demonstrate that our method significantly outperforms existing methods in segmenting geometric textures and can facilitate various applications.

Introduction

Geometric textures/patterns constitute local, repetitive geometric variations across a 3D surface, remaining independent of the global model structure [BTB*18, PTB*25]. High-quality 3D models featuring intricate geometric textures are typically acquired via physical scanning or manual design (see Figure 2). These assets drive critical downstream applications, ranging from digital twins and heritage preservation to immersive gaming and 3D printing [47]. Effective recognition of geometric textures is therefore essential, facilitating cultural heritage analysis (e.g., re-assembling fragmented artifacts [BTB*18]) and enabling texture-aware 3D editing workflows. However, current practice relies heav-

High-quality 3D digital models with intricate geometric textures obtained from scanning and manual sculpting

Figure 2. High-quality 3D digital models with intricate geometric textures obtained from scanning (left) and manual sculpting (right).

© 2026 The Author(s).

Proceedings published by Eurographics - The European Association for Computer Graphics.

This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited. ily on manual inspection and laborious analysis by users or modelers, which is both time-consuming and inefficient.

However, existing model analysis and segmentation methods predominantly operate either at high-level semantics [HHF*19, VPP*23] or at low-level geometric features (e.g., curvature, normals, and noise) [45, 16], often failing to capture the intricate patterns of middle-level geometric textures. While professional tools like ZBrush facilitate interactive, local modifications, they lack the capability for holistic, global editing of regions characterized by specific texture patterns in a single pass. Consequently, despite growing research interest in recognizing and editing geometric textures on 3D surfaces [GDJ*25], a robust and comprehensive solution remains an open challenge.

In this work, we propose a framework for geometric texture recognition on 3D surface models. Leveraging a 2D-to-3D paradigm, our method begins by rendering multi-view images from the input 3D mesh. We then employ a visual segmentation model fine-tuned on our newly constructed benchmark to identify texture regions within the rendered images. Corresponding features are extracted from these segmented patches and clustered into semantic groups. Subsequently, we introduce a weighted voting strategy to fuse these 2D predictions and lift them into a coherent 3D segmentation. Finally, we adopt a filter-based geometric editing technique and develop an interactive tool that enables users to perform holistic, one-pass editing of specific texture patterns on the 3D model based on the segmentation results. The main contributions are summarized as follows:

  • A texture material dataset and a geometric texture segmentation benchmark. The datasets are publicly available at: https://github.com/ALingll/GTSeg;

  • A geometric texture segmentation method for 3D meshes, based on multi-view rendering and visual foundation models;

  • An interactive, texture-oriented 3D model editing tool, supporting model segmentation and retrieval as well as geometric texture enhancement, removal, and transfer.

Related Works

Geometric Texture over 3D model surfaces are high-frequency, repetitive surface details distinct from global shape and random noise (excluding hollowing and winding structures). Existing research on geometric textures mainly focused on synthesis and interactive editing. Early works established procedural and example-based methods for texture synthesis [2 Elb05, 60], a direction recently advanced by deep generative models [19, 51]. Complementarily, editing tools have evolved from manual manipulation of displacement maps [23] to recent texture generation based on the vision-language model [29]. However, as the prerequisite for automated analysis and editing, the fundamental problem of geometric texture recognition and segmentation is significantly less explored. Several studies have attempted to bridge this gap. One direction is to separate texture and non-texture regions using learning-based methods [GDJ*24, PTB24]. Tortorici et al. [40] introduced a convolution-based descriptor for recognizing user-specific textures.

Despite these efforts, current methods still exhibit poor generalization when dealing with multiple textures, multi-scale structures, and complex topologies. To the best of our knowledge, an open-source benchmark, a robust and automated solution for geometric texture segmentation and recognition that handles diverse and complex patterns over a complicated model, is still missing in the literature [BTB*18, PTB*25].

Model Segmentation is an essential task in computer graphics and vision. Early methods mainly rely on low-level geometric primitives such as curvature, surface normals, and geodesic distances to partition 3D models, including region growing [28], graph cuts [KT03], spectral clustering, and MRF/CRF-based optimization [38]. Although these methods perform stably on segmenting homogeneous low-level geometric regions (e.g., planar surfaces, smooth curved patches), they fail to capture high-frequency, repetitive geometric patterns that define geometric textures. Recent deep learning works focus on semantic or part-level segmentation. For native 3D representations, point cloud networks (e.g., PointNet [34, 35], DGCNN [46]) and mesh-based networks [HHF*19] achieve state-of-the-art performance on benchmarks such as ShapeNet Part [CFG*15] and PartNet [30]. However, these methods are limited to coarse-grained category or part labels (e.g., separating chair legs from seats) and cannot recognize fine-grained geometric textures [52, 48, 26]. Recent works following a 2D-to-3D pipeline, leverage pre-trained 2D foundation models (e.g., SAM [KMR*23], CLIP [CLZ*25]) to project multi-view 2D predictions back to 3D [59, 43, 41, 57]. While these methods enable open-vocabulary segmentation, they still focus on semantic object categories rather than geometric texture patterns.

To the best of our knowledge, the segmentation of geometric textures on 3D models remains an open problem [33]. Current attempts indicate that image-like local descriptors offer the most promising solution for geometric texture recognition and segmentation [BTB*18]. Hence, we borrow the ability from the visual foundation model and employ the 2D-to-3D pipeline to address this problem.

Image Segmentation aims to partition an image into regions that are coherent in semantics, instance identity, or visual attributes. Earlier deep segmentation methods primarily focus on dense prediction and instance mask modeling [25, 36, 17]. Recent segmentation research has increasingly shifted toward unified and promptable foundation models. MaskFormer and Mask2Former cast semantic, instance, and panoptic segmentation as mask classification [11, 9]. SAM3 introduces strong promptable concept segmentation, enabling the detection and segmentation of all instances matching a text or exemplar concept [CGH∗25]. SAMesh lifts multi-view SAM2 masks onto mesh surfaces for zero-shot mesh part segmentation [TZF∗24b]. These studies suggest that SAM-family models are effective front-ends for 3D model segmentation. Recently, textureSAM validates the zero-shot capability of SAM in 2D texture segmentation after fine-tuning with sufficient training data [10].

However, unlike the 2D textures provided by TextureSAM, the identical geometric texture can look dramatically different across viewpoints, which implies the critical role of proper training data and geometric constraints. Therefore, in this work, we do not directly treat 2D masks as the final predictions; instead, we leverage both image and geometry evidence, combining curvature re-rendering, mask-level texture-feature clustering, and multi-view 3D voting to obtain view-consistent geometric texture segmentation.

Geometric Texture Segmentation Benchmark

To begin, we construct a large-scale geometric texture segmentation benchmark, consisting of synthetic and annotated 3D models. Collecting reconstructed models through scanning is extremely expensive and labor-intensive. Instead, we choose to sculpt 3D models via texture displacement maps, see Figure 3.

Figure 3. The generation pipeline of our TDSet dataset and GTSeg benchmark.

Texture Displacement Maps Dataset. We use Chord [53] to generate displacement maps based on the open-source 2D texture images dataset DTD [8]. We construct a displacement map dataset, TDSet, consisting of 350 diverse displacement maps. The displacement maps in TDSet are employed to synthesize 3D geometric textures over 3D models.

3D Geometric Texture Benchmark. Next, we construct a geometric texture benchmark called GTSeg, which consists of a training set and a test set. For the training set, we collect 80 synthetic models from Thingi10K [58]. We randomly select four or five displacement maps from TDSet and apply them to the surface of each model. All geometric texture instances are automatically annotated. For each model, we repeat this process ten times. The final training set consists of 800 models, each with multiple geometric texture categories and instances. Moreover, we construct a test set comprising 20 synthetic models generated following the above procedure (with no overlap with the training set) and 10 real scans from [XZX∗14] and public domain. We manually annotate the geometric texture instances on the real scans.

Geometric Texture-oriented Image Segmentation Model

In this section, we develop a 2D image segmentation model specifically for segmenting 3D geometric textures. We first justify our choice of representation, and subsequently detail the training dataset construction pipeline and the model fine-tuning protocol.

Although recent advancements have made significant progress in texture-aware 2D image segmentation, they cannot be directly generalized to 3D geometric texture recognition. TextureSAM [10] mitigates the semantic bias of foundation models

by fine-tuning on 2D RGB images where object regions are synthetically replaced with descriptive visual textures. In contrast, 3D geometric textures lack explicit colors and depend solely on local topographic variations. Furthermore, the 2D projection of the same 3D geometric pattern exhibits severe perspective distortions and foreshortening across different viewing angles. Consequently, without explicit adaptation to intrinsic 3D geometry and multi-view consistency, models constrained to 2D visual priors inevitably fail to achieve view-consistent segmentation of complex 3D surface patterns.

Grayscale Curvature and Mask Rendering. Surface curvature is an intrinsic geometric property that remains strictly invariant to lighting conditions and rigid transformations. To bridge 3D geometry with 2D segmentation, we compute per-vertex mean curvature as the raw scalar sis_i. To mitigate the impact of mesh singularities and ensure high image contrast, we clip outliers according to the 5th/95th percentiles. The curvature values are then normalized to [0,1][0,1] via min-max normalization.

Here, we split the models in the training set of GTSeg into two sets: 76 models are used for training and 4 for validation. We rendered each model to generate curvature maps and ground-truth masks from 14 uniformly sampled viewpoints. That is to say, the training set contains 10,640 image and mask pairs, while the validation set contains 560 image and mask pairs. All rendered views and synthesized variants derived from the same source model are assigned to the same split to prevent data leakage. Finally, we generate a dataset of 11,200 pairs of view-dependent curvature maps and corresponding ground-truth masks. The training set will be employed to adapt the vision foundation model to our task later.

Model Fine-tuning. While the pre-trained SAM3 model demonstrates strong zero-shot capabilities in general concept segmentation, it struggles to precisely delineate non-semantic, mid-level geometric textures. To efficiently bridge this domain gap without catastrophic forgetting, we employ Low-Rank Adaptation (LoRA) [HSW∗22] to fine-tune the model on our synthesized dataset and obtain GTSAM.

The recognition of geometric textures relies heavily on mid-level patterns rather than high-level linguistic concepts. Hence, we freeze the entire language encoder to preserve general semantic priors and reduce computational overhead. To this end, we inject trainable rank-decomposition matrices into the Vision Encoder, Geometry Encoder, and the decoding modules. This targeted injection maximizes the model’s sensitivity to high-frequency textural signals while maintaining its robust zero-shot foundation.

Geometric Texture-oriented Model Segmentation

In this section, we introduce a framework to address the problem of geometric texture segmentation, via formulating it as a task of multi-view consistency perception. The overview of our framework is visualized in Figure 4.

Figure 4. Overview of our 3D geometric texture segmentation framework.

Given a 3D mesh M\mathcal{M}, we first render a set of grayscale curvature maps {Ci}i=1N\{\mathcal{C}_i\}_{i=1}^{N} from a collection of predefined camera poses {Pi}i=1N\{\mathcal{P}_i\}_{i=1}^{N}. Subsequently, the rendered images are fed to our GTSAM to predict a set of pixel-level geometric texture masks. We introduce a feature association module based on DINOv2 to maintain semantic consistency across disparate views. This module extracts and clusters high-dimensional feature descriptors from each mask, thereby establishing label correspondences across different viewpoints in a unified latent space. Finally, multi-view masks with consistent labels are projected onto the model surface M\mathcal{M}. To resolve prediction conflicts arising from occlusions or perspective distortions, we implement a spatial-consistency weighted voting mechanism. This mechanism performs per-face arbitration to make definitive label decisions, yielding precise and smooth geometric texture segmentation results.

Texture-oriented Image Segmentation

We set NN cameras to render the input mesh, ensuring complete coverage and sufficient overlap. In our implementation, we let N=50N = 50. The tt-th camera renders a curvature image ItI_t. Next, we send the rendered curvature images to our GTSAM. For ItI_t, GTSAM predicts a set of NtN_t instance masks. Each instance mask Mt,mM_{t,m} is paired with a confidence score ct,m∈[0,1]c_{t,m} \in[0,1].

For each predicted mask Mt,mM_{t,m}, we perform 8-connected component analysis. Components with an area smaller than the threshold Amin⁡A_{\min} are discarded. We retain those sufficiently large connected components and treat them as independent mask instances. This prevents disconnected 2D regions from being merged into the same texture cluster.

Feature-based Patch Clustering.

Although GTSAM produces high-quality class-agnostic masks {Mt,m}\{M_{t,m}\}, these masks are generated independently for each view and lack explicit semantic correspondence. To assign accurate instance labels to disparate views, we leverage DINOv2 as a robust feature extractor. It’s capable of achieving cross-view semantic consistency in the absence of per-view labels [DHM*25].

Region Re-rendering. After that, we project each mask component onto the mesh. The set of faces covered by the mask component CaC_a is denoted as PaP_a. We discard mask components that contain very few faces and store the remaining components as a group.

To extract a view-normalized texture descriptor, we re-render each projected region. The region center and average normal are estimated as:

oa=1∣V(Pa)∣∑vi∈V(Pa)vi,na=∑vi∈V(Pa)ni∥∑vi∈V(Pa)ni∥2(1)\mathbf{o}_a = \frac{1}{|\mathcal{V}(P_a)|}\sum_{\mathbf{v}_i \in\mathcal{V}(P_a)} \mathbf{v}_i,\qquad\mathbf{n}_a = \frac{\sum_{\mathbf{v}_i \in\mathcal{V}(P_a)} \mathbf{n}_i}{\left\|\sum_{\mathbf{v}_i \in\mathcal{V}(P_a)} \mathbf{n}_i\right\|_2} \tag*{(1)}

An orthographic camera is then placed along na\mathbf{n}_a and oriented toward oa\mathbf{o}_a, producing a local curvature rendering JaJ_a for the mask instance.

Next, we extract the DINOv2 patch tokens of local curvature rendering JaJ_a [31]. The visible region is cropped and resized to a fixed resolution. Each rendered view is augmented with rotations (0∘0^\circ, 90∘90^\circ, 180∘180^\circ, and 270∘270^\circ) and horizontal flip. We then generate patch tokens using DINOv2 for each augmented view. The region mask is down-sampled to the DINO patch grid, and the tokens covered by the mask are sampled to create a token pool: Ta={z^a,1,…,z^a,Ka}\mathcal{T}_a = \{\hat{z}_{a,1}, \ldots, \hat{z}_{a,K_a}\}. Each token is normalized into a unit vector.

Consistency Filtering. To eliminate masks that contain inconsistent textures, we compute a consistency score γa\gamma_a for each JaJ_a. This score is defined as the median similarity between each local feature z^a,i\hat{z}_{a,i} and the normalized average feature direction of the entire mask. The score is formulated as:

γa=median⁡i(z^a,iT∑jz^a,j∥∑jz^a,j∥2).(2)\gamma_a = \operatorname{median}_i \left( \frac{\hat{z}_{a,i}^{\mathsf{T}} \sum_j \hat{z}_{a,j}}{\left\lVert\sum_j \hat{z}_{a,j} \right\rVert_2} \right). \tag*{(2)}

We eliminate masks with γa\gamma_a smaller than a predefined threshold τs\tau_s. In our implementation, we let τs=0.65\tau_s = 0.65. Two masks of the Merlion example are provided in Figure 5. The mask on the left contains only one geometry texture, therefore exhibits high consistency. Meanwhile, the mask on the right is eliminated since it contains two types of texture, which deliver a consistency score smaller than the threshold.

Two masks of the Merlion example showing curvature maps and token-to-centroid cosine similarity

Figure 5. The intra-mask consistency on two masks of the Merlion example. For each view, we visualize the curvature map (left) with its candidate mask marked in red. The token-to-centroid cosine similarity map is provided on the right. The value of cosine similarity is visualized from red (high) to blue (low).

Feature Clustering. To merge instances representing the same geometric texture, for each pair of mask instances JaJ_a and JbJ_b, we compute a token-token similarity matrix Sijab=z^a,iTz^b,jS_{ij}^{ab} = \hat{z}_{a,i}^{\mathsf{T}}\hat{z}_{b,j}. We utilize bidirectional top-kk matching to reduce the sensitivity to boundary tokens and partial occlusions. Specifically, we first identify the maximum matching scores along both directions: ui=max⁡jSijabu_i = \max_j S_{ij}^{ab} and vj=max⁡iSijabv_j = \max_i S_{ij}^{ab}. The pairwise instance similarity is then defined as the average of the top-kk mean values from both sets:

sim⁡(a,b)=12[TopKMean⁡k({ui}i)+TopKMean⁡k({vj}j)].(3)\operatorname{sim}(a,b) = \frac{1}{2}\left[\operatorname{TopKMean}_k(\{u_i\}_i) + \operatorname{TopKMean}_k(\{v_j\}_j)\right]. \tag*{(3)}

To segment geometric texture instances, we not only consider the similarity of the rendered textures, but also consider their connectivity over the surface. We store the disconnected regions with the same pattern as different texture instances. We first construct a mutual-kkNN graph GG over all valid mask instances [54]. The nearest neighbors kk of aa are denoted as Nk(a)\mathcal{N}_k(a). We create an edge between aa and bb if they are mutual neighbors (b∈Nk(a)b \in\mathcal{N}_k(a) and a∈Nk(b)a \in\mathcal{N}_k(b)) and their similarity satisfies sim⁡(a,b)≥τclu\operatorname{sim}(a,b) \geq\tau_{\mathrm{clu}}. Subsequently, the raw mask clusters {Qr}\{Q_r\} are obtained by extracting the connected components of GG, formulated as {Qr}=CC⁡(G)\{Q_r\} = \operatorname{CC}(G).

To refine the clusters, we calculate a cluster support score for each mask a∈Qa \in Q, defined as the median similarity to its cluster peers: ρa=median⁡b∈Q,b≠asim⁡(a,b)\rho_a = \operatorname{median}_{b \in Q,b \ne a}\operatorname{sim}(a,b). Low-support members are split into singleton clusters. Finally, the mask confidence used for 3D voting is reweighted by incorporating its internal consistency γa\gamma_a and cluster support ρa\rho_a:

s^a=max⁡(smin⁡,saγaαρaβ).(4)\hat{s}_a = \max\left(s_{\min}, s_a \gamma_a^\alpha\rho_a^\beta\right). \tag*{(4)}

In our implementation, we let k=8k = 8 for the bidirectional top-kk matching, k=5k = 5 for the mutual-kkNN graph construction, and τclu=0.8\tau_{\mathrm{clu}} = 0.8. For the confidence reweighting, we set α=1.0\alpha= 1.0, β=1\beta= 1, and smin⁡=0.25s_{\min} = 0.25. A visualization of the resulting mask clusters is shown in Figure 6.

Visualization of mask clustering in the token-distance embedding space

Figure 6. Visualization of mask clustering in the token-distance embedding space. Nodes denote mask instances, with sizes proportional to mask areas and colors indicating final clusters. Positions are obtained by metric MDS [1] from the pairwise distance: 1−sim⁡(a,b)1 - \operatorname{sim}(a,b). Edges represent valid mutual-kkNN connections, with thicker edges indicating higher similarity.

Mask Fusion by 3D Voting

To fuse the 2D mask clusters {Qr}\{Q_r\} into the 3D mesh, we project the masks onto the mesh surface and accumulate votes on the visible faces. For a mask Ca∈QrC_a \in Q_r and a view ff, a mesh face ff will receive a positive weight w+w_+ when its projected face center falls inside CaC_a, and will receive a negative penalty −w−-w_- otherwise. This vote is further scaled by the mask confidence score cac_{a} and an angular weight cos⁡(θa,f)\cos(\theta_{a,f}), where θa,f\theta_{a,f} is the angle between the face normal and the viewing direction. The raw voting score is accumulated as:

Vr,fraw=∑a∈Vr,fcacos⁡(θa,f)⋅W(a,f),(5)V_{r,f}^{\mathrm{raw}} = \sum_{a \in\mathcal{V}_{r,f}} c_a \cos(\theta_{a,f}) \cdot W(a,f), \tag*{(5)}

where Vr,f⊆Qr\mathcal{V}_{r,f} \subseteq Q_r denote the subset of masks from which face ff is visible, W(a,f)=w+W(a,f) = w_+ if the projected face center of ff lies inside mask CaC_a, and −w−-w_- otherwise. To avoid bias towards frequently observed regions, the raw score is averaged by the visibility count, yielding the final score:

Vr,f=Vr,frawmax⁡(1,∣Vr,f∣).(6)V_{r,f} = \frac{V^{\mathrm{raw}}_{r,f}}{\max(1, \lvert V_{r,f}\rvert)}. \tag*{(6)}

A face is assigned to cluster rr if its average score Vr,f>τVV_{r,f} > \tau_V and it accumulates at least nmin⁡+n_{\min}^{+} positive hits. Face-label conflicts are resolved by weighted voting. When a face receives positive votes from multiple texture clusters, the cluster with the highest accumulated normalized score is selected.

Finally, we refine the assignments by extracting the connected components over the 3D mesh and discarding isolated components whose area is smaller than a minimum value. Ultimately, we get a set of 3D instances {Pr}\{P_r\} from {Qr}\{Q_r\}. In our implement, we set w+=2.0w_{+}=2.0, w−=1.0w_{-}=1.0 and nmin⁡+=3n_{\min}^{+}=3.

Finally, different initial 3D parts may correspond to the same physical region due to different perspectives. We therefore merge parts that exhibit strong spatial overlap. For any two 3D semantic parts PaP_a and PbP_b, we calculate their 3D Intersection-over-Union (IoU) as IoU⁡(a,b)=∣Pa∩Pb∣/∣Pa∪Pb∣\operatorname{IoU}(a,b)=\lvert P_a\cap P_b\rvert/\lvert P_a\cup P_b\rvert, where ∣⋅∣\lvert\cdot\rvert denotes the number of mesh faces.

If IoU⁡(a,b)≥τiou\operatorname{IoU}(a,b)\geq\tau_{\mathrm{iou}}, the two parts are connected in a region-adjacency graph. We set a threshold of τiou=0.8\tau_{\mathrm{iou}}=0.8 to ensure that only highly overlapping parts are merged. The final non-overlapping 3D regions, denoted as {Pk}k=1K\{P_k\}_{k=1}^{K}, are obtained by extracting the connected components of this graph.

The first row of Figure 9 shows the final result on the Merlion example. The resulting face regions are rendered with distinct colors. The result indicates that our method effectively recognizes all the distinct geometric textures over the model. Even the text on the base is successfully segmented.

Comparisons of the segmentation results with Shape Diameter, SAMesh, and our method on reconstructed real objects

Figure 9. Comparisons of the segmentation results with Shape Diameter, SAMesh, and our method on reconstructed real objects. Colors of instances are assigned randomly.

Applications

Geometric Texture Enhancement and Removal. For model editing, normal filters are capable of removing all the high-frequency local characteristics, such as noises [55, 49]. Specifically, the static/dynamic filter effectively identifies all the relief structures over the model surface, without recognizing their higher-level patterns. Based on our segmentation results, we employ the static/dynamic filter to interactively enhance or remove a specific type of geometric texture over the model (see Figure 1). Our segmentation method dramatically improves editing efficiency.

Geometric texture segmentation and applications

Figure 1. Our method can segment 3D meshes based on 3D geometric texture, without requiring predefined label sets or precisely tailored text prompts. We develop an interactive tool to support a range of applications, including surface editing and model retrieval.

Geometric Texture Replacement. After removing the user-specific geometric texture instance, the user can select an arbitrary texture from our DTSet, and apply it to the target region to achieve replacement of geometric texture (see Figure 1).

Model Retrieval. Effective retrieval of the reconstructed model that shares similar geometric texture is critical for the projection and restoration of the heritage. The query in Figure 1 is the brick wall texture of Merlion. The retrieval is performed among four models. For each model, we performed segmentation and extracted the DINOv2 feature tokens of all the texture instances. Then, we measure the similarity between all the texture instances and the target (Eq. (3)), returning the model that contains the texture instance with the highest similarity.

Experimental Results

Ablation Study

Firstly, we evaluate the contribution of three key components. The LoRA-adapted SAM3 is replaced with the original zero-shot model. Next, the step of feature-based association is removed from the framework. All the masks are grouped solely according to the overlap of their projected 3D regions. Then, feature-based association is removed from the framework, and masks are grouped solely according to the overlap of their projected 3D regions. The experimental results are shown in Table 1. For all quantitative evaluations, the results are presented as percentages.

MethodMUCov ↑MWCov ↑SQ50 ↑RQ50 ↑PQ50 ↑
w/o SAM3 Fine-tuning35.6449.0481.6634.5729.15
w/o Clustering39.9154.2677.9641.0332.79
w/o Filtering54.3371.0879.1859.8748.55
Ours61.7878.2483.1570.9959.32

Table 1. Ablation study of our instance segmentation pipeline. All variants are evaluated using class-agnostic, face-level metrics, and all values are reported as percentages.

The results indicate that replacing the fine-tuned model with zero-shot SAM3 brings the largest degradation in segmentation accuracy. Removing feature-based clustering also substantially reduces performance. Consistency filtering provides a further improvement over the unfiltered variant. The complete method consistently achieves the best performance across all metrics. Overall, the results demonstrate that all three components are necessary for accurate and reliable 3D geometric-texture instance segmentation.

Comparisons on Image Segmentation

We compare the results of segmentation on rendered curvature maps with zero-shot SAM3 [CGH*25] and Texture-

SAM [10]. SAM3 and our methods are given the identical prompt: “repetitive pattern”, in all experiments.

Figure 7 qualitatively illustrates the performance of our method compared to the methods described above in wild examples. We performed quantitative evaluation on the GTSeg test set. Statistics are provided in Table 2. To comprehensively assess both semantic and instance-level quality, we adopt mIoU and Dice coefficient to measure overall spatial overlap, Precision and Recall for pixel-level exactness, and Average Precision (AP, AP50_{50}, AP75_{75}) for instance-level localization accuracy.

Comparisons of the segmentation results with Texture-SAM, SAM3, and our GTSAM on five rendered views of reconstructed real objects.

Figure 7. Comparisons of the segmentation results with Texture-SAM, SAM3, and our GTSAM on five rendered views of reconstructed real objects.

MethodmIoU ↑Dice ↑Precision ↑Recall50_{50} ↑AP50_{50} ↑AP75_{75} ↑Mean AP ↑
TextureSAM15.6125.8715.675.730.700.170.43
SAM359.3369.0372.6236.9329.7617.0323.40
Ours (GTSAM)78.1885.7488.9081.4274.5052.7262.06

Table 2. Quantitative comparison on segmentation of 2D images.

As shown in Table 2, our fine-tuned SAM3 significantly outperforms all baseline methods on every metric. Specifically, while TextureSAM achieves a high Recall (0.913), its excessively low Precision (0.129) and mIoU (0.128) indicate severe over-segmentation and massive false positives, rendering its instance-level AP scores negligible. The SAM3 demonstrates solid foundational capability with an mIoU of 0.669; however, its performance degrades drastically under stricter boundary criteria (AP50_{50} drops to 0.124), indicating its struggle with precise boundary adherence required by the GTSeg domain. In contrast, our fine-tuned SAM3 effectively bridges this domain gap, achieving a state-of-the-art mIoU of 0.933 and a Dice score of 0.965. Most notably, our method maintains exceptional accuracy under strict intersection thresholds, boosting the Mean AP from 0.218 to 0.890 and AP75_{75} from 0.124 to 0.831. These results demonstrate that our fine-tuning strategy retains the strong recall of the foundation model while drastically improving its precision and boundary localization.

Comparisons on Model Segmentation

We collect 12 models reconstructed from real objects from public resources [4] and synthetic models from SHREC’25 examples [32], with various resolutions, functions, complexities, and semantics. Their geometric textures cover diverse scales, structures, and shapes with open boundaries.

Qualitative Evaluation. We compare our segmentation results with ShapeDiameter [39] and SAMesh [TZF∗24a]. Figure 8 shows a qualitative comparison of three unlabeled models from SHREC’25. Figure 9 and Figure 10 further compare the performance of these methods on real scans.

Comparisons of the segmentation results with Shape Diameter, SAMesh, and our method on synthetic 3D models from SHREC’25

Figure 8. Comparisons of the segmentation results with Shape Diameter, SAMesh, and our method on synthetic 3D models from SHREC’25 [32].

Comparisons of segmentation results across Input, Shape Diameter, SAMesh, and Ours

Figure 10. Comparisons of the segmentation results with Shape Diameter, SAMesh, and our method on reconstructed real objects. Colors of instances are assigned randomly.

ShapeDiameter relies on volumetric thickness and global shape structure, making it insensitive to fine-grained surface-level texture boundaries, particularly when textured patterns are shallow, repetitive, or confined to the same semantic part. SAMesh offers better flexibility by projecting 2D segmentation onto 3D meshes, but its behavior is dominated by object/part-level visual priors, leading to cross-view inconsistencies when texture elements share similar semantic appearance, yet differ geometrically. Our method addresses these limitations by using curvature renderings as input, directing

segmentation toward intrinsic surface variation rather than color, illumination, or semantic category. Combined with mask-level clustering and multi-view voting, it produces coherent, view-consistent regions for repeated geometric patterns and resolves texture boundaries defined by subtle local surface relief. These comparisons confirm that conventional mesh segmentation and general 2D-to-3D pipelines are insufficient for geometric texture segmentation, and that curvature-based representation with texture-aware clustering provides a more suitable formulation.

Quantitative Evaluation. We compute two metrics for all mesh vertices to assess the quality of the segmentation results: i) MUCOV, which quantifies the overall region-wise overlap between the predicted segments and the ground truth; and ii) Recall50_{50}, which measures the fraction of ground-truth instances correctly identified with an IoU threshold of 50% [37 LMB∗14]. The statistics are reported in Table 3. The segmentation results of our method achieve the best MUCOV and Recall50_{50} scores. These results confirm that our target problem is challenging, and validate the effectiveness of our proposed method in segmenting geometric texture instances over 3D models.

MethodMUCOV ↑Recall50_{50} ↑
TextureSAM + Voting18.1313.43
ShapeDiameter35.1819.80
SAMesh32.3521.33
Ours61.7870.54

Table 3. Quantitative comparison on segmentation of 3D models.

Conclusion

Existing model segmentation methods focus predominantly on semantics or low-level geometry. In this work, we presented a method for the automatic recognition and segmentation of geometric textures on 3D surface models. We first constructed a specialized benchmark for geometric texture segmentation. Then, we proposed a geometric texture-oriented segmentation method that follows a 2D-to-3D pipeline. By leveraging a fine-tuned visual segmentation model and DINO feature-based clustering, our method effectively identifies repetitive texture patterns across multiple views. Furthermore, we designed a weighted voting fusion strategy to achieve precise 3D segmentation. Building upon this, we developed a tool that enables users to segment geometric texture instances and to edit or retrieve models based on the segmentation result. Experimental results demonstrate the effectiveness of our method.

Limitations and Future Works. However, our method fails in segmenting weak relief, hollow structures or heavy occlusions, as shown in Figure 11. In the future, we plan to enrich our benchmark with more complex synthetic models and models reconstructed from real objects. Other future directions include: speeding up the segmentation process, integrating generative models for texture synthesis and transfer, and enabling interactive model segmentation [LXD∗24,ZHC∗25].

Figure 11. Two failure cases of our method.

Acknowledgements

This work was supported in part by NSFC (62302313), ICFCRT (W2441020), GD Basic and Applied Basic Research Foundation (2023B1515120026), Shenzhen Science and Technology Program (KQTD20210811090044003, KJZD20240903100022028), and Scientific Development Fund from Guangdong Provincial Key Laboratory of Visual Media and Multidimensional Intelligence.

References

  1. [1]Borg I., Groenen P. J.: Modern multidimensional scaling: Theory and applications. Springer, 2005.
  2. [2]Bhat P., Ingram S., Turk G.: Geometric texture synthesis by example. In Proceedings of the 2004 Eurographics/ACM SIGGRAPH Symposium on Geometry processing (2004), pp. 41–44.
  3. [4]Binninger A., Wiersma R., Herholz P., Sorkine-Hornung O.: Tetweave: Isosurface extraction using on-the-fly delaunay tetrahedral grids for gradient-based mesh optimization. ACM Transactions on Graphics (TOG) 44, 4 (2025), 1–19.
  4. [8]Cimpoi M., Maji S., Kokkinos I., Mohamed S., Vedaldi A.: Describing textures in the wild. In Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit (2014), pp. 3606–3613.arxiv.org/abs/1311.3618
  5. [9]Cheng B., Misra I., Schwing A. G., Kirillov A., Girdhar R.: Masked-attention mask transformer for universal image segmentation. In Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit (2022), pp. 1290–1299.arxiv.org/abs/2112.01527
  6. [10]Cohen I., Meivar B., Tu P., Avidan S., Oren G.: Decoupling shape and texture in sam-2 via controlled texture replacement. In IEEE Winter Conf. Appl. Comput. Vis. (2026), pp. 4077–4086.DOI
  7. [11]Cheng B., Schwing A. G., Kirillov A.: Per-pixel classification is not all you need for semantic segmentation. In Adv. Neural Inf. Process. Syst. (2021).arxiv.org/abs/2107.06278
  8. [16]Guerrero P., Kleiman Y., Ovsjanikov M., Mitra N. J.: Pcpnet: Learning local shape properties from raw point clouds. Computer Graphics Forum 37, 2 (2018), 115–126.DOI
  9. [17]He K., Gkioxari G., Dollár P., Girshick R.: Mask R-CNN. In Proc. IEEE Int. Conf. Comput. Vis. (2017), pp. 2961–2969.
  10. [19]Hertz A., Hanocka R., Giryes R., Cohen-Or D.: Deep geometric texture synthesis. ACM Transactions on Graphics (TOG) 39, 4 (2020), 108–1.DOI
  11. [23][LHGM] Lai Y.-K., Hu S.-M., Gu D., Martin R. R.: Geometric texture synthesis and transfer via geometry images. In Proceedings of the 2005 ACM symposium on Solid and physical modeling, pp. 15–26. 2DOI
  12. [25][LSD15] Long J., Shelhamer E., Darrell T.: Fully convolutional networks for semantic segmentation. In Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit (2015), pp. 3431–3440. 2
  13. [26][LUX*25] Liu M., Uy M. A., Xiang D., Su H., Fidler S., Sharp N., Gao J.: Partfield: Learning 3d feature fields for part segmentation and beyond. In Proc. IEEE Int. Conf. Comput. Vis. (2025), IEEE, pp. 9704–9715. 2arxiv.org/abs/2504.11451
  14. [28][LYW*20] Liang C., Yin J., Wu J., Wang J., Wei M., Guo Y.: A survey of 3d mesh segmentation based on clustering analysis. Journal of Computer-Aided Design & Computer Graphics 32, 4 (2020), 680–692. 2
  15. [29][MWS*25] Meng H., Wang D., Shao Z., Liu L., Wang Z.: Text2vdm: Text to vector displacement maps for expressive and interactive 3d sculpting. In Proc. IEEE Int. Conf. Comput. Vis. (2025), pp. 16882–16892. 2arxiv.org/abs/2502.20045
  16. [30][MZC*19] Mo K., Zhu S., Chang A. X., Yi L., Tripathi S., Guibas Leonidas J. E. A.: Partnet: A large-scale benchmark for fine-grained and hierarchical part-level 3d object understanding. In Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit (2019), pp. 909–918. 2arxiv.org/abs/1812.02713
  17. [31][ODM*24] Oquab M., Darcet T., Moutakanni T., Vo H. V., Szafraniec M., Khalidov Vasil E. A.: DINOv2: Learning robust visual features without supervision. Transactions on Machine Learning Research (2024). 5arxiv.org/abs/2304.07193
  18. [32][PTB24] Paolini G., Tortorici C., Berretti S.: Binary segmentation of relief patterns on point clouds. Computers & Graphics 123 (2024), 104020. 2, 7DOI
  19. [33][PTB*25] Paolini G., Tortorici C., Berretti S., et al.: Shrec’25 track: Retrieval and segmentation of multiple relief patterns. In Eurographics Workshop on 3D Object Retrieval (2025), pp. 1–8. 1, 2DOI
  20. [34][QSMG17] Qi C. R., Su H., Mo K., Guibas L. J.: PointNet: Deep learning on point sets for 3d classification and segmentation. In Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit (2017), pp. 652–660. 2arxiv.org/abs/1612.00593
  21. [35][QYSG17] Qi C. R., Yi L., Su H., Guibas L. J.: PointNet++: Deep hierarchical feature learning on point sets in a metric space. In Adv. Neural Inf. Process. Syst. (2017). 2arxiv.org/abs/1706.02413
  22. [36][RFB15] Ronneberger O., Fischer P., Brox T.: U-Net: Convolutional networks for biomedical image segmentation. In Med. Image Comput. Comput. Assist. Interv. (2015), pp. 234–241. 2arxiv.org/abs/1505.04597
  23. [37][RZ17] Ren M., Zemel R. S.: End-to-end instance segmentation with recurrent attention. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017), pp. 293–301. 7arxiv.org/abs/1605.09410
  24. [38][Sha08] Shamir A.: A survey on mesh segmentation techniques. In Comp. Graph. Forum 27, 2 (2008), pp. 1539–1556. 2DOI
  25. [39][SSCO08] Shapira L., Shamir A., Cohen-Or D.: Consistent mesh partitioning and skeletonisation using the shape diameter function. The Visual Computer 24, 4 (2008), 249–259. 7DOI
  26. [40][TBOW21] Tortorici C., Berretti S., Obeid A., Werghi N.: Convolution operations for relief-pattern retrieval, segmentation and classification on mesh manifolds. Pattern Recognition Letters 142 (2021), 32–38. 2DOI
  27. [41][TCC*26] Team S. D., Chen X., Chu F.-J., Gleize P., Liang K. J., Alexander Sax E. A.: Sam 3d: 3dfy anything in images. arxiv preprint arxiv:2511.16624 (2026). 2
  28. [43][TZF*24b] Tang G., Zhao W., Ford L., Benhaim D., Zhang P.: Segment any mesh: Zero-shot mesh part segmentation via lifting segment anything 2 to 3d. arXiv preprint arxiv:2408.13679 (2024). 2, 3arxiv.org/abs/2408.13679
  29. [45][WGS*18] Wang P., Gan Y., Shui P., Yu F., Zhang Y., Chen Songle E. A.: 3d shape segmentation via shape fully convolutional networks. Computers & Graphics 70 (2018), 128–139. 2
  30. [46][WSL*19] Wang Y., Sun Y., Liu Z., Sarma S. E., Bronstein M. M., Solomon J. M.: Dynamic graph cnn for learning on point clouds. ACM Trans. Graph. 38 (2019). 2arxiv.org/abs/1801.07829
  31. [47][XHW*26] Xiong W., Hu Z., Wu Y., Song P., Zheng J.: Paver: Element-based pattern creation on 3d free-form surfaces. Computer-Aided Design (2026), 104031. 1DOI
  32. [48][XYQ*25] Xu M., Yin X., Qiu L., Liu Y., Tong X., Han X.: Sam-pro3d: Locating sam prompts in 3d for zero-shot instance segmentation. In International Conference on 3D Vision (3DV) (2025), IEEE, pp. 1222–1232. 2arxiv.org/abs/2311.17707
  33. [49][XZSJ18] Xiong W., Zhang P., Sander P. V., Joneja A.: Shape-inspired architectural design. In Proc. ACM SIGGRAPH Interactive 3D Graphics and Games (2018), pp. 1–10. 6DOI
  34. [51][YCK*25] Yang Y., Chen Q., Kim V. G., Chaudhuri S., Huang Q., Chen Z.: Genvdm: Generating vector displacement maps from a single image. In Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit (2025), pp. 26618–26629. 2arxiv.org/abs/2503.00605
  35. [52][YLX*24] Yin Y., Liu Y., Xiao Y., Cohen-Or D., Huang J., Chen B.: Sai3d: Segment any instance in 3d scenes. In Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit (2024), pp. 3292–3302. 2arxiv.org/abs/2312.11557
  36. [53][YRWX25] Ying Z., Rong B., Wang J., Xu M.: Chord: Chain of rendering decomposition for pbr material estimation from generated texture images. In Proc. SIGGRAPH Asia (2025), ACM. 3arxiv.org/abs/2509.09952
  37. [54][YZZW24] Yan M., Zhang J., Zhu Y., Wang H.: Maskclustering: View consensus based mask graph clustering for open-vocabulary 3d instance segmentation. In Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit (2024), pp. 28274–28284. 5arxiv.org/abs/2401.07745
  38. [55][ZDZ*15] Zhang W., Deng B., Zhang J., Bouaziz S., Liu L.: Guided mesh normal filtering. In Comp. Graph. Forum (2015), vol. 34, pp. 23–34. 6DOI
  39. [57][ZHR*26] Zhou J., Huang X., Ren Y., Fan D., Gu S., Bin Tian E. A.: Ov3dseg-vggt: Open-vocabulary 3d segmentation with visual geometry-grounded transformers. Visual Informatics (2026), 100311. 2DOI
  40. [58][ZJ16] Zhou Q., Jacobson A.: Thing10k: A dataset of 10,000 3d-printing models. arXiv preprint arXiv:1605.04797 (2016). 3arxiv.org/abs/1605.04797
  41. [59][ZXL*24] Zhong Z., Xu Y., Li J., Xu J., Li Z., Liu Y., Chaohui E. A.: Meshsegmenter: Zero-shot mesh semantic segmentation via texture synthesis. In Eur. Conf. Comput. Vis. (2024), pp. 182–199. 2arxiv.org/abs/2407.13675
  42. [60][ZXZD24] Zheng J., Xiong W., Zeng J., Davis E. D.: Design element placement on 3d surfaces, July 11 2024. US Patent App. 18/279,130. 2

Paper details

vlnc:W_jehj4pwb
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 7

downloads · 1

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents