betaFeedback

WorksComputer Science

Partition-Invariant Tuning for 3D Scene Understanding

Abstract

Scene-level point cloud understanding remains challenging due to diverse geometries and spatial layouts. While pre-trained 3D point cloud foundation models (PFMs) offer strong transferability, full fine-tuning (FFT) incurs substantial computational and storage costs. Parameter-efficient fine-tuning (PEFT) provides a promising alternative, but existing PEFT methods largely focus on object-level point clouds and overlook serialization-induced partition variations in large-scale scenes. To address this issue, we propose PointPiT, a partition-invariant tuning framework for scene-level point clouds. Specifically, a Scene-aware Structural Adapter (SSA) integrates local geometric patterns with global scene context to mitigate partition-induced representation shifts. Moreover, Gradient Subspace Optimization (GSO) selects informative and partition-stable update directions, suppressing partition-dependent variations during optimization. Extensive experiments across multiple scene-level benchmarks demonstrate that PointPiT achieves competitive or even superior performance to full fine-tuning with less than 1% of backbone's parameters, while achieving consistent state-of-the-art performance among representative PEFT methods.

Introduction

PFMs have demonstrated strong transferability across diverse 3D scene understanding tasks [1, 2, 3, 4], providing powerful pretrained representations for downstream adaptation [5, 6, 7, 8]. Despite significant capabilities, downstream adaptation of large pretrained point cloud backbones still largely depends on FFT, incurring considerable computational and storage costs while requiring substantial task-specific supervision [9, 10, 11]. To alleviate these limitations, PEFT has emerged as an efficient alternative, achieving competitive performance with only a small fraction of trainable parameters [12, 13, 14, 15]. However, existing methods largely focus on object-level point clouds, leaving large-scale scene adaptation underexplored.

When existing PEFT methods are extended to large-scale scene-level point clouds, the potential of pretrained representations is often not fully realized. Large-scale 3D scenes comprise dense point sets with substantial geometric heterogeneity, for which serialization-based grouping is essential to tractable computation [16, 17, 18, 1]. Yet such grouping admits multiple valid partitions, exposing identical physical points to distinct local contexts. This variability can steer the limited adaptation capacity of PEFT toward partition-specific artifacts rather than task-relevant semantics.

Such partition sensitivity manifests differently across adaptation paradigms. As illustrated in Figure 1, FFT can largely reconcile partition-induced discrepancies through unrestricted backbone updates, whereas linear probing simply inherits the partition sensitivity of the frozen representation.

Figure 1. Illustration of partition sensitivity on the ScanNet Val [22].

More critically, LoRA [19] adapts the backbone within a restricted update space yet lacks an explicit mechanism to distinguish task-relevant variation from partition-specific perturbations, thereby rendering its limited adaptation budget vulnerable to the latter. Such entanglement between task adaptation and partition-specific perturbations limits the effective exploitation of pretrained scene representations by existing PEFT methods [20, 21].

In this paper, we propose PointPiT, a partition-invariant tuning framework tailored for large-scale scene-level point clouds. PointPiT mitigates partition sensitivity through both forward representation adaptation and backward optimization. Specifically, the Scene-aware Structural Adapter (SSA) combines backbone-encoded geometric features with global scene context to mitigate partition-induced representation shifts. Gradient Subspace Optimization (GSO) further characterizes gradient informativeness and partition-dependent variation to identify a more stable update subspace, reducing sensitivity to partition-dependent gradients during back-propagation.

Extensive experiments demonstrate the effectiveness and efficiency of PointPiT. When integrated into Point Transformer V3 (PTv3) [1], PointPiT reduces the number of trainable parameters by more than 99% compared with full fine-tuning of the backbone, while maintaining comparable or even better performance across benchmarks. Our main contributions are summarized as follows:

  • We reveal partition sensitivity as a critical challenge for PEFT in large-scale 3D scenes, exposing the mismatch between scene serialization and parameter-efficient adaptation.

  • We propose PointPiT, where SSA mitigates partition-induced representation shifts and GSO favors more stable gradient directions during optimization.

  • Extensive experiments on multiple scene-level benchmarks demonstrate that PointPiT achieves SOTA performance among PEFT methods, while attaining performance comparable to FFT with substantially fewer trainable parameters.

Figure 2. Overview of PointPiT. Serialized scene points are grouped, position encoded, and embedded into tokens before entering the frozen backbone. PointPiT consists of a Scene-aware Structural Adapter (SSA) for forward representation adaptation and Gradient Subspace Optimization (GSO) for backward optimization. SSA is inserted into each backbone block to enhance scene-level structural representations, while GSO constrains gradient updates to partition-stable directions. During training, only SSA and the task head are updated.

METHODOLOGY

Overview

Given a large-scale point cloud scene X∈RN×d\mathbf{X} \in\mathbb{R}^{N \times d}, where NN denotes the number of points and dd represents the input feature dimension, PointPiT aims to efficiently adapt pretrained point cloud foundation models to downstream scene understanding tasks. Building on the pioneer works [1], we define the point tokenization process as T=E∘G∘S\mathcal{T} = \mathcal{E} \circ\mathcal{G} \circ\mathcal{S}, where S(⋅)\mathcal{S}(\cdot) transforms unordered points into ordered sequences, G(⋅)\mathcal{G}(\cdot) partitions serialized points into local groups, and E(⋅)\mathcal{E}(\cdot) projects grouped points into feature tokens. The initial token representation is therefore obtained as H0=T(X)\mathbf{H}^{0} = \mathcal{T}(\mathbf{X}). The pretrained backbone consists of LL hierarchical blocks, each augmented with SSA, whose forward process is defined as:

Hl+1=Bl(Hl),l=0,1,⋯ ,L−1,(1)\mathbf{H}^{l+1} = \mathcal{B}^{l}(\mathbf{H}^{l}), \quad l = 0,1,\cdots,L-1, \tag*{(1)}

where Bl\mathcal{B}^{l} denotes the ll-th backbone block. To enable efficient adaptation for large-scale scenes, PointPiT introduces two complementary components. The proposed Scene-aware Structural Adapter (SSA) enhances forward representation adaptation by incorporating both local geometric patterns and global scene context. Meanwhile, Gradient Subspace Optimization (GSO) optimizes the backward update trajectory by preserving task-relevant adaptation directions.

Scene-aware Structural Adapter

Large-scale scenes exhibit diverse geometric structures, where serialized local groups may suffer from limited contextual perception due to restricted receptive fields. To improve scene-level adaptation, SSA combines geometric features with global scene context.

Given the intermediate representation Hl∈RNl×Cl\mathbf{H}^{l} \in\mathbb{R}^{N_l \times C_l}, SSA first adapts the geometry-aware features encoded by the pretrained backbone through a lightweight bottleneck projection:

Hlocl=σ(HlWdl)Wul,(2)\mathbf{H}^{l}_{\mathrm{loc}} = \sigma\left(\mathbf{H}^{l}\mathbf{W}^{l}_{d}\right)\mathbf{W}^{l}_{u}, \tag*{(2)}

where Wdl∈RCl×r\mathbf{W}^{l}_{d} \in\mathbb{R}^{C_l \times r} and Wul∈Rr×Cl\mathbf{W}^{l}_{u} \in\mathbb{R}^{r \times C_l} denote the projection matrices, rr represents the hidden dimension, and σ(⋅)\sigma(\cdot) denotes the nonlinear activation function.

To incorporate global scene-level structural information, SSA aggregates the input into a scene descriptor zl=AvgPool⁡(Hl)∈R1×Cl\mathbf{z}^{l} = \operatorname{AvgPool}(\mathbf{H}^{l}) \in\mathbb{R}^{1 \times C_l} and performs attention-weighted global aggregation:

Hctxl=1NlSoftmax⁡Nl(Hl(zlWql)TCl)(HlWvl),(3)\mathbf{H}^{l}_{\mathrm{ctx}} = \mathbf{1}_{N_l}\operatorname{Softmax}_{N_l}\left(\frac{\mathbf{H}^{l}(\mathbf{z}^{l}\mathbf{W}^{l}_{q})^{T}}{\sqrt{C_l}}\right)(\mathbf{H}^{l}\mathbf{W}^{l}_{v}), \tag*{(3)}

where Wql,Wvl∈RCl×Cl\mathbf{W}^{l}_{q}, \mathbf{W}^{l}_{v} \in\mathbb{R}^{C_l \times C_l}, Softmax⁡Nl\operatorname{Softmax}_{N_l} normalizes over points, and 1Nl∈RNl×1\mathbf{1}_{N_l} \in\mathbb{R}^{N_l \times1} broadcasts the aggregated context to all points.

Finally, the local and contextual representations are adaptively fused through a learnable gating mechanism:

H~l=Hl+gl⊙Hlocl+(1−gl)⊙Hctxl,(4)\widetilde{\mathbf{H}}^{l} = \mathbf{H}^{l} + \mathbf{g}^{l} \odot\mathbf{H}^{l}_{\mathrm{loc}} + (1-\mathbf{g}^{l}) \odot\mathbf{H}^{l}_{\mathrm{ctx}}, \tag*{(4)}

where gl=Sigmoid⁡([Hlocl,Hctxl]Wgl+bgl)\mathbf{g}^{l} = \operatorname{Sigmoid}([\mathbf{H}^{l}_{\mathrm{loc}},\mathbf{H}^{l}_{\mathrm{ctx}}]\mathbf{W}^{l}_{g} + \mathbf{b}^{l}_{g}) is a point-wise gate, Wgl∈R2Cl×Cl\mathbf{W}^{l}_{g} \in\mathbb{R}^{2C_l \times C_l}, and gl∈(0,1)Nl×Cl\mathbf{g}^{l} \in(0,1)^{N_l \times C_l}. ⊙\odot represents element-wise multiplication. Wql\mathbf{W}^{l}_{q}, Wvl\mathbf{W}^{l}_{v}, and Wgl\mathbf{W}^{l}_{g} are implemented as rank-tt factorizations, with only their factor matrices trained. Consequently, SSA establishes a structural adaptation pathway that mitigates partition-induced representation shifts, while providing more consistent feature representations for GSO to optimize gradient updates.

Gradient Subspace Optimization

Although SSA mitigates partition-induced variation in forward representations, backward optimization remains susceptible to partition-dependent gradient variations. GSO therefore favors informative and partition-stable update directions. For each trainable module with parameters ϕl∈RDl\boldsymbol{\phi}_{l} \in\mathbb{R}^{D_l}, we maintain a rolling buffer of BB scenes, each with KK valid partitions. Let gbk\mathbf{g}_{bk} denote the stored gradient for the kk-th partition of scene bb, and gb=K−1∑k=1Kgbk\mathbf{g}_{b} = K^{-1}\sum_{k=1}^{K}\mathbf{g}_{bk}. The gradient second moment and partition covariance are then estimated from these stored gradients as follows:

Fl=1BK∑b,kgbkgbkT,(5)\mathbf{F}_{l} = \frac{1}{BK}\sum_{b,k}\mathbf{g}_{bk}\mathbf{g}_{bk}^{T}, \tag*{(5)}
Gp,l=1BK∑b,k(gbk−gb)(gbk−gb)T.\mathbf{G}_{p,l} = \frac{1}{BK}\sum_{b,k}(\mathbf{g}_{bk}-\mathbf{g}_{b})(\mathbf{g}_{bk}-\mathbf{g}_{b})^{T}.

GSO seeks a subspace that balances informative update directions against partition-dependent gradient variations:

Ul∗=argmax⁡0≤q≤ql, U∈RDl×q Tr⁡[UT(Fl−λGp,l)U],UTU=Iq,(6)\mathbf{U}^{*}_{l} = \underset{0 \le q \le q_{l},\,\mathbf{U}\in\mathbb{R}^{D_l \times q}}{\operatorname{argmax}}\ \operatorname{Tr}\left[\mathbf{U}^{T}(\mathbf{F}_{l}-\lambda\mathbf{G}_{p,l})\mathbf{U}\right], \quad\mathbf{U}^{T}\mathbf{U}=\mathbf{I}_{q}, \tag*{(6)}

where λ\lambda controls partition regularization and the subspace dimension is capped at qq.

To avoid constructing Dl×DlD_l \times D_l matrices, we stack the stored gradients into Zl∈RDl×BK\mathbf{Z}_{l} \in\mathbb{R}^{D_l \times BK} and define C=IB⊗(IK−1K1KT/K)\mathbf{C} = \mathbf{I}_{B} \otimes(\mathbf{I}_{K} - \mathbf{1}_{K}\mathbf{1}_{K}^{T}/K). With the thin SVD Zl=QlΣlVlT\mathbf{Z}_{l} = \mathbf{Q}_{l}\mathbf{\Sigma}_{l}\mathbf{V}_{l}^{T}, the subspace is recovered from the leading eigenvectors of

Ml=1BKΣlVlT(I−λC)VlΣl.(7)\mathbf{M}_{l} = \frac{1}{BK}\mathbf{\Sigma}_{l}\mathbf{V}_{l}^{T}(\mathbf{I}-\lambda\mathbf{C})\mathbf{V}_{l}\mathbf{\Sigma}_{l}. \tag*{(7)}

Specifically, Ul∗=QlEl\mathbf{U}_l^* = \mathbf{Q}_l\mathbf{E}_l, where El\mathbf{E}_l contains at most qq eigenvectors with positive eigenvalues. Thus, qeff=min⁡(q,n+(Ml))q_{\mathrm{eff}} = \min(q,n_+(\mathbf{M}_l)), where n+(Ml)n_+(\mathbf{M}_l) denotes the number of positive eigenvalues. The eigen-problem has dimension at most BKBK, rather than DlD_l.

During back-propagation, GSO projects each module’s gradient onto the subspace spanned by Ul∗\mathbf{U}_l^*, favoring directions with lower partition-dependent variation:

g^l=Ul∗(Ul∗)T∇ϕlL.(8)\hat{\mathbf{g}}_l = \mathbf{U}_l^*(\mathbf{U}_l^*)^{\mathsf{T}}\nabla_{\phi_l}\mathcal{L}. \tag*{(8)}

The parameters are then updated by ϕl←ϕl−ηg^l\phi_l \leftarrow\phi_l-\eta\hat{\mathbf{g}}_l, where η\eta is the learning rate. This restricts updates to the selected subspace without changing the set of trainable parameters. Consequently, GSO reduces sensitivity to partition-dependent gradient variations and complements the representation adaptation of SSA.

EXPERIMENTS

Experimental Setup

Datasets. To evaluate scene-level semantic segmentation, we benchmark PointPiT on three large-scale benchmarks, including ScanNet [22], ScanNet++ [30], and S3DIS [28]. These datasets contain diverse indoor scenes with different scales and semantic categories, providing comprehensive evaluations for scene-level adaptation.

Implementation Details. We adopt PTv3 [1] initialized with pre-trained Sonata [2] weights as the backbone. Following the PEFT setting [9], the backbone parameters are frozen during training, and only the proposed SSA modules and task heads are optimized. We set the prefix [25] token length to 4, the LoRA [19] rank to 32, and the adapter’s [24] intermediate dimension to 64. The hyperparameters are shared across all datasets throughout the validation process. All methods follow the official downstream training and evaluation protocol. For fair comparison, all PEFT methods and the corresponding full fine-tuning baseline use the same Sonata backbone without the decoder. All experiments are conducted on 8 NVIDIA A800 GPUs.

Evaluation Metrics. We report mean Intersection over Union (mIoU, %), mean Accuracy (mAcc, %), and overall Accuracy (allAcc, %) for semantic segmentation following prior works [16, 31, 32, 33]. We additionally report the number of trainable parameters to evaluate the parameter efficiency of different adaptation methods.

Main Results

Quantitative Results. As shown in Table 1, compared to full fine-tuning, PointPiT achieves competitive performance and even exceeds it on ScanNet Val [22] with less than 1% of the backbone’s parameters. Across semantic categories, PointPiT attains the best or second-best results in most cases. Furthermore, Table 2 demonstrates that PointPiT consistently outperforms all representative PEFT methods across diverse scene-level benchmarks. These benchmarks exhibit substantial differences in geometric structure and semantic granularity, demonstrating the effectiveness of PointPiT across diverse scene-level datasets.

MethodParam.mAccmIoUwallfloorcabinetbedchairsofatabledoorwindowbooks.picturecounterdeskcurtainrefrig.showertoiletsinkbathtubother
Full Fine-tuning
PTv3-PPT (sup.) [23]124.885.978.687.696.375.485.388.983.977.369.070.075.366.072.669.979.077.779.493.775.785.863.2
Sonata (dec.) [2]124.886.179.488.396.376.386.389.084.878.470.272.276.267.474.170.480.178.879.893.676.685.463.8
Sonata (w/o dec.) [2]108.5 (100%)86.178.987.696.276.185.388.884.677.869.870.875.566.673.769.979.477.680.093.076.185.863.4
General PEFT Methods
Sonata (lin.)0.02 (0.02%)83.172.282.893.570.482.285.878.671.558.463.069.355.065.459.871.571.072.791.368.380.952.6
+ Adapter [24]1.90 (1.76%)85.1 (+2.0)76.8 (+4.6)86.097.074.584.787.482.477.368.568.372.062.069.967.676.875.678.291.872.684.159.3
+ Prefix Tuning [25]0.08 (0.08%)84.2 (+1.1)73.5 (+1.3)84.796.370.081.985.980.172.461.864.069.857.566.563.772.872.674.792.769.380.752.6
+ LoRA [19]0.89 (0.82%)85.2 (+2.1)76.3 (+4.1)85.596.872.684.585.981.974.365.567.871.161.971.267.177.875.976.792.772.884.359.7
PEFT Methods for Point Cloud
+ IDPT [9]2.61 (2.42%)82.9 (-0.2)72.6 (+0.4)84.495.269.181.785.279.471.160.562.068.057.965.761.772.870.873.191.069.481.052.0
+ DAPT [26]1.14 (1.06%)86.5 (+3.4)77.7 (+5.5)86.795.975.285.287.883.676.868.269.273.864.171.168.078.376.979.693.974.284.960.6
+ PointGST [10]1.05 (0.97%)85.8 (+2.7)77.7 (+5.5)85.796.473.984.989.382.576.968.369.073.865.471.567.978.276.979.593.874.484.361.4
+ PointTPA [27]1.18 (1.09%)86.3 (+3.2)78.4 (+6.2)88.096.675.385.988.583.578.168.270.375.165.172.669.078.976.379.693.775.284.563.6
+ PointPiT (ours)1.08 (0.99%)86.7 (+3.6)78.9 (+6.7)87.396.575.986.389.184.077.969.671.275.066.373.270.179.278.079.893.875.985.663.3

Table 1. Semantic segmentation results on ScanNet Validation [22]. We report mean accuracy (mAcc), mean IoU (mIoU), per-class IoU (%), and trainable parameters (M). The dec. and w/o dec. indicate Sonata with and without the decoder, respectively. Sonata (lin.) serves as the baseline for PEFT comparison. Among all PEFT methods, the best and second-best results are highlighted in black and underlined, respectively. The same notation applies to the following tables.

MethodS3DIS Area 5 mAccS3DIS Area 5 allAccS3DIS Area 5 mIoUS3DIS 6-fold mAccS3DIS 6-fold allAccS3DIS 6-fold mIoUScanNet200 Val mAccScanNet200 Val allAccScanNet200 Val mIoUScanNet++ Val mAccScanNet++ Val allAccScanNet++ Val mIoU
Full Fine-tuning
PTv3-PPT (sup.) [23]80.192.074.388.692.780.545.983.835.955.786.443.3
Sonata (dec.) [2]81.693.076.089.993.382.346.584.436.855.886.643.7
Sonata (w/o dec.) [2]80.393.174.587.392.379.547.883.737.350.786.341.8
General PEFT Methods
Sonata (lin.)80.990.873.087.490.876.541.681.229.349.484.936.5
+ Adapter [24]81.7 (+0.8)91.6 (+0.8)78.0 (+0.8)87.5 (+0.1)91.4 (+0.6)76.4 (-0.1)45.7 (+4.1)82.5 (+1.3)33.6 (+4.3)52.9 (+3.5)86.1 (+1.2)39.9 (+3.4)
+ Prefix Tuning [25]82.5 (+1.6)91.0 (+0.2)73.4 (+0.4)86.5 (-0.9)90.5 (-0.3)73.7 (-2.8)44.4 (+2.8)81.6 (+0.4)31.4 (+2.1)49.6 (+0.2)84.2 (-0.7)36.8 (+0.3)
+ LoRA [19]81.1 (+0.2)92.5 (+1.7)74.0 (+1.0)87.8 (+0.4)91.2 (+0.4)74.7 (-0.9)45.5 (+3.9)82.7 (+1.5)33.6 (+4.3)51.4 (+2.0)85.3 (+0.4)38.7 (+2.2)
PEFT Methods for Point Cloud
+ IDPT [9]81.2 (+0.3)90.5 (-0.3)72.0 (-1.0)86.8 (-0.6)90.6 (-0.2)75.2 (-1.3)41.9 (+0.3)81.0 (-0.2)29.0 (-0.3)50.0 (+0.6)84.1 (-0.8)36.6 (+0.1)
+ DAPT [26]81.2 (+0.3)92.7 (+1.9)74.6 (+1.6)88.1 (+0.7)91.8 (+1.0)78.2 (+1.7)45.8 (+4.2)82.9 (+1.7)34.2 (+4.9)50.9 (+1.5)85.2 (+0.3)39.6 (+3.1)
+ PointGST [10]81.4 (+0.5)92.9 (+2.1)74.0 (+1.0)88.6 (+1.2)92.2 (+1.4)78.9 (+2.4)46.5 (+4.9)83.2 (+2.0)35.0 (+5.7)52.0 (+2.6)85.9 (+1.0)40.0 (+3.5)
+ PointTPA [27]81.7 (+0.8)92.9 (+2.1)74.9 (+1.9)88.9 (+1.5)92.5 (+1.7)79.4 (+2.9)47.2 (+5.6)83.8 (+2.6)35.8 (+6.5)52.9 (+3.5)86.1 (+1.2)40.9 (+4.4)
+ PointPiT (ours)82.3 (+1.4)93.3 (+2.5)75.0 (+2.0)90.2 (+2.8)93.6 (+2.8)80.6 (+4.1)49.0 (+7.4)84.8 (+3.6)38.1 (+8.8)56.4 (+7.0)87.0 (+2.1)44.2 (+7.7)

Table 2. Semantic segmentation results on S3DIS Area 5 [28], S3DIS 6-fold [28], ScanNet200 Validation [29], and ScanNet++ Validation [30] sets. We report mean accuracy (mAcc), overall accuracy (allAcc), and mean IoU (mIoU). Trainable parameters are consistent with Table 1.

Qualitative Results. As illustrated in Figure 3, PointPiT produces more accurate predictions than FFT in complex scene regions, with fewer semantic misclassifications and more coherent object structures. These results demonstrate its ability to preserve fine-grained semantics under parameter-efficient adaptation.

Qualitative comparison of scene-level semantic segmentation

Figure 3. Qualitative comparison of scene-level semantic segmentation. The left and right parts present results on S3DIS Area 5 [28] and ScanNet [22], respectively. Compared with FFT, PointPiT generates more accurate and consistent predictions in complex scene regions.

Ablation Study

Effect of SSA and GSO. Table 3 demonstrates the complementary contributions of SSA and GSO. Without SSA, GSO operates only on the task head and yields a modest improvement. When combined with SSA, GSO further improves performance, indicating the benefit of partition-aware optimization over the adapted representations.

SSAGSOParam.ScanNet Val [22]
mAccmIoU
0.0283.172.2
✓1.0885.877.5
✓0.0284.672.9
✓✓1.0886.778.9

Table 3. Ablation study of different components of PointPiT.

Sensitivity to Serialization Configuration. Different serialization configurations result in distinct partition structures, which may introduce unstable adaptation patterns for PEFT methods. To isolate partition sensitivity, we evaluate the same checkpoint on identical scenes under different test-time serializations. By capturing more stable structural patterns and avoiding over-reliance on a specific partition layout, PointPiT maintains consistent performance across different configurations, as shown in Table 4. This suggests that the proposed adaptation strategy improves the generalization ability of scene-level models under varying serialization schemes.

MethodZ-orderHilbertTrans-ZTrans-H
Sonata (w/o dec.)78.977.678.178.2
Sonata (lin.)72.268.470.168.9
+LoRA76.371.373.572.0
+PointTPA78.473.976.874.6
+PointPiT78.978.078.678.2

Table 4. Sensitivity to different serialization strategies. We report mIoU (%) on ScanNet Val [22].

Analysis of Key Hyperparameters. Table 5 shows that r=64r = 64, q=32q = 32, and λ=0.5\lambda= 0.5 achieve the best performance. Smaller rr or qq limits adaptation capacity or informative updates, while excessive values introduce diminishing returns. Similarly, overly strong partition regularization may suppress useful gradients. These results indicate that PointPiT achieves effective adaptation with moderate model capacity and selective gradient updates, without relying on excessive parameter expansion or partition regularization.

rrmIoUqqmIoUλ\lambdamIoU
1676.5877.60.078.1
3278.01678.40.178.6
6478.93278.90.578.9
12878.76478.61.078.3
25678.512878.22.077.6

Table 5. Overall analysis of key hyperparameters in PointPiT.

Conclusion

In this paper, we propose PointPiT, a partition-invariant tuning framework for large-scale scene-level point clouds. We identify the mismatch between serialization-induced partition variations and parameter-efficient adaptation, and address it through scene-aware structural adaptation and gradient subspace optimization. Extensive experiments demonstrate that PointPiT achieves consistent state-of-the-art performance among PEFT methods and competitive or even superior performance to full fine-tuning with less than 1% of the backbone’s parameters. As an early exploration of scene-level PEFT, we expect PointPiT to serve as a strong baseline and provide insights for future research on efficient 3D scene adaptation.

References

References

  1. [1]Xiaoyang Wu, Yuxin Lao, Li Jiang, Xiangyu Liu, and Heng-shuang Zhao, “Point transformer v3: Simpler, faster, stronger,” in CVPR, 2024.
  2. [2]Xiaoyang Wu, Daniel DeTone, Duncan Frost, Tianwei Shen, Chris Xie, Nan Yang, Jakob Engel, Richard Newcombe, Heng-shuang Zhao, and Julian Straub, “Sonata: Self-supervised learning of reliable point representations,” in CVPR, 2025.
  3. [3]Dongxu Zhang, Yiding Sun, Pengcheng Li, Yumou Liu, Hongqiang Lin, Haoran Xu, Xiaoxuan Mu, Liang Lin, Wenbiao Yan, Ning Yang, et al., “Pointcot: A multi-modal benchmark for explicit 3d geometric reasoning,” arXiv preprint arXiv:2602.23945, 2026.arxiv.org/abs/2602.23945
  4. [4]Yue Li, Qi Ma, Runyi Yang, Huapeng Li, Mengjiao Ma, Bin Ren, Nikola Popovic, Nicu Sebe, Ender Konukoglu, Theo Gevers, et al., “Scenesplat: Gaussian splatting-based scene understanding with vision-language pretraining,” in ICCV, 2025.arxiv.org/abs/2503.18052
  5. [5]Yiding Sun, Jihua Zhu, Haozhe Cheng, Chaoyi Luv, Zhichuan Yang, Lin Chen, and Yaonan Wang, “Align then adapt: Rethinking parameter-efficient transfer learning in 4d perception,” TMM, 2026.arxiv.org/abs/2602.23069
  6. [6]Liyao Tang, Zhe Chen, and Dacheng Tao, “On geometry-enhanced parameter-efficient fine-tuning for 3d scene segmentation,” NeurIPS, 2026.arxiv.org/abs/2505.22444
  7. [7]Zhixuan You, Jihua Zhu, Yiding Sun, Zihao Guo, Haozhe Cheng, Dongxu Zhang, Lin Chen, and Hainan Luo, “Gauss-fusion: Towards multimodal 3d gaussian pretraining,” arXiv preprint arXiv:2607.05906, 2026.arxiv.org/abs/2607.05906
  8. [8]Xingguang Han, Yiding Sun, and Chaoyi Lu, “Rethinking regressor in 3d gaussian pretraining,” PRCV, 2025.DOI
  9. [9]Yaohua Zha, Jinpeng Wang, Tao Dai, Bin Chen, Zhi Wang, and Shu-Tao Xia, “Instance-aware dynamic prompt tuning for pre-trained point cloud models,” in ICCV, 2023.
  10. [10]Dingkang Liang, Tianrui Feng, Xin Zhou, Yumeng Zhang, Zhikang Zou, and Xiangxi Bai, “Parameter-efficient fine-tuning in spectral domain for point cloud learning,” TPAMI, 2025.
  11. [11]Zihao Guo, Jihua Zhu, Yiding Sun, Lin Chen, and Danwei Wang, “Parameter-efficient fine-tuning for spiking point cloud models,” arXiv preprint arXiv:2607.29048, 2026.arxiv.org/abs/2607.29048
  12. [12]Dongxu Zhang, Yingsen Wang, Yiding Sun, Haoran Xu, Peilin Fan, and Jihua Zhu, “Cmhanet: A cross-modal hybrid attention network for point cloud registration,” NEUCOM, 2026.arxiv.org/abs/2603.12721
  13. [13]Zhuoyue Zhang*, Yiding Sun*, Chaowei Fang, Haozhe Cheng, Jian Liu, Jihua Zhu, and Ajmal Saeed Mian, “Diffusion masked pretraining for dynamic point cloud,” arXiv preprint arXiv:2605.03639, 2026.arxiv.org/abs/2605.03639
  14. [14]Yankai Wang, Yiding Sun, Qirui Wang, Pengbo Li, Chaoyi Lu, and Dongxu Zhang, “Pointfrt: Explicit reinforcement fine-tuning for point cloud few-shot learning,” arXiv preprint arXiv:2603.23957, 2026.arxiv.org/abs/2603.23957
  15. [15]Xu Han, Yuan Tang, Jinfeng Xu, and Xianzhi Li, “Most: Efficient monarch sparse tuning for 3d representation learning,” in CVPR, 2025.arxiv.org/abs/2503.18368
  16. [16]Hengshuang Zhao, Li Jiang, Jiaya Jia, Philip H. S. Torr, and Vladlen Koltun, “Point transformer,” in ICCV, 2021.
  17. [17]Xiaoyang Wu, Yixing Lao, Li Jiang, Xihui Liu, and Heng-shuang Zhao, “Point transformer v2: Grouped vector attention and partition-based pooling,” in NeurIPS, 2022.arxiv.org/abs/2210.05666
  18. [18]Yiding Sun, Chaoyi Lu, Haozhe Cheng, Jun Wang, Huimin Lu, Lin Chen, and Jihua Zhu, “Curve3d: Curvature-aware masked autoencoder for self-supervised point cloud understanding,” PR, 2026.
  19. [19]Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen, “LoRA: Low-rank adaptation of large language models,” in ICLR, 2022.
  20. [20]Yiwen Tang, Ray Zhang, Zoey Guo, Xianzhen Ma, Bin Zhao, Zhigang Wang, Dong Wang, and Xuelong Li, “Point-peft: Parameter-efficient fine-tuning for 3d pre-trained models,” in AAAI, 2024.
  21. [21]Yiding Sun, Dongxu Zhang, Jihua Zhu, Haozhe Cheng, Zhengqiao Li, Pengcheng Li, Chaowei Fang, Yonghao Dong, and Lin Chen, “Tri-efficient transfer learning for point cloud videos,” arXiv preprint arXiv:2606.24175, 2026.arxiv.org/abs/2606.24175
  22. [22]Angela Dai, Angel X. Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, and Matthias Nießner, “Scannet: Richly-annotated 3d reconstructions of indoor scenes,” in CVPR, 2017.arxiv.org/abs/1702.04405
  23. [23]Xiaoyang Wu, Zhuotao Tian, Xin Wen, Bohao Peng, Xihui Liu, Kaicheng Yu, and Hengshuang Zhao, “Towards large-scale 3d representation learning with multi-dataset point prompt training,” in CVPR, 2024.
  24. [24]Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attarian, and Sylvain Gelly, “Parameter-efficient transfer learning for nlp,” in ICML, 2019.arxiv.org/abs/1902.00751
  25. [25]Xiang Lisa Li and Percy Liang, “Prefix-tuning: Optimizing continuous prompts for generation,” in ACL, 2021.arxiv.org/abs/2101.00190
  26. [26]Xin Zhou, Dingkang Liang, Wei Xu, Xinkui Zhu, Yihan Xu, Zhikang Zou, and Xiang Bai, “Dynamic adapter meets prompt tuning: Parameter-efficient transfer learning for point cloud analysis,” in CVPR, 2024.
  27. [27]Siyuan Liu, Chaoqun Zheng, Xin Zhou, Tianrui Feng, Dingkang Liang, and Xiang Bai, “Pointpa: Dynamic network parameter adaptation for 3d scene understanding,” in CVPR, 2026.
  28. [28]Iro Armeni, Ozan Sener, Amir R. Zamir, Helen Jiang, Ioannis Brilakis, Martin Fischer, and Silvio Savarese, “3d semantic parsing of large-scale indoor spaces,” in CVPR, 2016.
  29. [29]David Rozenberszki, Or Litany, and Angela Dai, “Language-grounded indoor 3d semantic segmentation in the wild,” in ECCV, 2022.arxiv.org/abs/2204.07761
  30. [30]Chandan Yeshwanth, Yueh-Cheng Liu, Matthias Nießner, and Angela Dai, “Scannet++: A high-fidelity dataset of 3d indoor scenes,” in ICCV, 2023.
  31. [31]Yatian Pang, Wenxiao Wang, Francis E. H. Tay, Wei Liu, Yijun Tian, and Li Yuan, “Masked autoencoders for point cloud self-supervised learning,” in ECCV, 2022.
  32. [32]Charles R. Qi, Hao Su, Kaichun Mo, and Leonidas J. Guibas, “Pointnet: Deep learning on point sets for 3d classification and segmentation,” in CVPR, 2017.
  33. [33]Zih...

Paper details

vlnc:W_5tp7zpdx
Version
v1 · latest
Published
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents