betaFeedback

WorksComputer Science

Data-Driven Variable-Length Compression of 3D Representations

Abstract

Compressing 3D shapes is important for the efficiency of many applications, yet existing methods either cannot leverage dataset-level statistics or use a fixed compression ratio for all shapes. We propose a learned variable-length compression method that exploits structure across a training dataset while adapting compressed representation size to individual shape complexity using only fast feed-forward inference. Our method operates on general latent representations and improves rate-distortion for both meshes and signed distance fields. Code is available at https://github.com/voilalab/varlen-3D.

Introduction

Representing 3D shapes is an essential step in many applications, including 3D reconstruction, 3D generation, physical simulation, gaming, augmented reality, and manufacturing design. The shapes in these applications differ widely in complexity. Most current 3D representations, such as voxels and meshes, represent shapes at full fidelity and therefore require substantial storage regardless of shape complexity. Storing, transmitting, and processing these representations without any compression can thus be costly. Traditional 3D compression methods [1, 2, 3, 4] exploit specific knowledge about the shape or the representation to achieve compression. As a result, they cannot readily adapt to new shape representations out-of-the-box or leverage training examples to improve compression. Recent work has explored learning-based methods in two directions. Per-shape methods fit a small model to each shape independently [5, 6, 7, 8, 9]. These methods do not require any training examples of other shapes or share any parameters between different shapes. However, they cannot exploit prior knowledge about what shapes typically look like, which limits how much they can compress. Moreover, they require computationally costly per-shape optimization at inference time. An alternative is to train an encoder and a decoder on a large collection of shapes [10, 11, 12], so that the encoder outputs a compact representation while the decoded shapes maintain high fidelity. Such learned compressors hold the promise of improving with more data and compute, but their compression ratio is usually fixed as a hyperparameter before training. Every shape is therefore compressed at the same ratio, regardless of its complexity, which limits the achievable rate–distortion tradeoff across shapes of variable complexity.

We propose a learning-based variable-length compressor that retrofits any pretrained shape autoencoder. Specifically, we use a frozen encoder to map a shape to a latent vector, and train a small hyper-autoencoder that maps this latent vector into an ordered sequence of tokens. A regression decoder then maps any prefix of this token sequence to approximate the full latent vector, and then the full shape, with quality improving as more tokens are kept. At inference time, users can choose how many tokens to keep per shape, guided by quality metrics computed in either the shape or latent space so more bits can be allocated to more complex shapes. Since the underlying autoencoder is frozen, our method can be combined with any existing shape autoencoder without retraining it.

We demonstrate that our method applies to different 3D representations, such as meshes and signed distance functions (SDFs), and that it can be supervised in both the latent space and the shape space. We evaluate on ShapeNet [13] utilizing both supervision strategies and compare rate-distortion, measured as Chamfer distance versus compression ratio, against per-shape and dataset-level compression baselines. We find that our method achieves a better rate-distortion tradeoff than both families of baselines (in the absence of quantization), with the property that the number of tokens needed to reach a target quality increases with shape complexity. Ablation studies show that our regression decoder outperforms a FlexTok-style flow decoder [14], with most of the gain coming from training on the downstream shape losses rather than intermediate latent losses.

In summary, our contributions are:

  • A novel 3D compression framework that reuses pretrained shape autoencoders and allows the compression ratio to be chosen per shape at inference time without retraining, and

  • State-of-the-art rate–distortion on meshes and SDFs, compared to both per-shape and dataset-level unquantized compression baselines.

Related Works

3D Compression. Traditional compression techniques for 3D assets are highly specialized to the type of representation, such as point clouds [2] and meshes [3], and are designed for compatibility with existing 2D image and video codecs rather than for optimal compression performance. Recently, per-shape optimization frameworks have been proposed for compressing 3D geometries, often signed distance functions (SDFs) [5, 12] or displacement fields [7, 8, 9], as implicit neural representations. While these methods often yield better rate–distortion tradeoffs than traditional codecs, they take much longer to encode each shape or change the encoded bit-rate for a given shape, and cannot exploit cross-shape or semantic redundancies.

Variable-Length Tokenization. Recent work has explored variable-length or hierarchical image and video representations for applications such as autoregressive generation and retrieval. Matryoshka Representation Learning [15] introduced variable-length latent image representations, while TiTok [16] adapted this idea to tokenization by using suffix token dropout to learn ordered token sequences whose prefixes provide progressively higher-fidelity reconstructions. ElasticTok [17] extended this approach to video, training a variable-length VQ-VAE from scratch. Both TiTok and ElasticTok demonstrate autoregressive generation from ordered tokens, but require costly training of specialized autoencoders on large datasets.

FlexTok [14] proposed an alternative formulation for variable length tokenization, leveraging a pretrained variational autoencoder (VAE). FlexTok converts the fixed-size latents of the pretrained VAE into an ordered sequence of tokens using suffix dropout, so that any variable-length token prefix can be used as context-tokens for a rectified-flow-based decoder [18] that is trained to approximate the tokens of the original VAE. While this formulation is suitable for generating feasible shapes even when few context tokens are available, it falls short when stricter reconstruction quality criteria are imposed. Additionally, FlexTok requires expensive retraining of their generative Flux VAE decoder [18].

Our proposed framework simultaneously addresses the limitations of these families of variable-length 3D compression methods by re-purposing pretrained shape autoencoders and tokenizers for the task of variable bit-rate compression. Existing shape encoders such as Mesh-GPT [10], 3DShape2VecSet [11], LaGeM [19], etc. are trained on vast 3D-asset datasets such as ShapeNet [13] and Objaverse-XL [20], and consequently have learned rich geometric priors which our compression pipeline can seamlessly inherit. Our framework operates in the latent/token space of these foundational autoencoder models, which enables it to leverage pretrained models without retraining them and makes it modality agnostic (e.g., applicable to SDFs, meshes, and images) as long as there is a pretrained autoencoder for each modality. Our method can seamlessly adapt to additional modalities as corresponding foundation autoencoders become available.

Method

Our method assumes access to a pretrained signal encoder E:S→Z\mathcal{E}:\mathcal{S}\to\mathcal{Z} that encodes a signal s∈Ss\in\mathcal{S} of a specific type of modality (e.g., S\mathcal{S} could be the space of meshes, SDFs, or images) into a latent vector or token set z∈Zz\in\mathcal{Z}, and its corresponding decoder D:Z→S\mathcal{D}:\mathcal{Z}\to\mathcal{S} that reconstructs the latent code back into an approximation of the original signal. Our goal is to convert zz to an ordered set of tokens {t1,t2,…,tK}∈T\{t_1,t_2,\ldots,t_K\}\in\mathbb{T}, where ti∈Rdt_i\in\mathbb{R}^d, so that any prefix of tokens {t1,…,tk}\{t_1,\ldots,t_k\} can be decoded into an approximation of the original latent vector zz, with k≤Kk\leq K and fidelity increasing with kk.

We train a hyper-autoencoder to perform this transformation, as shown in Figure 1. Our input zz is a flat list of tokens in the form {z1,z2,…,zZ}\{z_1,z_2,\ldots,z_Z\}. We apply Perceiver-style crossattention [21], where the keys and values are derived from zz as shown in Equation (1) and the queries come from a learned vector set Q(e):={q1(e),q2(e),…,qK(e)}Q^{(e)} := \{q^{(e)}_1, q^{(e)}_2, \ldots, q^{(e)}_K\} that is fixed once the hyper-encoder is trained. The outputs of this cross attention (CA) operation, which we call hyper-tokens, form a sequence {t1,t2,…,tK}\{t_1,t_2,\ldots,t_K\} of the same length KK as the learned query set. Compactly, this hyper-encoding operation is:

Overview of a variable-length hyper-autoencoder for a 3D signal

Figure 1. Overview of our variable-length hyper-autoencoder, shown for an arbitrary 3D signal. A pretrained signal encoder maps the input signal to per-element latents; our suffix-dropout autoencoder compresses them into an ordered sequence of tokens (green), where any prefix of kk tokens suffices to reconstruct the latents, which are decoded back to the signal space. Dashed arrows show the two supervision paths: the latent loss (token →\to token) and the downstream reconstruction losses (signal →\to signal). Note that backpropagation along the signal-level path is possible for our regression hyper-decoder but not for a flow-based hyper-decoder, as is used in FlexTok [14].

ti=CA(query=Q(e),key=z1:Z)∀i∈{1,…,K},(1)t_i = \mathrm{CA}(\text{query}=Q^{(e)},\text{key}=z_{1:Z}) \quad\forall i \in\{1,\ldots,K\}, \tag*{(1)}

Next, these produced hyper-tokens tit_i undergo a sequence of causal self-attention layers. The ordering of the hyper-tokens results from both the inherent ordering in the learned queries and the causal nature of the self-attention we apply to the hyper-tokens, which encourages earlier tokens to capture as much structure as possible.

Finally, our hyper-decoder reconstructs an approximation of the full original token/latent set even from a truncated prefix {t1,…,tk}\{t_1,\ldots,t_k\} of the hyper-token set. We perform this decoding using Perceiver-style cross attention, which uses the hyper-token sequence to derive the keys and values and uses a separate set of ZZ learned decoder tokens Q(d):={q1(d),q2(d),…,qZ(d)}Q^{(d)} := \{q^{(d)}_1,q^{(d)}_2,\ldots,q^{(d)}_Z\} as queries. The reconstructed token set {z~1,z~2,…,z~Z}\{\tilde{z}_1,\tilde{z}_2,\ldots,\tilde{z}_Z\} always has the same length ZZ irrespective of the number of suffix hyper-tokens that are dropped, due to the fixed length of the learned query tokens Q(d)Q^{(d)}. Notationally,

z~i(k)=CA(query=Q(d),key=t1:k)∀i∈{1,…,Z},(2)\tilde{z}^{(k)}_i = \mathrm{CA}(\text{query}=Q^{(d)},\text{key}=t_{1:k}) \quad\forall i \in\{1,\ldots,Z\}, \tag*{(2)}

where k≤Kk \le K is the length of the truncated hyper-token sequence. The superscript (k)(k) in z~i(k)\tilde{z}^{(k)}_i denotes the number of hyper-tokens used to reconstruct an approximation of the original latent set element ziz_i. We denote the hyper-autoencoder reconstruction of the full latents using kk hyper-tokens as z~(k)=HAEk(z)\tilde{z}^{(k)} = \mathrm{HAE}_k(z).

We train the hyper-autoencoder in two self-supervised stages: latent-space supervision and signal-space finetuning. In the first stage, given a dataset of signals in SS, we minimize the latent-space loss:

Llatent=Es∼SEk≤K[∥E(s)−HAEk(E(s))∥22].(3)\mathcal{L}_{\mathrm{latent}} = \mathbb{E}_{s\sim S}\mathbb{E}_{k\le K}\left[\left\lVert\mathcal{E}(s)-\mathrm{HAE}_k(\mathcal{E}(s))\right\rVert_2^2\right]. \tag*{(3)}

In the second stage, we finetune the hyper-autoencoder with signal-space losses to retain the performance of the pretrained signal autoencoder:

Lsignal=Es∼SEk≤KSignalLoss⁡(s,D(HAEk(E(s)))).(4)\mathcal{L}_{\mathrm{signal}} = \mathbb{E}_{s\sim S}\mathbb{E}_{k\le K}\operatorname{SignalLoss}\left(s,\mathcal{D}\left(\mathrm{HAE}_k(\mathcal{E}(s))\right)\right). \tag*{(4)}

Here, SignalLoss could be any loss function that is specific to autoencoder training for the given modality, for example mean squared error over SDF samples.

Including this initial stage of latent-space training offers two benefits: (i) latent-space supervision is faster, as it avoids the expensive gradient flow through the signal decoder D(⋅)\mathcal{D}(\cdot); (ii) latent-space supervision is more stable at early stages of training, when the initial latents are far from the pretrained decoder’s training distribution and thus signal-space error can produce uninformative gradients.

Because these loss functions encourage the hyper-decoder to be able to approximate the full original latent and signal even from an arbitrarily sized prefix sequence of hyper-tokens, the hyper-encoder naturally learns to encode coarse structure of the signals in hyper-tokens of earlier indices while reserving the higher-index hyper-tokens to enrich the reconstructions with higher frequency or fine-grained details.

Once the hyper-autoencoder is trained, a given signal can be converted to a sequence of hyper-tokens in a feed-forward manner which can either be truncated arbitrarily after encoding, or have its suffix dropped out to obtain compressed representations with varying bit-rates.

Results

We experimentally evaluate our method, demonstrating that given arbitrary latent representations of 3D shapes, it induces a coarse-to-fine variable-length compression. We compare our method against relevant baselines and show that it achieves state-of-the-art compression for 3D representations, including both meshes and SDFs by tokenizing pretrained latent spaces of MeshGPT [10] and 3DShape2VecSet [11] models respectively.

In Figure 2, we show sample 3D reconstructions of both meshes and SDFs while varying kk, the number of hyper-tokens retained. We visually observe that as kk increases, so does the amount of information preserved compared to the ground truth, with the highest kk resulting in the most faithful reconstruction. This relationship is also visible in the second and fourth plots in the bottom row of Figure 2, where we compare the reconstruction quality (Chamfer distance) vs. the number of hyper-tokens kk for both modalities. In each case, we observe an inverse relationship.

Figure 2. Variable-length compression of SDF and MeshGPT latents. Top: Sample SDF and mesh reconstructions, each decoded from the first kk tokens. As kk increases, so does the fidelity compared to the ground truth. Bottom, column 1: Baseline comparison on rate-distortion curve for SDF/implicit methods. Our method represents a Pareto frontier. Bottom, column 2: Chamfer distance vs. kk for SDF compression. Bottom, column 3: Baseline comparison of rate-distortion curve for mesh compression. We show for comparison compression ratios for both quantized and unquantized versions of MeshGPT—our method is unquantized, so our curve is shifted to the right. Bottom, column 4: Chamfer distance vs. kk for mesh compression.

In the first and third plots of Figure 2, we evaluate rate–distortion for our method as well as modality-specific baselines for both meshes and SDFs. For each modality, we graph our method’s reconstruction error (Chamfer distance) vs. compression ratio, alongside other variable-length compression baselines and the fixed-length signal autoencoder (VecSet for SDF and MeshGPT for mesh) from which our hyper-autoencoder is derived. We observe that for both modalities our method lies on the Pareto frontier in fidelity vs. compression ratio, and that the original signal autoencoder does not exceed our method’s performance, showing that our hyper-autoencoder does not induce performance degradation.

In Figure 3, we visualize example shapes from our SDF experiments stratified by the number of hyper-tokens needed to achieve a threshold error in reconstructing the original shape latents. We observe that shapes that require more tokens (larger kk) are visually more complex, indicating that our hyper-autoencoder learns to compress simpler shapes using fewer tokens, achieving variable-length compression.

Examples of 3D shapes with increasing geometric complexity

Figure 3. Number of hyper-tokens needed to achieve a target MSE threshold in latent space increases with the “geometric complexity” of shapes.

Ablations

We perform ablation studies on key elements of our method. In particular, compared to FlexTok [14], another variable-length tokenizer, we use a regression hyper-decoder whereas FlexTok uses a flow-based generative decoder. For the compression task, we find that this is a key difference because it enables our method to train on both signal- and latent-space

losses, rather than only latent-space losses. Our method can train on downstream signal-space losses through its regression decoder, but not through a flow decoder, as we cannot backpropagate through ODE sampling, whereas regression can be directly supervised by geometric losses. We hypothesize that the significant increase in performance when training on these signal-space losses is due to the relative lack of strong, perceptually aligned latent representations in the 3D domain vs. the image domain for which FlexTok was designed.

To isolate the strength of these effects, we compare three decoders: (1) our regression decoder trained with the downstream reconstruction losses, (2) the same regression decoder trained with the latent loss only, and (3) a FlexTok-style flow decoder. We show the results in Figure 4, and observe that the inclusion of signal-space reconstruction loss provides substantial improvements in fidelity.

Figure 4. Decoder ablation. Chamfer distance vs. kk on meshes for our regression decoder, the same regression decoder trained with the latent loss only, and the flow decoder. At low kk the flow decoder returns a sharper but wrong signal; at high kk the regression decoders are more faithful.

Conclusions

We propose a data-driven variable-length compression method for 3D representations. Our method can operate on the latent-space of any pretrained signal autoencoder and allocate fewer hyper-tokens to simpler signals and more to complex signals. Our approach demonstrates state-of-the-art compression results for both mesh and SDF representations of 3D shapes. Interesting future directions include extending our method to other signal modalities and 3D representations.

References

References

  1. [1]J. Rossignac, “Edgebreaker: connectivity compression for triangle meshes,” IEEE Transactions on Visualization and Computer Graphics, 1999.DOI
  2. [2]mpeg, “MPEG-I – video based (V-PCC) and geometry based (G-PCC) point cloud compression,” https://mpeg.expert/v-g-pcc/index.html.
  3. [3]YiHyun Choi, Jong-Beom Jeong, Soonbin Lee, and Eun-Seok Ryu, “Overview of the video-based dynamic mesh coding (V-DMC) standard work,” in 2022 13th International Conference on Information and Communication Technology Convergence (ICTC), 2022.DOI
  4. [4]Google Inc., “Draco: A library for compressing and decompressing 3D geometric meshes and point clouds,” https://github.com/Google/draco, 2017.
  5. [5]Towaki Takikawa, Joey Litalien, Kangxue Yin, Karsten Kreis, Charles Loop, Derek Nowrouzezahrai, Alec Jacobson, Morgan McGuire, and Sanja Fidler, “Neural geometric level of detail: Real-time rendering with implicit 3D shapes,” IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021.
  6. [6]David B. Lindell, Dave Van Veen, Jeong Joon Park, and Gordon Wetzstein, “BACON: Band-limited coordinate networks for multiscale scene representation,” IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022.arxiv.org/abs/2112.04645
  7. [7]Venkataram Edavamadathil Sivaram, Tzu-Mao Li, and Ravi Ramamoorthi, “Neural geometry fields for meshes,” ACM SIGGRAPH 2024 Conference Papers, 2024.DOI
  8. [8]Sai Karthikey Pentapati, Gregoire Phillips, and Alan C. Bovik, “Mesh compression with quantized neural displacement fields,” Computer Graphics Forum, vol. 44, 2025.
  9. [9]Sai Karthikey Pentapati, Gregoire Phillips, and Alan C. Bovik, “Hierarchical neural surfaces for 3D mesh compression,” in 2025 Picture Coding Symposium (PCS), 2025.arxiv.org/abs/2512.15985
  10. [10]Yawar Siddiqui, Antonio Alliegro, Alexey Artemov, Tatiana Tommasi, Daniele Sirigatti, Vladislav Rosov, Angela Dai, and Matthias Nießner, “MeshGPT: Generating triangle meshes with decoder-only transformers,” IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024.arxiv.org/abs/2311.15475
  11. [11]Biao Zhang, Jiapeng Tang, Matthias Nießner, and Peter Wonka, “3DShape2VecSet: A 3D shape representation for neural fields and generative diffusion models,” ACM Trans. Graph., vol. 42, no. 4, July 2023.arxiv.org/abs/2301.11445
  12. [12]Hanwen Jiang, Haitao Yang, Georgios Pavlakos, and Qixing Huang, “CoFiE: Learning compact neural surface representations with coordinate fields,” Advances in Neural Information Processing Systems (NeurIPS), 2024.arxiv.org/abs/2406.03417
  13. [13]Angel X Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, Jianxiong Xiao, Li Yi, and Fisher Yu, “ShapeNet: An information-rich 3D model repository,” arXiv preprint arXiv:1512.03012, 2015.arxiv.org/abs/1512.03012
  14. [14]Roman Bachmann, Jesse Allardice, David Mizrahi, Enrico Fini, Oğuzhan Fatih Kar, Elmira Amirloo, Alaaeldin El-Nouby, Amir Zamir, and Afshin Dehghan, “FlexTok: Resampling images into 1D token sequences of flexible length,” International Conference on Machine Learning (ICML), 2025.arxiv.org/abs/2502.13967
  15. [15]Aditya Kusupati, Gantavya Bhatt, Aniket Rege, Matthew Wallingford, Aditya Sinha, Vivek Ramanujan, William Howard-Snyder, Kaifeng Chen, Sham Kakade, Prateek Jain, and Ali Farhadi, “Matryoshka representation learning,” Advances in Neural Information Processing Systems (NeurIPS), 2022.arxiv.org/abs/2205.13147
  16. [16]Qihang Yu, Mark Weber, Xueqing Deng, Xiaohui Shen, Daniel Cremers, and Liang-Chieh Chen, “An image is worth 32 tokens for reconstruction and generation,” Advances in Neural Information Processing Systems, 2024.arxiv.org/abs/2406.07550
  17. [17]Wilson Yan, Volodymyr Mnih, Aleksandra Faust, Matei Zaharia, Pieter Abbeel, and Hao Liu, “ElasticTok: Adaptive tokenization for image and video,” International Conference on Learning Representations (ICLR), 2025.arxiv.org/abs/2410.08368
  18. [18]Or Greenberg, “Demystifying flux architecture,” arXiv preprint arXiv:2507.09595, 2025.arxiv.org/abs/2507.09595
  19. [19]Biao Zhang and Peter Wonka, “LaGeM: A large geometry model for 3D representation learning and diffusion,” International Conference on Learning Representations (ICLR), 2025.arxiv.org/abs/2410.01295
  20. [20]Matt Deitke, Ruoshi Liu, Matthew Wallingford, Huong Ngo, Oscar Michel, Aditya Kusupati, Alan Fan, Christian Laforte, Vikram Voleti, Samir Yitzhak Gadre, et al., “Objaverse-XL: A universe of 10M 3D objects,” Advances in Neural Information Processing Systems (NeurIPS), 2023.arxiv.org/abs/2307.05663
  21. [21]Andrew Jaegle, Felix Gimeno, Andy Brock, Oriol Vinyals, Andrew Zisserman, and Joao Carreira, “Perceiver: General perception with iterative attention,” International Conference on Machine Learning (ICML), 2021.arxiv.org/abs/2103.03206

Paper details

vlnc:W_89kmdrgj
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents