WorksComputer Science
Modeling Uncertainty Propagation in 3D Morphable Models
Abstract
We study the problem of mapping parameter-space variance of a 3D Morphable Model (3DMM) to per-vertex variance. Traditionally, per-vertex variance has been used to visually interpret the learnt parameter uncertainty. However, mapping parameter-space variance to per-vertex using sampling or error propagation is computationally expensive. We address this limitation with Closed-form Local Uncertainty Estimation (CLUE), which transforms the parameter-space distribution of 3DMMs to per-vertex through a closed-form approximation. CLUE emerges from applying the law of total variance to the 3DMM formulation and is modular, computationally inexpensive, and accurate. Subsequently, we study integrating CLUE for in-the-wild human mesh recovery training and inference. We note that CLUE uses the geometry of the mesh as an inductive bias when predicting the variance, leading to improved accuracy. Through extensive qualitative and quantitative experiments, we show that when used in training, CLUE (1) outpeforms probabilistic and deterministic baselines on metrics evaluating joints, mesh and uncertainty, and (2) has the highest uncertainty-to-error spearman correlation, with applications including data mining and interpretable uncertainty. Finally, we share SAM 3D Body (CLUE) which augments the regressor with high resolution uncertainty maps. We make our code available at https://github.com/vita-epfl/CLUE
Introduction
3D Morphable Models (3DMMs) Loper et al. [34]; Pavlakos et al. [39]; Li et al. [30] have been critical to the success of digital humans in real world applications. A popular application of 3DMMs is in-the-wild Human Mesh Recovery (HMR) [21, 6, 56], which reconstructs humans by estimating their 3DMM parameters from in-the-wild images. However, in-the-wild images often include factors such as occlusions, poor image quality and challenging poses. To address these uncertainties, methods have been developed to predict a distribution over the 3DMM parameters using the the Gaussian Negative Log-Likelihood (GNLL) and its variants Sengupta et al. (2021c,b); Dwivedi et al. (2024). The parameter variance is then mapped to the mesh to obtain interpretable, spatially localized per-vertex uncertainty. However, mapping the parameter distribution onto the mesh is non-trivial.
Popular methods in human mesh regression Sengupta et al. [46] compute per-vertex variances through repeated sampling. A large () number of samples are drawn from the parameter distribution which are then used to obtain a set of meshes. The per-vertex variance is computed empirically from this set of meshes. This approach is, however, computationally intensive and sensitive to the number of samples. In statistics, error propagation for Guides in Metrology (2008) is a powerful tool to map uncertainty across a smooth function. This typically involves computing the jacobian of the output w.r.t the input and projecting the input variance through it. While error propagation provides a closed-form approximation of the per-vertex uncertainty, the technique is computationally expensive. Moreover, operators such as the Jacobian may lack compatibility with popular on-device machine learning frameworks such as XNNPACK Google [17].

Figure 1. a) CLUE: Closed-form Local Uncertainty Estimation is a novel transformation that maps parameter-space distribution to local, per-vertex distribution for 3D Morphable Models. b) Accurate and Real-time: CLUE closely approximates sampling-based methods, while being faster. c) In-the-wild: We study in-the-wild training and inference, and share SAM 3D Body (CLUE), augmenting a frozen SAM 3D Body model with high-resolution uncertainty maps.
To address this limitation, we present Closed-form Local Uncertainty Estimation (CLUE), a novel formulation to map the parameter-space variance to per-vertex. CLUE uses the law of total variance and error propagation to analyse the 3DMM formulation, disentangling variance from pose, shape and expression. As a result, CLUE is interpretable not only by formulation but also visually. By making pragmatic assumptions, we drastically increase computational efficiency while retaining accuracy on par with linear error propagation. The reduced compute footprint coupled with accuracy of the transformation allow us to integrate CLUE in training probabilistic in-the-wild human mesh regressors. Recent advances in optimizing under uncertainty [49] highlight the use of inductive bias when modeling the predicted variance. In a similar vein, constrains the predicted parameter variance to explain the geometry of the 3DMM, resulting in improved accuracy of the HMR model. CLUE is also agnostic to the architecture of the HMR model, allowing for a seamless integration in training and inference.
We study three popular 3DMMs: (1) FLAME [30], (2) SMPL-X [39], and (3) MHR [14], which are also representative of SMPL Loper et al. [34] and STAR Osman et al. [37]. Next, we perform two sets of experiments: (a) statistical experiments to evaluate CLUE as a transformation from parameter to per-vertex distribution, and (b) real-world experiments to qualitatively and quantitatively evaluate the use of CLUE in training HMR models. Our experiments confirm that CLUE is accurate and computationally inexpensive, as well as out-performs architecture agnostic probabilistic baselines over the joints, mesh, negative log-likelihood and correlation metrics. Furthermore, we discuss applying CLUE for data mining and interpretable uncertainty for downstream decision making. Finally, we share SAM 3D Body (CLUE), which augments a frozen SAM 3D Body model with probabilistic vertices for uncertainty based reasoning. We share example visualizations in Figure 10, Figure 11. We make our code publicly available at https://github.com/vita-epfl/CLUE

Figure 10. Augmenting SAM 3D Body with probabilistic reasoning using CLUE (Part 1).

Figure 11. Augmenting SAM 3D Body with probabilistic reasoning using CLUE (Part 2).
Related Work
3D Morphable Model. 3DMMs are data-driven statistical models that represent different variations in human identity, expression and pose. They are typically constructed by applying Principal Component Analysis (PCA) to a registered collection of 3D meshes. These models represent geometry in rest-pose through a linear combination of shape, expression and pose-corrective bases. The rest-pose geometry can further be articulated through Linear Blend Skinning (LBS).
Following Eq. (1), a 3DMM modifies a mean template through a series of additive offsets. In this formulation, , , and are the linear bases representing shape, expression, and pose correctives, respectively. To articulate the mesh, represents the rigid transformation of each joint , while denote the skinning weights that determine how these transformations influence each vertex. We kindly refer to FLAME Li et al. [30], SMPL-X Pavlakos et al. [39], Basel Face Model (IEE, 2009) for a more detailed description of how 3DMMs are built.
Deterministic mesh regression. There have been significant works which leverage 3DMMs for HMR such as Kanazawa et al. [21], SPIN Kolotouros et al., [25], PARE Kocabas et al., [24], Mesh Graphormer Lin et al., [32], CLIFF Li et al., [31], SAM 3D Body Yang et al., [56] and Fast SAM 3D Body Yang et al., [55]. For faces, the classical 3DMM Blanz & Vetter [4] underpins a large family of analysis-by-synthesis fitters, and deep regressors Deng et al. [8] extend this to in-the-wild imagery. Yet, deterministic methods produce a point estimate and do not capture ambiguities in their prediction.
Probabilistic approaches. Under strong occlusion, ProHMR Kolotouros et al., [26] explicitly encodes the posterior as a multimodal distribution captured using normalizing flows. The use of normalizing flows is further refined in HuManiFlow Sengupta et al., [47] and Biggs et al. [3]. Diffusion and score-based generative models Cho & Kim, [7]; Stathopoulos et al. [52] explore its use in improved HMR. Similarly, for faces under occlusion, Diverse3DFace Dey & Boddeti, [9] and OFER Selvaraju et al., [43] use generative modeling over the FLAME parameters. While multimodal approaches effectively capture diverse modes through sampling, they do not explicitly model the confidence of individual modes. Unimodal methods focus on modeling ambiguity due to occlusion, lighting, noise and partial visibility. One line of work models pose parameters which live on , requiring that pose distributions must respect the rotation manifold Sengupta et al., [45]; Dünkel et al. [10]; Shetty et al. [48]. Methods such as Egger et al. [12, 13]; Li et al. [28] focus on inference under occlusion and do not explicitly model the uncertainty. Along similar lines as this work, Sengupta et al. [44] uses error propagation, but the solution is applicable only in the rest pose with no rotations. When predicting Gaussians over both the shape and pose parameters, Sengupta et al. [46] defaults to using Monte Carlo (MC) sampling. To eliminate Monte Carlo sampling, POCO Dwivedi et al., [11] regresses uncertainty in a single forward pass. However, per-part uncertainty is derived heuristically and does not address the mapping of parameter distribution to the target distribution.
Uncertainty Estimation. Several approaches exist Nix & Weigend, [36]; Gal & Ghahramani [16]; Lakshminarayanan et al. [27]; Blundell et al. [5] to quantify uncertainty in the prediction. Among them, Kendall & Gal [22]; Nix & Weigend [36] jointly optimize the mean and variance of the prediction using GNLL. In our context, this translates to learning the 3DMM parameters and its variance. Transforming the parameter distribution to the mesh involves error propagation (for Guides in Metrology, 2008; Schaefer et al., 2023), with the linear variant being , where . As we show later, computing the full Jacobian is computationally expensive. Sengupta et al. [44] uses error propagation but assumes that there is no rotation w.r.t the canonical pose. TIC-TAC Shukla et al., [49] extends error propagation to general nonlinear networks, but remains computationally expensive and abstracts away the interpretability in the formulation. This leaves a gap in recent works on a closed-form transformation which is not only agnostic to any 3DMM formulation but is also accurate, fast and interpretable. With CLUE, we propose an efficient solution for error propagation tailored for 3DMMs.
Closed-form Local Uncertainty Estimation
Let be the random variable denoting a vertex after applying additive offsets, and be the random variable denoting the rigid transformation of the vertex. We derive CLUE by asking what is ? Our key insight is the law of total variance, which ties the uncertainty in as a function of an explanatory variable . We provide brief insights here and detail in the appendix (Sec. A).
Assumptions. We inherit our assumptions from the standard 3DMM formulation and related literature (Loper et al., 2015; Sengupta et al., 2021c;b; Schaefer et al., 2023; Dwivedi et al., 2024): the pose , identity , and expression parameters are independent. Next, we drop the pose correctives to allow for a simpler formulation, since the magnitude of change induced by the pose is much larger than those from pose correctives (15 v.s. 0.5cm) ([37]). Dropping pose correctives allows for a simpler formulation; yet for completeness we discuss incorporating pose correctives in the appendix (Sec: A.6).
Derivation
Additive Offset Variance. Let be the random variable denoting vertex after the additive offsets, , where the shape and expression parameters have variances and . Because is linear in these parameters, we have,
We denote the entries of by , .
Law of Total Variance. The skinned vertex is denoted as , where is the set of joints and are the skinning weights. We have two sources of randomness, (1) the rest-pose vertex , and (2) the pose . The law of total variance splits into the variance due to the pose, shape and expression:
Intuitively, the rotational variance quantifies how much a vertex moves when the pose varies for an expected body shape. The shape and expression variance quantifies the movement due to the shape and expression given the expected pose.
Rotational Variance. Let us consider the rest-pose vertices deterministic and let the pose vary. Because is composed along the kinematic chain, a small rotation of joint swings all descendent joints rigidly about the mean posed location of that joint. A point at thus moves by , a small angle times a lever arm. For the skinned vertex, the lever arm is averaged over the joints that carry it and lie below , and the displacements of all joints add:
A joint contributes only if a descendent of the joint skins vertex . For instance uncertainty travels down the chain, from the shoulder to the hand, but not across it. The lever arm grows with the distance from the joint, so the same angular uncertainty at the shoulder produces a larger positional uncertainty at the fingertip than at the elbow. For each joint, we have a Gaussian on its axis-angle coordinates in the axes of the parent joint, with independent per-axis variances . An increment of these coordinates is the small rotation composed with the mean rotation, where is the left Jacobian of . Thus, the covariance of the perturbation is therefore , where is the mean global rotation of the parent. The variance of the displacement comes from the cross product. Intuitively, a rotation about moves the point in by its lever arm, and likewise for the other axes. Evaluating the lever arm at the mean vertex, with components (), and writing for the entries of , the rotational variance is
Shape and Expression Variance. Let us now fix the pose at its mean and let the rest-pose offsets vary. The expected posed vertex is . The offset covariance of Eq. (2) is simply rotated into the posed body by the vertex’s blended rotation . Using our identity and keeping the diagonal of , we have
Substituting Eq. (4) and Eq. (5) into Eq. (3) gives the per-vertex variance of the 3DMM formulation.
In contrast to first order error propagation, CLUE leverages the 3DMM formulation without needing to compute the full Jacobian. The per-vertex variance cleanly disentangles the different sources of uncertainty and combines them through a simple summation. As a result, estimating this quantity is inexpensive in terms of compute time and memory while remaining accurate. We provide pseudo-code in the appendix (Sec. C) as well as the code implementation for FLAME, SMPL-X and MHR in the repository.
Training: CLUE with SAM 3D Body
Methods such as Sengupta et al. [46]; Dwivedi et al. [11] which rely on the negative log-likelihood to train the variance may suffer from suboptimal convergence. This is because the variance head lacks direct supervision and may take on arbitrary values that minimize the objective sub-optimally [53, 19, 49, 50]. In Shukla et al. [49], a solution was proposed which used inductive bias when modeling the predicted variance. CLUE presents one such inductive bias by incorporating the geometry of the 3DMM when learning the parameter variance.
Variance Head. CLUE is architecture agnostic, hence we make no changes to the architecture of SAM 3D Body. We use minimal additional parameters to create a variance head. The variance head takes as input the pose token from SAM 3D Body which is also consumed by the MHR head. In addition, the variance head also takes in the mean parameters output by the MHR head. Both the pose token and the mean parameters are concatenated before being fed to a two layer feedforward network with one hidden layer of width 1024 and ReLU activation. The output of this network is the variance for the shape and pose parameters, constrained by sigmoid activation but scaled by learnable scalars. These predicted variances are consumed by CLUE. Note that all our probabilistic baselines share the same variance head architecture.
Losses. By defining a transformation between the parameter, vertex and joints space, CLUE defines an equivalency between them. This implies that the parameter variance can be mapped to vertex and joint space without any additional learnable parameters. This is particularly useful because while the joints are traditionally inferred from the vertices, CLUE allows inferring the joints as well as their variances from the vertices trivially. Thus, this enables CLUE to use the joint-space negative log-likelihood (joints-NLL), replacing the traditional deterministic joint-space loss. Therefore, the geometry of the 3DMM guides the optimization of the parameter variance, which has to explain both: the parameter-NLL as well as joints-NLL. To compute joints-NLL, we assume that neighboring vertices are correlated. Hence the joint standard deviation is the weighted average of the vertex standard deviation, . In contrast, assuming independence meant that the joint variance was underestimated by 5 to 13×.
Limitations
CLUE like jacobian based error propagation is a linear approximation, and overestimates the per-vertex variance when used as a statistical transformation. This is particularly visible for extreme body poses as shown in Figure 2. Furthermore, CLUE leverages the structure of the 3DMM and does not extend to neural shape representation approaches. Finally, CLUE makes several approximations to drastically reduce the number of computations needed without sacrificing accuracy, such as first-order analysis and dropping covariances between the vertices.

Figure 2. FLAME Error Map: We visualize the per-vertex error between the ground truth variance (computed with samples) and the estimated variance from different baselines. To do so, we synthesize different faces by setting randomly chosen parameters and their associated variance to non-zero. We observe that sampling with , Jacobian based error propagation and CLUE have low errors. Unlike sampling, Jacobian and CLUE may overestimate the variance due to first order assumptions. Additional visualizations in Figure 7.

Figure 7. Complete error maps for FLAME, SMPLX, MHR. Both, Jacobian based linear error propagation and CLUE overestimate the variance due to first order assumptions. However when used in training, the learnt parameter variances compensate for them.
Experiments
We perform two sets of experiments, designed to evaluate the accuracy of CLUE as a transformation from parameter-space to vertex-space, as well as the role of CLUE to improve optimization when training in-the-wild human mesh recovery models.
Statistical Evaluation
The first set of experiments evaluates CLUE as a transformation from parameter-space to the space of meshes, studying accuracy and compute cost. These experiments rely on synthesizing parameter variance, and comparing how CLUE propagates these variances in comparison to sampling with and error propagation. We perform these studies across FLAME, SMPL-X and Momentum Human Rig (MHR).
| Method | Time (ms) | Mem. (MB) | GFLOPs |
| Sampling | 138.7 | 14.2 | 1.43 |
| Sampling | 1379.4 | 140.9 | 14.26 |
| Jacobian | 226.4 | 410.4 | 6.01 |
| CLUE (ours) | 3.4 | 4.3 | 0.039 |
Table 1. FLAME compute. Median of 10 trials after 3 warm-ups. Additional results in Table 5.
Accuracy (Figure 2). We compute the ground truth variance by simulating parameter variance, drawing samples, and computing per-vertex variance empirically from the resulting meshes. For each baseline, we compute the absolute difference between its variance and that of the ground truth. We visualize these as error maps by randomly setting different parameter groups and their associated variance. For sampling based methods, as the value of increases, there is a drop in the error across all parameter groups. We also observe that and jacobian error propagation closely approximates the ground truth. While CLUE similarly has low errors across different parameter groups, we highlight the intricate pattern on the forehead, which also comes up in the error map for CLUE. We hypothesize that since CLUE does not model pose correctives, it does not capture intricate changes in the mesh as the jaw joint moves.
Compute. We evaluate on runtime, peak incremental memory, and GFLOPs. We implement the Jacobian in forward mode in PyTorch [38]. We use the CPU for computing the transformations to reflect on-device computing constraints, where the GPU would be prioritized for batched inferring from the incoming data stream. CLUE has the fastest runtime, with error propagation via the Jacobian consuming the largest memory. CLUE requires MB of additional memory because the tensors needed to compute the variance are already on-device. For the larger SMPL-X and MHR rigs (Table 5), our implementation exploits the sparsity of the kinematic tree, computing lever arms only for the joints that skin a vertex and their ancestors. The low computational costs as well as accuracy of the transformation makes CLUE suitable for on-device uncertainty estimation.
| Method | FLAME | SMPL-X | MHR | ||||||
| ms | MB | GFLOPs | ms | MB | GFLOPs | ms | MB | GFLOPs | |
| Sampling | 141.6 | 14.2 | 1.43 | 368.5 | 29.5 | 5.42 | 681.7 | 44.5 | 1.71 |
| Sampling | 1404.7 | 140.9 | 14.33 | 3733.6 | 293.6 | 54.20 | 6802.4 | 442.9 | 17.09 |
| Jacobian | 228.6 | 410.4 | 6.08 | 177.0 | 383.6 | 10.23 | 3272.2 | 1984.4 | 6.40 |
| CLUE (ours) | 3.5 | 4.3 | 0.039 | 4.0 | 9.5 | 0.015 | 10.9 | 16.9 | 0.055 |
Table 5. Compute cost across body models. Time (ms), memory (MB) and GFLOPs of one variance evaluation. Median of 10 trials after 3 warm-ups on one CPU (Intel i7-8700).
Probabilistic In-the-wild Human Mesh Recovery
The second set of experiments studies CLUE when used to supervise the training of unimodal in-the-wild human mesh recovery models such as SAM 3D Body. We compare our method with deterministic training, and architecture-agnostic probabilistic losses for uncertainty such as 3DMM Parameter-NLL, Matrix Fisher, and Normalizing Flows (Sengupta et al., 2021c;b; Dwivedi et al., 2024; Kolotouros et al., 2021). We randomly initialize the SAM 3D Body Yang et al. [56] architecture with frozen DINOv3 Simeoni et al. [51] backbone as our HMR model. We describe the losses, datasets and training process in the appendix (Sec. B).
Quantitative Study
3DPW and EMDB. (Table 2) We observe that all probabilistic losses outperform the deterministic loss. By introducing inductive bias when modeling the mesh, CLUE results in improved optimization and is the strongest baseline over all metrics. We also report the evaluation of SAM 3D Body on our pipeline which we recall is trained on a significantly larger dataset. Our numbers differ from those published due to possible differences with the SAM 3D Body evaluation pipeline which is not public. While CLUE is competitive with SAM 3D Body on 3DPW, it falls short on EMDB. We hypothesize that probabilistic training can reason better when dealing with ambiguities, but cannot model poses that are out-of-distribution accurately. This suggests a strategy of probabilistic modeling coupled with a diverse dataset.
| Method | #Training Images | 3DPW MPJPE | 3DPW PA-MPJPE | 3DPW PVE | 3DPW PA-PVE | EMDB MPJPE | EMDB PA-MPJPE | EMDB PVE | EMDB PA-PVE |
| SAM 3D Body (released) | 3.8M (w/ multi-view) | 56.0 | 35.1 | 67.1 | 47.2 | 63.2 | 38.2 | 74.7 | 47.1 |
| Deterministic Training | 1.2M (single-view) | 62.7 | 37.2 | 72.9 | 49.7 | 71.9 | 43.4 | 85.5 | 52.6 |
| Parameter NLL | ” | 57.9 | 35.5 | 68.9 | 48.3 | 66.4 | 41.2 | 79.5 | 50.9 |
| Matrix-Fisher | ” | 58.0 | 35.7 | 68.8 | 48.1 | 68.8 | 42.5 | 81.7 | 51.8 |
| Normalizing Flow | ” | 58.3 | 35.6 | 68.9 | 48.0 | 68.0 | 41.4 | 80.7 | 50.8 |
| CLUE | ” | 56.1 | 34.9 | 67.0 | 47.2 | 65.4 | 40.8 | 77.9 | 50.7 |
Table 2. 3DPW and EMDB (MoCap SMPL). All metrics in mm (). The first row measures SAM 3D Body on our evaluation pipeline. The other rows compare deterministic and probabilistic losses under identical training and evaluation protocol.
Pseudo Ground Truth datasets. Table 3 scores each baseline against the released SAM 3D Body as ground truth on five image datasets without motion capture, in MHR space. By having the prediction and ground truth in MHR space, we can compute uncertainty related metrics. CLUE is the only probabilistic objective that improves every group of metrics. Figure 4 visualizes the spearman correlation between the vertex error after rigid alignment of the ground truth and predicted mesh. We observe that CLUE has the highest correlation across all groups.

Figure 4. Correlations across vertex groups. We compute the Spearman correlation between the predicted per-vertex variance and the rigidly aligned vertex error for each of the 19 MHR vertex groups. We observe that CLUE has a higher spearman correlation for each of the vertex groups, enabling applications in data mining and uncertainty based decision making.
| Method | MPJPE | PA-MPJPE | PVE | PA-PVE | vNLL | AUSE | |
| Deterministic | 31.6 | 19.7 | 35.4 | 24.2 | – | – | – |
| Param-NLL | 32.0 | 18.5 | 35.0 | 22.3 | 2.35 | 0.68 | 0.078 |
| Matrix-Fisher | 35.9 | 20.5 | 38.9 | 24.4 | 2.74 | 0.56 | 0.129 |
| Normalizing Flow | 32.0 | 18.1 | 35.3 | 22.2 | 3.08 | 0.69 | 0.077 |
| CLUE | 27.5 | 17.7 | 30.4 | 21.7 | 2.13 | 0.75 | 0.067 |
Table 3. Pseudo-ground-truth evaluation, averaged over five in-the-wild datasets (H36M, MPI-INF-3DHP, PoseTrack, CrowdPose, OCHuman): joint and surface error (mm, ), vertex NLL (), Pearson correlation between predicted uncertainty and error (), and the area under the sparsification error curve (AUSE, ; per person, ranking by the mean predicted vertex sigma). Per-dataset results are in the appendix (Table 4).
| H36M | MPI-3D | PoseTrack | CrowdPose | OCHuman | Average | |||||||
| Deterministic | 19.2 | 13.5 | 25.4 | 17.6 | 24.9 | 16.7 | 33.2 | 20.0 | 55.2 | 30.5 | 31.6 | 19.7 |
| Param-NLL | 22.6 | 13.2 | 27.9 | 17.0 | 25.4 | 15.5 | 31.3 | 17.8 | 53.1 | 28.8 | 32.0 | 18.5 |
| Matrix-Fisher | 26.2 | 15.0 | 33.7 | 19.9 | 32.5 | 18.8 | 35.3 | 19.8 | 51.6 | 28.9 | 35.9 | 20.5 |
| Normalizing Flow | 22.3 | 13.3 | 30.1 | 17.3 | 28.1 | 16.1 | 31.7 | 17.5 | 47.8 | 26.4 | 32.0 | 18.1 |
| CLUE | 18.2 | 12.7 | 24.8 | 16.2 | 24.6 | 15.7 | 29.1 | 17.7 | 40.5 | 26.1 | 27.5 | 17.7 |
| H36M | MPI-3D | PoseTrack | CrowdPose | OCHuman | Average | |||||||
| Deterministic | 22.6 | 17.2 | 28.7 | 21.5 | 28.3 | 20.6 | 35.9 | 24.2 | 61.2 | 37.3 | 35.4 | 24.2 |
| Param-NLL | 25.5 | 16.3 | 30.2 | 20.1 | 28.1 | 18.9 | 33.5 | 21.5 | 57.6 | 35.0 | 35.0 | 22.3 |
| Matrix-Fisher | 28.4 | 18.3 | 35.9 | 23.0 | 34.6 | 21.6 | 37.8 | 23.8 | 57.7 | 35.5 | 38.9 | 24.4 |
| Normalizing Flow | 25.1 | 16.9 | 32.7 | 20.7 | 30.7 | 19.4 | 34.1 | 21.3 | 53.7 | 32.7 | 35.3 | 22.2 |
| CLUE | 20.6 | 15.6 | 27.0 | 19.4 | 26.8 | 18.8 | 31.1 | 21.5 | 46.4 | 33.2 | 30.4 | 21.7 |
| H36M | MPI-3D | PoseTrack | CrowdPose | OCHuman | Average | |||||||
| Deterministic | – | – | – | – | – | – | – | – | – | – | – | – |
| Param-NLL | 2.10 | 0.68 | 2.30 | 0.71 | 2.22 | 0.69 | 2.32 | 0.69 | 2.83 | 0.66 | 2.35 | 0.68 |
| Matrix-Fisher | 2.58 | 0.36 | 2.71 | 0.51 | 2.70 | 0.61 | 2.73 | 0.62 | 2.96 | 0.66 | 2.74 | 0.56 |
| Normalizing Flow | 2.92 | 0.67 | 3.11 | 0.73 | 2.90 | 0.68 | 2.99 | 0.67 | 3.48 | 0.68 | 3.08 | 0.69 |
| CLUE | 1.82 | 0.71 | 2.00 | 0.72 | 2.07 | 0.74 | 2.15 | 0.77 | 2.61 | 0.79 | 2.13 | 0.75 |
| H36M | MPI-3D | PoseTrack | CrowdPose | OCHuman | Average | |||||||
| Deterministic | – | – | – | – | – | – | ||||||
| Param-NLL | 0.081 | 0.072 | 0.087 | 0.072 | 0.080 | 0.078 | ||||||
| Matrix-Fisher | 0.159 | 0.142 | 0.117 | 0.117 | 0.111 | 0.129 | ||||||
| Normalizing Flow | 0.080 | 0.068 | 0.084 | 0.074 | 0.080 | 0.077 | ||||||
| CLUE | 0.065 | 0.067 | 0.079 | 0.066 | 0.059 | 0.067 |
Table 4. Pseudo-ground-truth evaluation on five in-the-wild datasets: (a) joint accuracy, (b) surface accuracy (mm, ), (c) uncertainty quality — per-vertex NLL () and the Pearson correlation between predicted and vertex error (), and (d) the area under the sparsification error curve (AUSE, ; per person, ranking by the mean predicted vertex , normalised by each dataset’s full-set error). (a) Joints: MPJPE / PA-MPJPE. (b) Surface: PVE / PA-PVE. (c) Uncertainty: vertex-NLL / –error correlation. (d) Uncertainty: AUSE.
Qualitative Study
Error Maps. Figure 3 colours each prediction by its per-vertex error against the reference mesh, on one shared scale. We observe two patterns. First, the baselines struggle with limbs that are occluded or have complex poses. CLUE is resistant to occlusion due to the inductive bias of the mesh in modeling the variance and allows for probabilistic reasoning under occlusion. Second, in complex body poses such as floor exercises and splits, the parameter-space baselines spread error over the torso and head as well, whereas CLUE confines the error to the extremities.

Figure 3. Per-vertex error maps. Each row shows the input, the reference mesh (yellow), and the predictions of different baselines, coloured by the per-vertex error. The largest baseline errors sit on limbs that are occluded or having complex poses. CLUE uses the mesh as an inductive bias when predicting the variance, allowing it to have lower errors when dealing with ambiguity. Additional visualizations are shown in Figure 8.

Figure 8. Error Maps for in-the-wild Human Mesh Recovery. We show that CLUE has higher accuracy when dealing with occlusions or complex poses.
Uncertainty based data mining. Since CLUE predicts fine-grained uncertainty maps (evidenced by high spearman correlations), we can answer questions that parameter-space uncertainty cannot: which vertex groups of the body is the model unsure about? For this study, we look for a limb whose uncertainty stands out from the rest of the body. However, raw uncertainty values are not comparable as they differ in scale between methods and vertices. We therefore convert each vertex’s uncertainty into a percentile, i.e. how unusual that value is for that vertex and that method, measured against the SA-1B training set. We also recall that per-vertex variances are computed empirically for all baselines except CLUE. In Figure 5 we observe that CLUE correctly distinguishes vertex groups which are occluded or undergo motion blur, while the rest of the body remains confident. The baselines saturate uncertainty across the whole body. While they can say that an image is hard, they cannot localize where.

Figure 5. Uncertainty based data-mining. Per-vertex variance is computed empirically by sampling for all other baselines, and using CLUE for itself. Similar to spearman correlation, for each method we normalize a vertex’s uncertainty based on its per-vertex percentile for that method. These images were retrieved algorithmically by querying for images with the highest uncertainty discrepancy between a limb and the body. In contrast to CLUE, the baselines’ uncertainty on the same images is body-wide or diffused and lacks fine-grained resolution. Additional visualizations are shown in Figure 9.

Figure 9. Uncertainty based data-mining.
Uncertainty based decision making. We present a use-case in egocentric face tracking. This study uses an internal dataset and GNM Ploumpis et al. [40] as the 3DMM. The goal is to regress a 3D mesh that reproduces the lower-face expression of a person from egocentric viewpoints. This setup is typical of VR-headsets such as Apple Vision Pro, Meta Quest Pro, and Galaxy XR, which further impose strict runtime and memory restrictions to enable realtime on-device operation. We employ a similar training protocol as described earlier to train the probabilistic regressor. We observe in Figure 6 that as the degree of occlusion increases, the uncertainty is localized and scaled correctly. Given this uncertainty signal, a system no longer has to track blindly through an occlusion: it can pause face tracking, lower its weight in the fusion, or hand over to another sensor until the face is visible again.

Figure 6. Uncertainty for downstream decision making. Top: frames from a headset’s egocentric camera with the mouth progressively occluded. Bottom: the tracked 3DMM per-vertex standard deviation (mm). The uncertainty concentrates on the occluded region, rising monotonically with the degree of occlusion. With this signal, a system can suspend or down-weight vision-based face tracking during occlusion, or defer to another modality, instead of tracking blindly.
Conclusion
We proposed CLUE, a novel formulation which transforms the parameter-space distribution of a 3DMM onto the mesh in closed-form. Through statistical experiments across multiple 3DMMs, we showed that CLUE is an accurate transformation while being computationally efficient. We also noted that in comparison to a direct application of linear error propagation, CLUE is interpretable in terms of formulation and visually. We studied integrating CLUE for in-the-wild human mesh recovery training and inference. Because CLUE carries the predicted parameter variance through the geometry of the 3DMM, it serves as an inductive bias to train the variance head. Subsequently, we show that CLUE outperforms deterministic training as well as architecture agnostic unimodal uncertainty losses. Next, we analyze the variance learnt by CLUE, showing that it has the highest spearman correlation for all vertex groups, and discuss applications of the learnt uncertainty in data mining and downstream decision making. Finally, we share SAM 3D Body (CLUE) to make per-vertex uncertainty a standard output of in-the-wild mesh recovery.
AI USE STATEMENT
In this work, we used generative AI tools for minor edits in writing. This includes finding ways to convey our thoughts in a compact fashion. GenAI tools were also used for grammatical checks or minor corrections. Bulk of the paper was written by humans, for humans. No AI was used in writing this statement. We additionally used GenAI tools for code implementation and documentation. However, there was always a human-in-the-loop to verify code correctness. Finally, GenAI was used to double-check the math in this paper. However, the authors developed the idea and formulated the derivation. We take responsibility for the final content of this work, including text, claims or artifacts produced with the aid of generative AI.
REPRODUCIBILITY STATEMENT
We provide significant implementation details: the pseudo-code for CLUE, and the variance head when using CLUE for in-the-wild human mesh recovery. We also use the open-weight SAM 3D Body model as well as publicly available datasets. Barring our study on downstream decision making, all other experiments can be reproduced with publicly available data, models, and the implementation details provided in the paper.
REFERENCES
Derivation Details
We provide a detailed, annotated version of the derivation for convenience. We also provide code in the accompanying repository for 3DMM specific modifications, and present our general analysis here.
Assumptions and approximations. CLUE uses the law of total variance and first-order error propagation specialised to the skinning equation. We make several assumptions that allow this approximation to be real-time yet accurate in comparison to large-scale sampling and jacobian based error propagation.
Independence and diagonal covariance. Following literature, , and are mutually independent with diagonal covariances, and the perturbations of different joints are independent with zero mean. The pose uncertainty of joint is a Gaussian on its axis-angle coordinates and is expressed in the axes of the parent of . The predicted per-axis variances and its effect on the rotation is a small rotation composed with the mean rotation. We keep only per-vertex variances and not covariances for simplicity.
First order in the pose. We drop terms of second or higher order in the pose perturbations. In particular , and we drop the cross term. This simplicity allows us to have a lower computational cost in relative to full error propagation.
Joint locations at the mean shape. When analyzing the shape and expression variance, the joint locations and hence are held at the mean.
No pose correctives. We don’t model pose correctives, but show how they can be added if needed in this derivation.
Preliminary: 3DMM Formulation
Rest-pose vertex. Vertex in the rest pose is the template vertex with additive linear offsets of shape and expression,
Here, and and similar for the expression component.
Kinematic chain. Each joint has a parent , a rest-pose location , an offset from its parent, and a local rotation given by the pose , where is the set of joints. Let be the ordered set of joints on the path from the root to , including . The global transformation of joint is the chain product
where is the global rotation of joint and its location in the posed body. The chain can be represented as a recursive relation between a joint and its parent, and .
We use two properties. First, appears in if and only if ; we write for the joints at or below . Second, each factor is a translation followed by a rotation,
Pivot removal. SMPL applies after removing the rest-pose transformation, . In the rest pose all rotations are the identity, so and
Skinning. The posed vertex is a weighted average of the vertex transformed by each joint,
with the skinning weights, which are non-zero only for a few joints near vertex .
Additive Variance.
We return to the formulation of the rest pose vertex. The shape and expression parameters have variances and . Since is linear in these parameters and using , the covariance of the rest-pose vertex is
We denote the entries of by , .
Law of Total Variance.
We have two independent sources of randomness in the posed vertex : namely the shape and expression parameters, through , and the pose , through the transformations . The law of total variance separates them,
The first term asks that for a fixed rest-pose vertex, how much does the vertex move when the pose varies? The second asks whether for the expected pose, how much does the vertex move when the shape and expression vary? We derive the two terms in turn.
Rotation Variance.
The intuition is to fix the rest-pose vertex and let only the pose vary.
Step 1: Effect of one joint's perturbation (joint a) on one transformation (j)
Let . Splitting the chain product of at joint and using the translate-then-rotate form of the factor,
where the block form of follows from the recursion . is the rigid transformation that places the frame of joint in the world: rotation , origin at the posed joint location .
Now let us perturb the rotation of joint . We have a Gaussian on the axis-angle coordinates of the joint, with , in the axes of the parent of . The perturbed rotation is the unperturbed one composed with a small rotation,
where is the left Jacobian of , which maps an increment of the axis-angle coordinates to the equivalent small rotation. Substituting it in we have,
the perturbed transformation is the unperturbed one followed by an extra factor. To see what this factor does, follow a point through its three matrices, from right to left.
(i) expresses in the frame of joint . The inverse of a rigid transformation subtracts the translation and undoes the rotation,
(ii) applies the small rotation in that frame,
(iii) brings the result back to world coordinates,
Using the property, and (i), the last term equals . Writing for the perturbation expressed in world coordinates, the extra factor acts as
Intuitively, perturbing the rotation of joint rotates every point carried by the chain below by the small angle about the posed location of joint .
From the joint transformation to the vertex. We write for the extra factor, so that and for any point . To accurately locate the vertex, we need to adjust for the pivot. However, pivot removal for and is the same since this step does not depend on the pose, perturbed or not. Therefore, . Hence, acts on the posed vertex like on any other point. For every joint at or below ,
For , does not appear in the chain of , so there is no factor and is unchanged.
Step 2: Effect of perturbing all joints
Let every joint be perturbed. Each contributes an extra perturbation. Expanding the product and dropping products of two or more small terms, the displacements of Step 1 add up to,
where a bar denotes evaluation at the mean pose . Substituting into the skinning equation (Eq. 6),
The double sum runs over the pairs with , which is the same set of pairs as . Exchanging the order of summation therefore gives , and since does not depend on it factors out of the inner sum
For each joint it collects exactly those skinning terms whose joint lies at or below , the only terms that affects. Since does not depend on ,
with the total skinning weight of vertex on the joints at or below . We call the lever arm of vertex about joint . It vanishes when no joint in skins vertex , so only the joints on the chains above vertex contribute.
Step 3: variance
The first sum in Step 2 is deterministic whereas the second is the random. Writing and assuming the perturbations of different joints to be independent with zero mean,
Here, are the predicted per-axis variances of the axis-angle coordinates of joint , in the axes of its parent, is the left Jacobian of Step 1 evaluated at the mean pose, and is the mean global rotation of the parent. is the covariance in world axes, , . The outer expectation over in Eq. (8) is taken to first order by evaluating the lever arm at the mean rest-pose vertex ,
Writing as the cross-product matrix of this vector, the rotation variance is the diagonal of . This gives us
Shape and Expression Variance.
Now let the pose take its expected value and let only the shape and expression vary. To first order in the pose perturbation (A2), the expected transformation of each joint is the transformation at the mean pose, . The expected vertex is then
We note that is random only in its first three components (the offsets). The joint locations and the translations are evaluated at the mean shape and interact only with the homogeneous coordinate of 1 and do not contribute to the variance. What remains is the skinning-blended rotation
applied to the rest-pose offsets. Using with the covariance of Eq. (7), the shape and expression variance is the diagonal of ,
Intuitively, the rest-pose offset covariance of the vertex is rotated into the posed body by the vertex’s own blended rotation.
Finally, substituting Eq. (9) and Eq. (10) into Eq. (8) gives the per-vertex variance
Pose Correctives
We use STAR as the 3DMM to analyse pose correctives. Following STAR’s notation convention, is the per-joint pose corrective function which regresses corrective offsets given the joint and its direct neighbors’ quaternion values.
Here, the second principal component as well as are stochastic, while and are deterministic. We note that the variance of this equation follows the familiar format where is a deterministic matrix and is random.
Since the activation weights are fixed, is a deterministic vector. The variance of a hadamard product is . This allows us to compute the variance from pose correctives.
Training Details
Training. All methods share the same training recipe and differ only in the probabilistic modeling. The DINOv3 ViT-H/16+ image encoder is initialised and kept frozen. All other modules of the SAM 3D Body architecture are randomly initialized and trained from scratch. We optimise with AdamW (learning rate , , weight decay 0.05), a linear warm-up over the first 1k steps followed by cosine decay to 5% of the peak learning rate, and gradient clipping at norm 1.0. Each step processes 32 images per GPU on 4 GPUs (128 images per step); we train for 30k steps. To train SAM 3D Body (CLUE), we freeze SAM 3D Body and only keep the variance head learnable. We train this head for 10k steps.
Losses. All methods share the same deterministic terms: an L2 loss on the 2D keypoints using the predicted perspective camera, an L2 loss on the 3D joints, and a joint-limit penalty on body joint angles. The methods differ only on how the MHR parameters are supervised. Deterministic training uses L2 loss on all MHR parameters. Parameter-space NLL replaces these deterministic losses on the MHR parameters. Normalizing flow applies NLL and RealNVP flow NLL on the joint parameters, keeping shape loss deterministic. Matrix-Fisher uses a matrix-Fisher NLL on each supervised joint rotation, with concentrations predicted hierarchically along the kinematic chain, and the Gaussian NLL for lengths and shape. CLUE uses the same losses as parameter-space NLL, but replaces L2 loss over the joints with NLL. All loss terms have unit weight except the joint-limit penalty (0.01).
Metrics. We follow standard practices and report (1) joint based metrics such as MPJPE and PA-MPJPE, and (2) vertex based metrics such as PVE and PA-PVE. When the prediction and (pseudo) ground truth are in the same space (MHR space), we additionally report variance based metrics such as vertex-NLL, spearman and pearson correlation AUSE. Since baselines measure uncertainty in the parameter space, we draw 100 samples per prediction to compute the mesh space variance.
Datasets. For training, we use the Segment Anything-1B (SA-1B) dataset which consists of 1.2 million+ training images Kirillov et al. [23]. Additionally, we use MPII and COCO with MHR annotations for training. For our evaluation, we use 3DPW and Mocap 3D, with 3DPW as our primary dataset containing ground truth motion capture data in SMPL format. Since our predictions are in MHR space and the ground truth is in SMPL, we use on-the-fly optimization to map MHR to SMPL as done in Yang et al. [56]. Ferguson et al. [14]. Additionally, we also use MPII Andriluka et al. [1], Human3.6M Ionescu et al. [20], COCO Lin et al. [33], MPI-INF-3D Mehta et al. [35], PoseTrack Andriluka et al. [2], CrowdPose Li et al. [29] and OCHuman Zhang et al. [57]. Unlike 3DPW, these datasets do not have 3DMM annotations, and hence we use SAM 3D Body as the pseudo ground truth.
Pseudo-Code
We present the algorithm for an efficient implementation of Closed-form Local Uncertainty Estimation in Algorithm 1.
Full Results
We gather our results and visualizations in this section.
Algorithm 1 CLUE
Require: : shape and expression parameter variances concatenated; : pose variances, per axis-angle coordinate in the parent’s axes; : template mesh; : blendshape basis (shape expression); : joint regressor; : LBS weights; : mean shape and expression coefficients concatenated; : mean pose parameters (axis-angle); : parent map
Ensure: : per-vertex variance
— Precompute once at initialisation to reduce runtime —
Build ancestor matrix .
for do
while do ▷ No parent is represented as -1
;
end while
end for
▷ , outer product per vertex and blendshape
▷ Fusing skinning weights and ancestor matrix
— Per-forward pass —
Step 1: Shape / expression covariance ▷ Eq. (7)
▷ , offset covariance
Step 2: Forward kinematics
▷ , shaped mesh
▷ , joint locations
▷ rotations and translations of the pivot-removed
▷ , posed joint centres (pivots)
Step 3: Vertex transformation
Modelling effect of different joint transformations on the vertex
▷ ; ,
Broadcast helps in parallelizing offset calculation w.r.t pivot
▷ rotation around pivot;
Reducing this w.r.t fused skinning weights + ancestor matrix
▷ ; , lever arm about joint
Step 4: Rotational variance ▷ Eq. (9)
▷ each
▷ , left Jacobian of at the mean pose
▷ , world axes,
▷
Step 5: Rotated shape / expression variance ▷ Eq. (10)
▷ , LBS-blended rotation
▷
Step 6: Total variance ▷ Eq. (11)
return
References
- [1]Mykhaylo Andriluka, Leonid Pishchulin, Peter Gehler, and Bernt Schiele. 2d human pose estimation: New benchmark and state of the art analysis. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2014.DOI
- [2]Mykhaylo Andriluka, Umar Iqbal, Eldar Insafutdinov, Leonid Pishchulin, Anton Milan, Juergen Gall, and Bernt Schiele. Posetrack: A benchmark for human pose estimation and tracking. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2018.arxiv.org/abs/1710.10000
- [3]Benjamin Biggs, David Novotny, Sebastien Ehrhardt, Hanbyul Joo, Ben Graham, and Andrea Vedaldi. 3d multi-bodies: Fitting sets of plausible 3d human models to ambiguous image data. Advances in neural information processing systems, 33:20496–20507, 2020.arxiv.org/abs/2011.00980
- [4]Volker Blanz and Thomas Vetter. A morphable model for the synthesis of 3d faces. In Seminal Graphics Papers: Pushing the Boundaries, Volume 2, pp. 157–164. 2023.
- [5]Charles Blundell, Julien Cornebise, Koray Kavukcuoglu, and Daan Wierstra. Weight uncertainty in neural network. In International conference on machine learning, pp. 1613–1622. PMLR, 2015.
- [6]Federica Bogo, Angjoo Kanazawa, Christoph Lassner, Peter Gehler, Javier Romero, and Michael J Black. Keep it smpl: Automatic estimation of 3d human pose and shape from a single image. In European conference on computer vision, pp. 561–578. Springer, 2016.arxiv.org/abs/1607.08128
- [7]Hanbyel Cho and Junmo Kim. Generative approach for probabilistic human mesh recovery using diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 4183–4188, 2023.arxiv.org/abs/2308.02963
- [8]Yu Deng, Jiaolong Yang, Sicheng Xu, Dong Chen, Yunde Jia, and Xin Tong. Accurate 3d face reconstruction with weakly-supervised learning: From single image to image set. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops, pp. 0–0, 2019.arxiv.org/abs/1903.08527
- [9]Rahul Dey and Vishnu Naresh Bodd etti. Generating diverse 3d reconstructions from a single occluded face image. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1547–1557, 2022.arxiv.org/abs/2112.00879
- [10]Olaf Dünkel, Tim Salzmann, and Florian Pfaff. Normalizing flows on the product space of so (3) manifolds for probabilistic human pose modeling. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2285–2294, 2024.arxiv.org/abs/2404.05675
- [11]Sai Kumar Dwivedi, Cordelia Schmid, Hongwei Yi, Michael J Black, and Dimitrios Tzionas. Poco: 3d pose and shape estimation with confidence. In 2024 International Conference on 3D Vision (3DV), pp. 85–95. IEEE, 2024.arxiv.org/abs/2308.12965
- [12]Bernhard Egger, Andreas Schneider, Clemens Blumer, Sandro Schönborn, and Thomas Vetter. Occlusion-aware 3d morphable face models. 2016.DOI
- [13]Bernhard Egger, Sandro Schönborn, Andreas Schneider, Adam Kortylewski, Andreas Morel-Forster, Clemens Blumer, and Thomas Vetter. Occlusion-aware 3d morphable models and an illumination prior for face image analysis. International Journal of Computer Vision, 126(12): 1269–1287, 2018.DOI
- [14]Aaron Ferguson, Ahmed AA Osman, Berta Bescos, Carsten Stoll, Chris Twigg, Christoph Lassner, David Otte, Eric Vignola, Fabian Prada, Federica Bogo, et al. Mhr: Momentum human rig. arXiv preprint arXiv:2511.15586, 2025.arxiv.org/abs/2511.15586
- [16]Yarin Gal and Zoubin Ghahramani. Dropout as a bayesian approximation: Representing model uncertainty in deep learning. In international conference on machine learning, pp. 1050–1059. PMLR, 2016.arxiv.org/abs/1506.02142
- [17]Google. XNNPACK: High-efficiency floating-point neural network inference operators for mobile, server, and web, 2026. URL https://github.com/google/XNNPACK. Accessed: 2026-02-24.
- [19]Alexander Immer, Emanuele Palumbo, Alexander Marx, and Julia Vogt. Effective bayesian heteroscedastic regression with deep neural networks. Advances in neural information processing systems, 36:53996–54019, 2023.DOI
- [20]Catalin Ionescu, Dragos Papava, Vlad Olaru, and Cristian Sminchisescu. Human3.6m: Large scale datasets and predictive methods for 3d human sensing in natural environments. IEEE Transactions on Pattern Analysis and Machine Intelligence, 36(7):1325–1339, jul 2014.DOI
- [21]Angjoo Kanazawa, Michael J Black, David W Jacobs, and Jitendra Malik. End-to-end recovery of human shape and pose. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 7122–7131, 2018.arxiv.org/abs/1712.06584
- [22]Alex Kendall and Yarin Gal. What uncertainties do we need in bayesian deep learning for computer vision? In I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017. URL https://proceedings.neurips.cc/paper_files/paper/2017/file/2650d6089a6d640c5e85b2b88265dc2b-Paper.pdf.
- [23]Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C. Berg, Wan-Yen Lo, Piotr Dollár, and Ross Girshick. Segment anything. arXiv:2304.02643, 2023.arxiv.org/abs/2304.02643
- [24]Muhammed Kocabas, Chun-Hao P Huang, Otmar Hilliges, and Michael J Black. Pare: Part attention regressor for 3d human body estimation. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 11127–11137, 2021.arxiv.org/abs/2104.08527
- [25]Nikos Kolotouros, Georgios Pavlakos, Michael J Black, and Kostas Daniilidis. Learning to reconstruct 3d human pose and shape via model-fitting in the loop. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 2252–2261, 2019.arxiv.org/abs/1909.12828
- [26]Nikos Kolotouros, Georgios Pavlakos, Dinesh Jayaraman, and Kostas Daniilidis. Probabilistic modeling for human mesh recovery. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 11605–11614, 2021.arxiv.org/abs/2108.11944
- [27]Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. Simple and scalable predictive uncertainty estimation using deep ensembles. Advances in neural information processing systems, 30, 2017.arxiv.org/abs/1612.01474
- [28]Chunlu Li, Andreas Morel-Forster, Thomas Vetter, Bernhard Egger, and Adam Kortylewski. Robust model-based face reconstruction through weakly-supervised outlier segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 372–381, 2023.arxiv.org/abs/2106.09614
- [29]Jiefeng Li, Can Wang, Hao Zhu, Yihuan Mao, Haoshu Fang, and Cewu Lu. Crowdpose: Efficient crowded scenes pose estimation and a new benchmark. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019, pp. 10863–10872. Computer Vision Foundation / IEEE, 2019. doi: 10.1109/CVPR.2019.01112. URL http://openaccess.thecvf.com/content_CVPR_2019/html/Li_CrowdPose_Efficient_Crowded_Scenes_Pose_Estimation_and_a_New_Benchmark_CVPR_2019_paper.html.DOI
- [30]Tianye Li, Timo Bolkart, Michael J Black, Hao Li, and Javier Romero. Learning a model of facial shape and expression from 4d scans. ACM Transactions on Graphics (TOG), 36(6):1–17, 2017.DOI
- [31]Zhihiao Li, Jianzhuang Liu, Zhensong Zhang, Songcen Xu, and Youliang Yan. Cliff: Carrying location information in full frames into human pose and shape estimation. In European Conference on Computer Vision, pp. 590–606. Springer, 2022.arxiv.org/abs/2208.00571
- [32]Kevin Lin, Lijuan Wang, and Zicheng Liu. Mesh graphormer. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 12939–12948, 2021.
- [33]Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In European conference on computer vision, pp. 740–755. Springer, 2014.arxiv.org/abs/1405.0312
- [34]Matthew Loper, Naureen Mahmood, Javier Romero, Gerard Pons-Moll, and Michael J. Black. SMPL: A skinned multi-person linear model. ACM Trans. Graphics (Proc. SIGGRAPH Asia), 34(6):248:1–248:16, October 2015.
- [35]Dushyant Mehta, Helge Rhodin, Dan Casas, Pascal Fua, Oleksandr Sotnychenko, Weipeng Xu, and Christian Theobalt. Monocular 3d human pose estimation in the wild using improved cnn supervision. In 3D Vision (3DV), 2017 Fifth International Conference on. IEEE, 2017. doi: 10.1109/3DV.2017.00064. URL http://gvv.mpi-inf.mpg.de/3dhp_dataset.DOI
- [36]D.A. Nix and A.S. Weigend. Estimating the mean and variance of the target probability distribution. In Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN’94), volume 1, pp. 55–60 vol.1, 1994. doi: 10.1109/ICNN.1994.374138.DOI
- [37]Ahmed AA Osman, Timo Bolkart, and Michael J Black. Star: Sparse trained articulated human body regressor. In European Conference on Computer Vision, pp. 598–613. Springer, 2020.DOI
- [38]Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32, 2019.arxiv.org/abs/1912.01703
- [39]Georgios Pavlakos, Vasileios Choutas, Nima Ghorbani, Timo Bolkart, Ahmed A. A. Osman, Dimitrios Tzionas, and Michael J. Black. Expressive body capture: 3D hands, face, and body from a single image. In Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), pp. 10975–10985, 2019.arxiv.org/abs/1904.05866
- [40]S. Ploumpis, J. Bednarik, G. Zoss, R. Guseinov, L. Prasso, P. Chandran, O. Boyne, V. Choutas, T. Bolkart, D. Wang, M. Chai, D. Qiu, S. Winberg, G. Rainer, L. Bridgeman, L. L. Helminger, E. Collins, D. Vicini, J. Riviere, Y. Boetzel, A. Koumis, S. Moschoglou, J. Busch, C. Herrera, J. Still, S. Ysebert, P. Lincoln, S. O. Escolano, C. Rehmann, E. Wood, T. Beeler, and S. Zafeiriou. Gmm head: A generative anthropometric model of the human head. 2026. URL https://arxiv.org/abs/2607.23687.
- [43]Pratheba Selvaraju, Victoria Fernández Abrevaya, Timo Bolkart, Rick Akkerman, Tianyu Ding, Faezeh Amjadi, and Ilya Zharkov. Ofer: Occluded face expression reconstruction. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 26985–26995, 2025.arxiv.org/abs/2410.21629
- [44]Akash Sengupta, Ignas Budvytis, and Roberto Cipolla. Probabilistic estimation of 3d human shape and pose with a semantic local parametric model. arXiv preprint arXiv:2111.15404, 2021a.arxiv.org/abs/2111.15404
- [45]Akash Sengupta, Ignas Budvytis, and Roberto Cipolla. Hierarchical kinematic probability distributions for 3d human shape and pose estimation from images in the wild. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 11219–11229, 2021b.arxiv.org/abs/2110.00990
- [46]Akash Sengupta, Ignas Budvytis, and Roberto Cipolla. Probabilistic 3d human shape and pose estimation from multiple unconstrained images in the wild. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 16094–16104, 2021c.arxiv.org/abs/2103.10978
- [47]Akash Sengupta, Ignas Budvytis, and Roberto Cipolla. Humaniflow: Ancestor-conditioned normal-ising flows on so (3) manifolds for human pose and shape distribution estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4779–4789, 2023.
- [48]Karthik Shetty, Annette Birkhold, Bernhard Egger, Srikrishna Jaganathan, Norbert Strobel, Markus Kowarschik, and Andreas Maier. Propliks: Probabilistic 3d human body pose estimation. arXiv preprint arXiv:2412.04665, 2024.arxiv.org/abs/2412.04665
- [49]Megh Shukla, Mathieu Salzmann, and Alexandre Alahi. TIC-TAC: A framework for improved covariance estimation in deep heteroscedastic regression. In Ruslan Salakhutdinov, Zico Kolter, Katherine Heller, Adrian Weller, Nuria Oliver, Jonathan Scarlett, and Felix Berkenkamp (eds.), Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pp. 45244–45257. PMLR, 2024. URL https://proceedings.mlr.press/v235/shukla24a.html.arxiv.org/abs/2310.18953
- [50]Megh Shukla, Aziz Shameem, Mathieu Salzmann, and Alexandre Alahi. Towards self-supervised covariance estimation in deep heteroscedastic regression. In International Conference on Learning Representations, volume 2025, pp. 93359–93381, 2025.arxiv.org/abs/2502.10587
- [51]Oriane Simeoni, Huy V. Vo, Maximilian Seitzer, Federico Baldassarre, Maxime Oquab, Cijo Jose, Vasil Khalidov, Marc Szafraniec, Seung Eun Yi, Michael Ramamonjisoa, Francesco Massa, Daniel HAZIZA, Luca Wehrstedt, Jianyuan Wang, Timothée Darcet, Théo Moutakanni, Leonel Santana, Claire Roberts, Andrea Vedaldi, Jamie Tolan, John Brandt, Camille Couprie, Julien Mairal, Herve Jegou, Patrick Labatut, and Piotr Bojanowski. DINOv3. Transactions on Machine Learning Research, 2026. ISSN 2835-8856. URL https://openreview.net/forum?id=2NlGyqNjns. Featured Certification.
- [52]Anastasis Stathopoulos, Ligong Han, and Dimitris Metaxas. Score-guided diffusion for 3d human recovery. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 906–915, 2024.arxiv.org/abs/2403.09623
- [53]Andrew Stirn, Harm Wessels, Megan Schertzer, Laura Pereira, Neville Sanjana, and David Knowles. Faithful heteroscedastic regression with neural networks. In International conference on artificial intelligence and statistics, pp. 5593–5613. PMLR, 2023.arxiv.org/abs/2212.09184
- [55]Timing Yang, Sicheng He, Hongyi Jing, Jiawei Yang, Zhijian Liu, Chuhang Zou, and Yue Wang. Fast sam 3d body: Accelerating sam 3d body for real-time full-body human mesh recovery. arXiv preprint arXiv:2603.15603, 2026a.arxiv.org/abs/2603.15603
- [56]Xitong Yang, Devansh Kukreja, Don Pinkus, Taosha Fan, Jinhyung Park, Soyong Shin, Jinkun Cao, Jia-Wei Liu, Nicolás Ugrinovic, Anushka Sagar, et al. Sam 3d body: Robust full-body human mesh recovery. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7209–7219, 2026b.
- [57]Song-Hai Zhang, Ruilong Li, Xin Dong, Paul Rosin, Zixi Cai, Xi Han, Dingcheng Yang, Haozhi Huang, and Shi-Min Hu. Pose2Seg: Detection free human instance segmentation. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 889–898. IEEE, 2019.arxiv.org/abs/1803.10683