WorksComputer Science
mmTherm: Absolute 3D Human Pose Estimation from Low-Resolution Thermal Images and mmWave Radar
Abstract
Absolute 3D human pose estimation (HPE) aims to recover both a person's body pose and their global position in 3D space. Privacy-preserving millimeter-wave (mmWave) radar provides distance and direction measurement capabilities, but its sparse point clouds make HPE under-constrained. Low-resolution thermal imaging can provide complementary observations of body direction and structure. However, our evaluation shows that a strong feature-level fusion baseline loses much of its localization and HPE advantage when generalizing to unseen subjects, with radial localization becoming less accurate than a radar-only baseline. Motivated by this finding, we introduce mmTherm, a lightweight framework that explicitly incorporates the complementary geometric properties of thermal imaging and mmWave radar. It combines body structure and joint directions estimated from thermal images with range information from radar to localize the body and reconstruct its 3D pose. Experiments on OctoNet show that mmTherm reduces cross-user absolute joint error by 33% and root localization error by 38% relative to the strongest corresponding baselines, while requiring substantially fewer parameters and less computation at inference.
Introduction
3D HPE plays an important role in understanding human activities and supports applications such as human-computer interaction, healthcare, and AR/VR. Traditional 3D HPE approaches rely on RGB cameras, which may degrade in low light and raise privacy concerns in settings such as smart homes. As an alternative, radio-frequency (RF) sensing has emerged as a promising tool for 3D HPE [1, 18, 36], as RF signals reflected from human bodies carry information about human movement. One widely used RF sensing modality is mmWave radar [12, 24, 32]. In practice, radar measurements are commonly processed into point clouds for 3D HPE [17, 23, 28].
However, point clouds generated by mmWave radar are typically sparse and noisy. First, radar signal reflections from the target depend on posture, body orientation, and radar viewpoint, making some body parts have few or even no detectable returns [1, 21, 26]. As a result, radar point clouds provide incomplete observations of the human body rather than dense measurements. Second, reflections from surrounding objects and multipath propagation can introduce background clutter and “ghost” points into point clouds [6, 11, 33]. Consequently, radar point clouds neither provide complete coverage of the human body nor reliably indicate which returns correspond to individual body parts (Figure 1a), making accurate 3D HPE challenging.
Figure 1. Complementary geometry for absolute 3D HPE. (a) Radar provides range information through sparse and noisy point clouds. (b) Thermal images provide coarse body structure and direction; orange planes illustrate depth ambiguity. (c) mmTherm explicitly combines thermal direction and radar range evidence for pose reconstruction.
To address these challenges, existing works either improve the quality of mmWave point clouds [21, 27, 28, 30, 33] or leverage deep learning methods to reconstruct human skeletons or meshes from sparse and noisy point clouds [4, 8, 9, 15, 16, 25]. Although point-cloud enhancement and deep-learning methods can suppress noise and infer missing points, the physical limitations of radar visibility are not removed. Therefore, for body parts that cannot be directly observed, their positions must be inferred from limited measurements. Consequently, despite these advances, the ambiguity caused by incomplete and noisy radar observations remains challenging and makes 3D HPE an under-constrained problem.
Recent works have incorporated learned human pose priors into mmWave point cloud-based HPE [8, 15] to constrain pose reconstruction from incomplete radar observations. While such priors improve the plausibility of generated human poses, they do not provide independent observations to resolve missing or ambiguous radar returns. We therefore introduce thermal imaging as a complementary sensing modality that provides additional geometric observations. By capturing body heat, thermal images reveal coarse body structures and direction, providing spatial priors for body regions with sparse radar returns and helping distinguish human targets from background clutter (Figure 1b). Specifically, we use a low-resolution () thermal array for cost and privacy considerations.
A natural way to incorporate these thermal observations into radar-based HPE is to fuse features from the two modalities. For example, MIRaSeg [22] employs feature-level fusion to learn cross-modal representations, with a jointly trained pose estimation pathway. Indeed, our reproduction of its pose pathway (MIRaSeg-Pose) achieves strong in-domain absolute 3D HPE performance, outperforming radar-based baselines [8, 15, 16] in both body localization and relative pose estimation. Surprisingly, however, its localization advantage largely vanishes when generalizing to unseen subjects. In our cross-user evaluation, MIRaSeg-Pose achieves a root localization error of 156 mm, comparable to the 154 mm of the radar-based baseline Wave2Body [15]. More notably, its radial localization error reaches 124 mm, more than twice that of Wave2Body (61 mm), despite achieving better directional accuracy. These findings suggest that implicit feature fusion, while effective within the training distribution, may learn dataset-specific relationships without reliably exploiting the distinct physical measurement capabilities of each modality. This motivates us to develop a fusion strategy that more explicitly accounts for the complementary geometric properties of the two modalities.
In this paper, we propose mmTherm, a lightweight geometry-guided fusion framework for absolute 3D HPE using mmWave radar and low-resolution thermal imaging. As shown in Figure 1c, the framework explicitly incorporates thermal direction and body-region geometry into learned fusion with radar-derived position and range evidence. In the first stage, thermal geometry guides a learned correction to the radar-estimated root location, producing an estimate of the global body position. In the second stage, the fused root defines a local radar neighborhood and a spatial reference for pose reconstruction. Thermal features and calibrated geometry guide joint-level radar point association. Joint directions inferred from thermal images are then combined with range estimates from radar evidence to initialize the pose, which is subsequently refined by a lightweight spatiotemporal network.
We evaluate mmTherm on OctoNet [31], a multimodal dataset with synchronized thermal and radar measurements and ground-truth 3D human poses. Under a cross-user split, mmTherm reduces absolute joint error by 33% and root localization error by 38% relative to the best-performing corresponding baselines, while requiring substantially fewer parameters and less computation. Further experiments show that the proposed geometry-guided framework maintains its advantage in absolute pose accuracy under strong clutter and severe radar point sparsity. Joint-level analysis additionally shows improved reconstruction of body parts without nearby radar returns compared with radar-only estimation, supporting the value of complementary thermal observations under incomplete radar coverage. Our contributions can be summarized as follows:
We investigate a generalization challenge in radar–thermal fusion-based 3D HPE: a strong feature-level fusion baseline achieves accurate in-domain localization but loses its advantage on unseen subjects. This observation motivates explicitly exploiting the complementary geometric capabilities of thermal imaging and mmWave radar.
We propose mmTherm, a lightweight two-stage framework that combines explicit thermal geometry with radar-derived position and range for root localization, followed by root-conditioned pose reconstruction, without requiring point-level human or clutter annotations.
We conduct extensive evaluations on OctoNet, demonstrating improvements in absolute 3D HPE under cross-user generalization and robustness to sparse and cluttered radar observations.
Related Work
3D HPE Using mmWave Radar. Early RF sensing systems such as RF-Pose demonstrate HPE capability of radio signals using visual supervision [36]. Subsequent systems, including mm-Pose, mmMesh, and MVDopplerPose, explore pose estimation and body reconstruction from mmWave radar observations [7, 10, 20, 24, 29]. However, sparse and noisy radar returns make pose reconstruction under-constrained [5, 9]. Existing efforts address these limitations through two broad directions. One improves radar point-cloud quality through signal processing or learned enhancement [26, 27]. OmniPC jointly exploits range, Doppler, and angular information [28], while PathClean suppresses multipath during point-cloud generation [33]. Learning-based methods such as mmDEAR and mmWaveFlow further enhance or generate point clouds for downstream reconstruction [21, 30]. The others develop deep learning methods for pose reconstruction, using human-body priors to improve inference plausibility or training coverage [4, 25]. Specifically, mmDiff incorporates body structure and motion continuity into conditional diffusion [8]. WiCompass learns a reference pose space from motion-capture data to guide the collection of underrepresented poses [16]. Wave2Body translates radar representations into a pretrained compositional body-token space [15]. Although these approaches improve radar-based HPE performance, the positions of body parts without radar returns must still be inferred from incomplete observations. mmTherm introduces complementary thermal measurements during inference, providing additional observations of coarse body structure and direction for absolute 3D HPE.
Thermal Sensing and Thermal–Radar Fusion. Low-resolution thermal arrays provide coarse spatial observations of the human body without conventional RGB imagery. TADAR investigates human detection and ranging [34], TAPOR reconstructs 3D hand poses [35], and other methods estimate body poses from ultra-low-resolution thermal images [2, 13]. Since these arrays do not directly measure depth, recovering range or 3D pose requires additional modeling or learned priors. Radar–thermal fusion has also been explored with higher-resolution thermal cameras. UTM uses uncertainty-guided feature fusion for human detection under visual degradation [3]; Zhao et al. associate infrared detections with radar tracks for nighttime pedestrian detection [37]; and RIDERS combines thermal images and radar measurements for dense metric depth estimation [14]. These thermal cameras retain much richer spatial detail than low-resolution arrays, placing them closer to conventional camera-based sensing. This does not match our design goal of low-cost sensing with deliberately limited visual detail, motivated by the resolution–privacy–cost trade-off of thermal sensors [19, 34]. More closely related, MIRaSeg fuses mmWave radar point clouds and low-resolution infrared images for human semantic segmentation, with a jointly trained pose pathway [22]. mmTherm is designed for absolute 3D HPE with a thermal array, explicitly using calibrated thermal directions and body regions to guide radar point association and pose reconstruction.
Method
Overview
Problem Formulation. mmTherm estimates an absolute 20-joint pose from synchronized low-resolution thermal images and mmWave radar point clouds. The co-located sensors are calibrated in a common radar coordinate system, in which we denote the predicted pose as . We use the pelvis as the root joint throughout this work.
mmTherm. Absolute 3D HPE requires both global body localization and articulation recovery. mmTherm achieves these through two stages of geometry-guided fusion (see Figure 2). Stage 1 combines thermal direction and body-region geometry with radar position and range to localize the root joint. Stage 2 selects radar points around the fused root and uses thermal features and geometry to associate them with individual joints. It then combines thermal-derived joint directions with radar-supported range measurements to initialize the pose, followed by a spatiotemporal refinement module.
Figure 2. Overview of mmTherm. Stage 1 combines thermal geometry with radar observations to localize the root. Using the predicted root and body box, Stage 2 estimates joint directions from thermal images and joint ranges through thermal-guided radar attention. These estimates form an initial 3D pose, which is then refined by a spatiotemporal network.
Geometry-Guided Root Joint Localization
Stage 1 first processes the two modalities through separate branches: the thermal branch estimates the root direction and body bounding box, while the radar branch predicts a 3D root position proposal from the point cloud. A fusion head then combines their outputs, using thermal geometry to guide a learned spatial correction to the radar proposal.
Thermal-Based Direction and Body Region Estimation. The thermal branch estimates the root direction and a coarse body region to guide radar-based localization and subsequent pose reconstruction. A convolutional encoder takes three consecutive normalized thermal frames and their per-frame validity flags, broadcast as image channels. It predicts a root heatmap and body bounding box , supervised by the projected ground-truth root and the bounding box of the projected skeleton. Soft-argmax yields the root location in the image. Using calibrated viewing rays, we convert this location into a unit direction and the box into an angular center with horizontal and vertical spans. These outputs describe the person’s direction and angular coverage without assigning depth.
Radar-Based Root Voting. Due to the sparsity and ambiguity, radar point clouds cannot be directly used to locate the root joint. This radar branch estimates the 3D position of the root joint by aggregating learned votes from individual points. It takes valid points from three temporal frames and encodes each point using its 3D coordinates, distance, Doppler velocity, and relative time, together with global mean- and max-pooled context. For each point, the branch predicts an offset toward a candidate root location and a contribution score . These predictions yield the root position proposal:
The weights are normalized over valid points. Both the aggregated proposal and individual votes are supervised by the ground-truth root position, without requiring point-level human or clutter annotations.
Geometry-Guided Fusion. The fusion head combines the radar branch-proposed position and range with thermal direction and body-region geometry to refine root localization. To represent their geometric relationship explicitly, we construct a descriptor containing , , pooled radar features, the directions and , their dot and cross products, and the angular spans of the thermal box. The dot and cross products encode directional agreement and discrepancy between the two estimates.
To describe observation quality and availability, also includes the maximum voting weight, normalized voting entropy, valid-point fraction, thermal heatmap peak, and thermal-frame validity flags. The unweighted centroid, log point count, and spatial spread additionally summarize the valid point cloud independently of learned voting weights. From this descriptor, an MLP predicts a spatial correction to the radar root location proposal:
The correction can adjust all three coordinates, so radar range is not fixed. The resulting fusion-based root position provides a spatial reference for the next stage.
Root-Conditioned Pose Reconstruction
Stage 2 reconstructs the full 3D pose from a thermal crop defined by and radar points selected around the fused root location . Since low-resolution thermal pixels and radar point clouds do not directly identify skeletal joints, the model learns joint-specific associations to estimate joint directions from thermal observations and joint ranges from radar evidence. These estimates initialize a 3D pose, which is then refined across joints and time.
Local Thermal and Radar Representations. To focus pose reconstruction on the person, we use the Stage 1 predictions to select the thermal body region and nearby radar points while preserving their geometric information. For thermal images, we expand by one pixel on each side to include a small margin around the predicted body region and resample the crop to . The thermal input has four channels: temperature standardized within the crop and the three components of the calibrated viewing direction. Temperature and direction are resampled together to preserve their spatial correspondence.
For each thermal frame, we process its paired radar point cloud and retain valid points within 1.5 of . Each retained point is represented by seven features: its 3D coordinates relative to the fused root, its distance to that root, and the three components of its unit direction from the radar origin. This encoding describes the point’s location relative to the person while preserving its viewing direction. We separately retain each point’s range from the radar origin for joint-range estimation.
Thermal-Estimated Joint Directions. We estimate each joint’s viewing direction from the thermal crop to guide radar association and pose initialization. A thermal encoder produces an grid of patch features. Twenty learned joint queries attend to these patches to obtain joint-specific thermal features. For each joint , we average the attention weights across heads and use them to aggregate the calibrated patch directions. Normalizing the resulting vector yields the estimated unit direction .
Thermal-Guided Radar Attention. We use thermal features and geometry to weight radar points according to their relevance to each joint. The radar attention query for joint is formed by adding a linear projection of its thermal feature to its learned joint query. The attention score for radar point combines feature compatibility with directional alignment and a body-region penalty:
Here, is the scaled query–key score, and is the angle between the estimated joint direction and the point’s direction from the radar origin. The penalty discourages attention to points whose directions fall outside the angular region defined by the thermal body box. The learned coefficients and are constrained to be nonnegative through softplus and are shared across attention heads. The penalty is zero inside the angular box. Outside it, we normalize the azimuth and elevation excesses by twice the corresponding median angular spacing between neighboring pixel rays. The penalty is half the sum of the squared normalized excesses, capped at six. Softmax normalization then produces joint-specific attention weights for aggregating radar features and estimating joint ranges.
Direction–Range Initialization. We combine joint directions estimated from thermal images with ranges inferred from radar points to construct an initial absolute 3D pose. We estimate each joint’s range as an attention-weighted sum of point ranges and a fallback range. The fallback is supplied by a learned root token that participates in the radar attention softmax without either geometric bias in Eq. (3). Its range is at inference and during training (§ 3.4). Combining range with thermal direction initializes every joint, including the root:
Direction and range share the radar origin as their reference.
Figure 5.
Spatiotemporal Refinement. We refine this approximate initialization across joints and time to account for coarse thermal resolution and sparse radar reflections that do not directly locate skeletal joints. Each joint receives the attention-weighted aggregation of retained point directions, excluding the root token. This radar direction falls back to the direction of only when no point is retained.
Joint tokens combine thermal and radar attention features, the encoded radar direction and fallback indicator, estimated range, and the dot product between thermal and radar directions. They also contain root-relative initialized coordinates, . With a body token, they pass through a spatial–temporal–spatial Transformer (depths 1–2–1) that exchanges information across joints and three frames, uses actual observation-time differences, and predicts coordinate corrections:
Although token coordinates are root-relative, corrections are added to the absolute initialization. The final root is refined with the other joints and may differ from Stage 1.
Training Objectives and Optimization
Staged Optimization. We first train the thermal branch with projected root and box supervision and the radar branch with 3D root supervision, including unweighted per-point vote losses. With both branches frozen, we train the fusion MLP using root-position error, Smooth L1 regression, and a correction-magnitude penalty weighted by 0.01. Its final layer is initialized near zero. Stage 2 is then trained and evaluated using predicted roots and boxes from the frozen Stage 1 model.
Pose Objectives. We supervise root-relative pose and absolute root location separately, together with the intermediate joint directions and ranges. The four objectives have equal weights:
The losses and supervise root-aligned joint positions and the final root position, respectively, using Smooth L1 with a transition scale of m. The direction loss measures cosine agreement, while the range loss uses Smooth L1 with a transition scale of m. All targets are derived from the ground-truth 3D skeleton.
Training Augmentations. To expose Stage 1 to sparse radar observations, half of the radar-branch and fusion-head training samples undergo radar point removal, with retention ratios sampled uniformly from . For Stage 2, we perturb the conditioning roots as , where one zero-mean Gaussian offset ( m per axis) is shared across all frames of a training clip. The perturbed roots affect only root-relative point encoding and the fallback range; cached radar points remain selected around the unperturbed roots. This discourages direct copying of the conditioning root into the final prediction. Both augmentations are disabled during validation and testing.
Experiments
Experimental Setup
Dataset. We evaluate mmTherm on OctoNet [31], which provides mmWave point clouds and thermal images from five nodes, with OptiTrack 3D pose ground truth. Recordings capture one person repeating an activity about ten times (median 67 s). Each prediction uses one radar–thermal pair, with training samples pooled across nodes.
Evaluation Protocols. For in-domain evaluation, recordings within each activity are split into training, validation, and test sets at a 7/1/2 ratio (254,174 test samples). Cross-user evaluation trains on users 1–7, validates on user 8, and tests on users 9–10 (233,518 samples). All node views and overlapping windows of a recording remain in the same split. We further evaluate unseen users and activities using Gym and Freestyle recordings from 29 additional people absent from the main splits (456,802 samples). These activities are excluded from training and checkpoint selection.
Metrics. We report errors in millimeters. Absolute (ABS) mean per joint position error (MPJPE) averages joint errors in the coordinate system. Root error measures pelvis-localization error. Root-aligned (RA) MPJPE subtracts each pose’s root without rotation or scale alignment. Both MPJPE metrics include all 20 joints. We also compare lateral root error, perpendicular to the true line of sight, and range error.
Comparisons. We compare mmTherm with mmDiff [8], WiCompass [16], Wave2Body [15], and MIRaSeg-Pose [22] under identical splits. The reproduced mmDiff pathway predicts root-relative poses (RA-MPJPE) only. MIRaSeg-Pose is our adaptation of MIRaSeg’s pose pathway to OctoNet’s absolute 20-joint skeleton, trained with pose supervision without segmentation labels.
Implementation Details. We train on an NVIDIA RTX PRO 6000 Blackwell Max-Q GPU. Stage 2 uses 128-dimensional tokens and four attention heads. We train Stage 2 and its ablations with AdamW (learning rate , weight decay 0.01), cosine decay over at most 60 epochs, and batches of 24 clips in FP32. We select the checkpoint with the lowest ABS MPJPE on the validation set.
Main Results
Table 1 shows that mmTherm outperforms all radar-only baselines on the available pose metrics under both protocols. Compared with MIRaSeg-Pose, in-domain absolute and root-aligned errors are comparable, while root localization improves. The advantage grows on unseen users, where mmTherm achieves lower errors on all three metrics, reducing absolute MPJPE and root error by 33% and 39%, respectively. The qualitative results in Figure 3 illustrate the benefits of incorporating thermal observations: both radar–thermal methods recover body postures more accurately than radar-only Wave2Body. Compared with MIRaSeg-Pose, mmTherm further improves both absolute body placement and limb articulation, as illustrated by the jumping-jack and jump-rope examples, producing skeletons that more closely match the references.
Table 1. 3D HPE errors on OctoNet (mm, lower is better). ABS: absolute MPJPE; Root: root position error; RA: root-aligned MPJPE.

Figure 3. Qualitative HPE results on unseen users. Each column uses the same frame across methods. Dashed skeletons are ground-truth poses. Values indicate absolute MPJPE in mm. The best result in each column is bold.
To test the combined shift in users and activities, we further evaluate frozen in-domain models on Gym and Freestyle. mmTherm achieves an absolute MPJPE of 193.2 mm and a root error of 138.0 mm, compared with 225.3 mm and 177.9 mm for MIRaSeg-Pose.
Robustness Evaluation
We next test whether the cross-user gains persist under ambiguous or incomplete radar observations, using strong clutter, point removal, and a missing-return diagnostic.
Strong Clutter. We select the 20% of cross-user recordings with the highest mean count of clustered radar points at least 1 m from the reference skeleton (0.35 m neighborhood radius; at least three points per cluster). Ground-truth poses serve only for offline selection. mmTherm achieves the lowest root-aligned and absolute errors on this subset (Figure 4a). The proxy does not control cluster origins or other recording differences.
Figure 4. Robustness evaluation. (a) The top 20% of recordings by the clutter ranking; black ticks show full-test errors. (b) Absolute MPJPE after radar point removal.
Radar Point Removal. We evaluate frozen models after randomly removing radar points before preprocessing, keeping thermal inputs unchanged (Figure 4b). At 10% retention, radar-only baselines show substantial increases in absolute MPJPE, while mmTherm exhibits a more moderate increase. MIRaSeg-Pose is the least sensitive to point removal, but mmTherm maintains lower absolute MPJPE at every retention level. This limited sensitivity may partly reflect weaker reliance on radar-derived range information in MIRaSeg-Pose’s feature-level fusion, consistent with its larger root and absolute errors on unseen users.
Missing Radar Returns. To examine pose reconstruction when local radar returns are missing, we count radar points within 0.2 m of each ground-truth joint location. Across the sampled frames, 79% of non-root joint locations have no points within this radius. Both radar–thermal methods achieve their largest root-aligned improvements over Wave2Body for these joints (Fig. 5a). The largest gains occur at the wrists, feet, and ankles (Fig. 5b).
Ablation Studies and Analysis
To understand the performance gains, we analyze root localization and pose reconstruction to understand the contributions of the two stages.
Figure 5 (PDF p. 7). Missing-return diagnostic. (a) Root-aligned error by radar point count within 0.2 m of reference joints. (b) Root-aligned error by body part. Error bars: 95% recording-bootstrap intervals.
Stage 1: Root Localization and Fusion. We first examine the final root predictions of the complete models in Table 2. MIRaSeg-Pose achieves good lateral accuracy, but its cross-user range error is approximately twice that of Wave2Body. mmTherm achieves the lowest range error under both protocols and the lowest lateral error on unseen users. These results suggest that effective radar–thermal fusion requires combining improved directional localization with reliable range estimation, particularly for unseen users. We therefore compare Stage 1 fusion strategies to examine the contribution of explicit thermal geometry.
| Method | In-domain Lat. | In-domain Range | In-domain 3D | Cross-user Lat. | Cross-user Range | Cross-user 3D |
| Wave2Body | 75.6 | 47.2 | 96.1 | 131.7 | 60.6 | 154.0 |
| MIRaSeg-Pose | 32.9 | 47.3 | 62.8 | 78.5 | 123.7 | 156.2 |
| mmTherm | 36.4 | 36.9 | 57.0 | 67.0 | 55.7 | 95.0 |
Table 2. Final root localization errors across methods on 20k sampled test frames (mm, lower is better).
To examine the role of root fusion, Table 3 compares the radar-only root proposal with feature fusion and our geometry-guided fusion. Feature fusion replaces explicit thermal geometry with encoder features and a node indicator while retaining the radar residual. Both fusion variants improve root localization over the radar-only baseline. Compared with feature fusion, our geometry-guided fusion further reduces lateral error while maintaining comparable range accuracy, lowering the 3D root error by 10.8 mm. These results support explicitly incorporating thermal geometry into root fusion.
| Stage 1 variant | Lateral | Range | 3D |
| Radar only | 123.0 | 59.9 | 145.1 |
| Feature fusion | 84.2 | 56.9 | 110.7 |
| Full fusion | 71.3 | 56.4 | 99.9 |
Table 3. Comparison of Stage 1 root fusion strategies on unseen users (mm, lower is better).
Stage 2: Pose Reconstruction. With Stage 1 conditioning fixed, we examine the cross-user results in Table 4. Removing Stage 2 radar points increases absolute MPJPE from 118.7 to 143.6 mm, while removing thermal temperatures increases it to 140.4 mm. Both ablations also worsen root localization and root-aligned pose accuracy. Jointly removing direction–range initialization and its auxiliary direction and range losses increases absolute MPJPE to 127.8 mm, with both root and root-aligned errors also increasing. These results support combining complementary radar and thermal observations with geometrically supervised initialization for pose reconstruction.
| Stage 2 variant | In-domain ABS | In-domain Root | In-domain RA | Cross-user ABS | Cross-user Root | Cross-user RA |
| Full model | 80.2 | 57.3 | 66.2 | 118.7 | 95.1 | 98.8 |
| w/o S2 radar points | 100.3 | 74.1 | 87.1 | 143.6 | 104.1 | 126.3 |
| w/o temperatures | 99.2 | 64.6 | 84.6 | 140.4 | 107.8 | 116.6 |
| w/o geometric init. | 83.1 | 62.0 | 65.5 | 127.8 | 98.2 | 101.0 |
Table 4. Stage 2 ablations (mm, lower is better). Removing temperatures preserves the thermal box and calibrated rays.
Model Cost
We compare model size, computational cost, and inference throughput in Table 5. Parameter counts include all components used at inference, and FLOPs are reported per pose as . Throughput is measured in FP32 with a batch size of 256. mmTherm uses the fewest parameters and FLOPs and achieves the highest throughput among the evaluated methods. Compared with MIRaSeg-Pose, it reduces the parameter count by a factor of 8.5 and the FLOPs by a factor of 11, while achieving 2.4× the throughput. These results demonstrate improved cross-user accuracy with a smaller model and lower inference computation.
| Method | Params (M) ↓ | GFLOPs ↓ | Throughput ↑ |
| mmDiff | 32.09 | 14.09 | 873 |
| WiCompass | 26.91 | 13.98 | 1,321 |
| Wave2Body | 16.91 | 0.66 | 13,006 |
| MIRaSeg-Pose | 11.64 | 3.63 | 8,988 |
| mmTherm | 1.37 | 0.33 | 21,397 |
Table 5. Model cost and throughput.
Conclusion
We presented mmTherm, a geometry-guided framework for absolute 3D HPE from mmWave radar and a low-resolution thermal array. It uses thermal direction and body structure to guide the interpretation of radar-derived position and range evidence. Root localization establishes a spatial reference for joint-level reconstruction and spatiotemporal refinement. Experiments on OctoNet demonstrate improved absolute pose accuracy and root localization on unseen users, while maintaining an advantage under clutter and sparse radar observations with fewer parameters and lower inference cost.
Acknowledgments
We gratefully acknowledge NVIDIA Corporation for providing the RTX PRO 6000 Blackwell Max-Q GPU through the NVIDIA Academic Grant Program.
References
- [1]Fadel Adib, Chen-Yu Hsu, Hongzi Mao, Dina Katabi, and Frédo Durand. Capturing the human figure through a wall. ACM Transactions on Graphics, 34(6):1–13, 2015.DOI
- [2]Tatsuki Arai, Mariko Isogawa, Kuniharu Sakurada, and Maki Sugimoto. 3d human pose estimation using ultra-low resolution thermal images. In SIGGRAPH Asia 2024 Posters, 2024.
- [3]Kaiwen Cai, Qiyue Xia, Peize Li, John Stankovic, and Chris Xiaoxuan Lu. Robust human detection under visual degradation via thermal and mmwave radar fusion. In Proceedings of the International Conference on Embedded Wireless Systems and Networks (EWSN), 2023.arxiv.org/abs/2307.03623
- [4]Haoran Cao, Jiadi Yu, Hao Kong, Yi-Chao Chen, Yanmin Zhu, Linghe Kong, and Feilong Tang. OccMesh: Occlusion-aware Multi-user 3D Human Mesh Reconstruction Using mmWave Signals. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 9(3):1–22, 2025.DOI
- [5]Anjun Chen, Xiangyu Wang, Shaohao Zhu, Yanxu Li, Jiming Chen, and Qi Ye. mmBody benchmark: 3d body reconstruction dataset and analysis for millimeter wave radar. In Proceedings of the 30th ACM International Conference on Multimedia, pages 3501–3510, 2022.arxiv.org/abs/2209.05070
- [6]Weiyan Chen, Hongliu Yang, Xiaoyang Bi, Rong Zheng, Fusang Zhang, Peng Bao, Zhaoxin Chang, Xujun Ma, and Daqing Zhang. Environment-aware multi-person tracking in indoor environments with mmwave radars. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 7(3):1–29, 2023.DOI
- [7]Jaeho Choi, Soheil Hor, Shubo Yang, and Amin Arbabian. MVDoppler-Pose: Multi-modal multi-view mmwave sensing for long-distance self-occluded human walking pose estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025.DOI
- [8]Junqiao Fan, Jianfei Yang, Yuecong Xu, and Lihua Xie. Diffusion model is a good pose estimator from 3d RF-Vision. In Computer Vision – ECCV 2024, pages 1–18. Springer, 2025.arxiv.org/abs/2403.16198
- [9]Junqiao Fan, Yunjiao Zhou, Yizhuo Yang, Xinyuan Cui, Jiarui Zhang, Lihua Xie, Jianfei Yang, Chris Xiaoxuan Lu, and Fangqiang Ding. M4Human: A large-scale multimodal mmWave radar benchmark for human mesh reconstruction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 42836–42846, 2026.arxiv.org/abs/2512.12378
- [10]Yuan-Hao Ho, Jen-Hao Cheng, Sheng Yao Kuan, Zhongyu Jiang, Wenhao Chai, Hsiang-Wei Huang, Chih-Lung Lin, and Jenq-Neng Hwang. RT-Pose: A 4d radar tensor-based 3d human pose estimation and localization benchmark. In Computer Vision – ECCV 2024, pages 107–125. Springer, 2025.arxiv.org/abs/2407.13930
- [11]Jihye Kim, Jaehoon Jung, Sohee Lim, and Seong-Cheol Kim. Multipath suppression using 2-d angle analysis based on the mimo fmcw radar. IEEE Transactions on Instrumentation and Measurement, 74:1–12, 2024.DOI
- [12]Hao Kong, Xiangyu Xu, Jiadi Yu, Qilin Chen, Chenguang Ma, Yingying Chen, Yi-Chao Chen, and Linghe Kong. m3track: mmwave-based multi-user 3d posture tracking. In Proceedings of the 20th Annual International Conference on Mobile Systems, Applications and Services, pages 491–503, 2022.
- [13]Kohei Kurihara, Tianren Wang, Teng Zhang, and Brian C. Lovell. Boundary guided image translation for pose estimation from ultra-low resolution thermal sensor. In 2020 25th International Conference on Pattern Recognition (ICPR), pages 9–15, 2021.DOI
- [14]Han Li, Yukai Ma, Yuehao Huang, Yaqing Gu, Weihua Xu, Yong Liu, and Xingxing Zuo. RIDERS: Radar-infrared depth estimation for robust sensing. IEEE Transactions on Intelligent Transportation Systems, 25(11):18764–18778, 2024.arxiv.org/abs/2402.02067
- [15]Bo Liang, Chen Gong, Wei Gao, and Chenren Xu. Wave2Body: Rethinking mmwave human pose estimation as radar-to-body token translation. arXiv preprint arXiv:2607.18875, 2026.arxiv.org/abs/2607.18875
- [16]Bo Liang, Chen Gong, Haobo Wang, Qirui Liu, Rungui Zhou, Fengzhi Shao, Yubo Wang, Wei Gao, Kaichen Zhou, Guolong Cui, and Chenren Xu. Wicompass: Oracle-driven data scaling for mmwave human pose estimation. In Proceedings of the 32nd Annual International Conference on Mobile Computing and Networking, page 194–210, New York, NY, USA, 2026. Association for Computing Machinery.arxiv.org/abs/2602.18726
- [17]Zhuoxuan Peng, Boan Zhu, Xingjian Zhang, Wenying Li, and S-H Gary Chan. ExPan: Expanding mmwave datasets for human pose estimation with unlabeled data and lidar datasets. arXiv preprint arXiv:2603.14507, 2026.arxiv.org/abs/2603.14507
- [18]Yili Ren, Zi Wang, Yichao Wang, Sheng Tan, Yingying Chen, and Jie Yang. GoPose: 3D Human Pose Estimation Using WiFi. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 6(2):1–25, 2022.DOI
- [19]Mikko Rinta-Homi, Naser Hossein Motlagh, Agustin Zuniga, Huber Flores, and Petteri Nurmi. How low can you go? performance trade-offs in low-resolution thermal sensors for occupancy detection: A systematic evaluation. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 5(3), 2021.
- [20]Arindam Sengupta, Feng Jin, Renyuan Zhang, and Siyang Cao. mm-Pose: Real-time human skeletal posture estimation using mmWave radars and CNNs. IEEE Sensors Journal, 20(17):10032–10044, 2020.DOI
- [21]Chang Su, Beihong Jin, Qiwen Shi, and Zhi Wang. mmWave-Flow: Unified enhancement and generation of mmWave human point clouds. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 31366–31376, 2026.
- [22]Xiangjie Tang, Ruili Shi, Shuai Wang, Zeyu Zhang, Bin Wang, Zhiqiang Li, and Shuai Wang. MIRaSeg: Exploring mmwave radar and low resolution infrared sensor fusion for robust human semantic segmentation. In Pattern Recognition and Computer Vision – PRCV 2025, pages 17–32. Springer, 2026.DOI
- [23]Shuai Wang, Dongjiang Cao, Ruofeng Liu, Wenchao Jiang, Tianshun Yao, and Chris Xiaoxuan Lu. Human Parsing with Joint Learning for Dynamic mmWave Radar Point Cloud. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 7(1):1–22, 2023.DOI
- [24]Hongfei Xue, Yan Ju, Chenglin Miao, Yijiang Wang, Shiyang Wang, Aidong Zhang, and Lu Su. mmMesh: Towards 3d real-time dynamic human mesh construction using millimeter-wave. In Proceedings of the 19th Annual International Conference on Mobile Systems, Applications, and Services, pages 269–282, 2021.DOI
- [25]Hongfei Xue, Qiming Cao, Yan Ju, Haochen Hu, Haoyu Wang, Aidong Zhang, and Lu Su. M⁴esh: mmWave-Based 3D Human Mesh Construction for Multiple Subjects. In Proceedings of the 20th ACM Conference on Embedded Networked Sensor Systems, pages 391–406, Boston Massachusetts, 2022. ACM.DOI
- [26]Hongliu Yang, Duo Zhang, Xusheng Zhang, Jie Xiong, Zizhou Fan, Wanru Ning, Weiyan Chen, Fुसang Zhang, Zijun Han, and Daqing Zhang. From Spatial Domain to Temporal Domain: Unleashing the Capability of CFAR for mmWave Point Cloud Generation. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 9(2): 1–29, 2025.DOI
- [27]Hongliu Yang, Zizhou Fan, Yueyang Wang, Jie Xiong, Duo Zhang, Xusheng Zhang, Junzhe Wang, Fusang Zhang, and Daqing Zhang. Bridging the Resolution Gap: Cost-Effective Human Point Cloud Generation via Low-Bandwidth mmWave Radar. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 10(2):1–26, 2026.DOI
- [28]Hongliu Yang, Zizhou Fan, Jie Xiong, Duo Zhang, Xusheng Zhang, Zijun Han, Fusang Zhang, and Daqing Zhang. Om-niPC: A Generalizable Point Cloud Generation Pipeline for mmWave Radar. In Proceedings of the 2026 ACM/IEEE International Conference on Embedded Artificial Intelligence and Sensing Systems, pages 291–304, Saint Malo France, 2026. ACM.DOI
- [29]Jianfei Yang, He Huang, Yunjiao Zhou, Xinyan Chen, Yuecong Xu, Shenghai Yuan, Han Zou, Chris Xiaoxuan Lu, and Lihua Xie. MM-Fi: Multi-modal non-intrusive 4d human dataset for versatile wireless sensing. In Advances in Neural Information Processing Systems, 2023.arxiv.org/abs/2305.10345
- [30]Jiarui Yang, Songpengcheng Xia, Zengyuan Lai, Lan Sun, Qi Wu, Wenxian Yu, and Ling Pei. mmwave point cloud density enhancement for accurate human body reconstruction. In 2025 IEEE International Conference on Robotics and Automation (ICRA), pages 11227–11233. IEEE, 2025.arxiv.org/abs/2503.02375
- [31]Dongsheng Yuan, Xie Zhang, Weiying Hou, Sheng Lyu, Yuemin Yu, Luca Jiang-Tao Yu, Chengxiao Li, and Chenshu Wu. OctoNet: A large-scale multi-modal dataset for human activity understanding grounded in motion-captured 3d pose labels. In Advances in Neural Information Processing Systems, 2025.DOI
- [32]Duo Zhang, Zhehui Yin, Xusheng Zhang, Junzhe Wang, Hongliu Yang, Zhiyun Yao, Zizhou Fan, Wenwei Li, and Daqing Zhang. MuPose: Breaking the Scalability Barrier of mmWave Multi-User Pose Estimation in the Wild. In Proceedings of the 24th Annual International Conference on Mobile Systems, Applications and Services, pages 622–636, University of Cambridge Cambridge United Kingdom, 2026. ACM.DOI
- [33]Duo Zhang, Zhehui Yin, Xusheng Zhang, Hongliu Yang, Junzhe Wang, Zhaoxin Chang, Fusang Zhang, and Daqing Zhang. PathClean: Breaking the Multipath Barrier for High-Fidelity mmWave Human Point Clouds. In Proceedings of the 32nd Annual International Conference on Mobile Computing and Networking, pages 1139–1155, Austin TX USA, 2026. ACM.DOI
- [34]Xie Zhang and Chenshu Wu. TADAR: Thermal array-based detection and ranging for privacy-preserving human sensing. In Proceedings of the Twenty-fifth International Symposium on Theory, Algorithmic Foundations, and Protocol Design for Mobile Networks and Mobile Computing, 2024.arxiv.org/abs/2409.17742
- [35]Xie Zhang, Chengxiao Li, and Chenshu Wu. TAPOR: 3d hand pose reconstruction with fully passive thermal sensing for around-device interactions. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 9(2), 2025.arxiv.org/abs/2501.17585
- [36]Mingmin Zhao, Tianhong Li, Mohammad Abu Alsheikh, Yonglong Tian, Hang Zhao, Antonio Torralba, and Dina Katabi. Through-wall human pose estimation using radio signals. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 7356–7365, 2018.DOI
- [37]Wei Zhao, Tingting Wang, Ao Tan, and Congcong Ren. Night-time pedestrian detection based on a fusion of visual information and millimeter-wave radar. IEEE Access, 11:68439–68451, 2023.



