Mimetic Embodiment Transfer of Human-Humanoid Interaction from In-the-Wild Videos

Introduction

Humans are adept at participating in the social world not only through explicit instruction but also by observing and responding to others’ behaviors [1]. Such a capacity is commonly referred to as mimesis: the representation and communication of experience through interaction in both semantic and dynamic manners [2], [3]. Indeed, interaction cannot be acquired by imitating isolated motions alone. Meaningful human-to-human interactions fundamentally arise from the evolving relationship between participants. For example, making a humanoid dance with another human requires not merely reproducing the learned whole-body motions, but also being aware of interpersonal spacing and orientation, synchronizing their rhythm and movement phase, coordinating contact, and continually adapting to the partner’s motion despite variations in embodiment. Herein, mimesis offers a natural account of how observed human behavior can translate into actions for humanoid robots, connecting motion recognition, motion generation, and embodied communication, thus opening up a vast range of applications in human-humanoid interactions.

In practice, existing humanoid motion datasets [10], [13], [14] are predominantly collected through high-fidelity tele-operation or motion-capture systems in controlled environments. Although these pipelines provide accurate, robot-compatible motions, they are often tied to specific embodiments and capture only a subset of interaction behaviors within limited environments. Advances in scene-aware and occlusion-robust human pose estimation [15], [16], mesh-based body fitting [17], and learning-based monocular reconstruction [18] have made it possible to recover whole-body human motion from videos. Despite promising results, they often overlook human-human interactions due to the scarcity of suitable datasets. Publicly available videos, by contrast, contain abundant and diverse human-human interactions captured in real-world settings, offering a scalable source for deriving human-humanoid interaction motions.

Recent advances in whole-body motion learning [19, 20, 21] have expanded the range of human-like behaviors that humanoids can synthesize. Still, learning behaviors such as human-to-human interactions remain fundamentally constrained by the availability and diversity of training data: a humanoid must not only reproduce an individual motion but also coordinate with its human partner [22]. Meanwhile, retargeting methods offer a means of converting observed human behavior into embodiment-specific compatibility, yet existing approaches [23, 24, 25] primarily optimize single-agent motion, neither preserving the inter-body relationships underlying the interaction nor identifying which semantic cues to optimize across embodiments. Therefore, the central challenge of mimetic embodiment transfer is to translate one participant’s motion into a humanoid embodiment while preserving the interaction semantics with its human partner.

To tackle this challenge, we propose HumiTransfer, a new mimetic embodiment transfer framework that translates in-the-wild human-human interactions into human-humanoid interactions, as shown in Figure 1. HumiTransfer takes various interaction sequences as video frames to extract human motion, then transfers one human participant’s poses to two distinct humanoid embodiments: the 23-DoF High-Torque Mini-hi and the 26-DoF Unitree R1 Edu, resulting in our HumiMimic dataset. We also benchmark existing retargeting methods to showcase the diversity and complexity of interactions in HumiMimic. In summary, our main contributions are twofold as follows:

• We propose HumiTransfer, a mimetic embodiment transfer framework that translates one human into a humanoid while preserving the semantics and distances underlying the interaction and satisfying embodiment-specific kinematic and physical constraints.

• We introduce HumiMimic, a scalable embodied dataset that contains 1,200 human-humanoid interaction sequences (∼500,000 frames) by extracting body frames from in-the-wild human-human interaction videos. Our code and dataset will be made publicly available.

Related Work

Human Motion Datasets. Human motion datasets have traditionally relied on motion capture systems in controlled environments [26, 27]. Human3.6M [5] and CMU Mo-Cap [28] provide accurate 3D poses, while AMASS [4] unifies diverse MoCap sources in the SMPL representation [29]. Multi-person interaction datasets, such as CHI3D [30] and Harmony4D [8], introduce physical interactions, while 3DPW [7] and KIT Whole-Body [6] incorporate more unconstrained settings. Recent humanoid datasets, H2O [10], AgiBot World [11], CLONED [12], and OmniH2O-6 [13], provide humanoid-compatible motion but are primarily collected through teleoperation or controlled data acquisition. In general, existing datasets lack the joint provision of large-scale, in-the-wild multi-person interactions and humanoid-compatible motion. To fill this gap, this work introduces HumiMimic, a dataset of human-humanoid interactions featuring high-fidelity, physically plausible poses/motions.

Human-to-Humanoid Retargeting. Human-to-humanoid retargeting ranges from inverse kinematics and optimization-based methods [23, 31] to learning-based approaches [32, 33]. Optimization-based approaches primarily preserve the motion of end-effector through joint or end-effector alignment [10, 34, 35]. Uniform transfer [25] preserves relative geometry through scaling, yet it is sensitive to embodiment differences. Proportional transfer [10, 23] adapts motion to the target embodiment but introduces distortions due to morphological dissimilarities. Under the umbrella of interactive robotics, these methods fall short of determining the relational cues critical to interaction, making it difficult to maintain high-fidelity, embodiment-specific interactions. Meanwhile, physics-based approaches improve the geometric quality of interactions [36], where recent interaction-aware approaches [24, 37] explicitly consider relationships between interacting bodies, while graphics-oriented methods [38, 39] preserve interaction semantics. Within this research scope, HumiTransfer extends optimization-based human-to-humanoid retargeting, enabling embodiment-specific interaction with another human participant in in-the-wild settings.

To recap, Table I summarizes our method with existing human-motion and humanoid datasets, highlighting its unique combination of scale, in-the-wild multi-person interactions, and support for diverse humanoid embodiments.

Dataset	Attribute	No. Subjects and Means of Collection	No. Sequences	Type of Actions	Multi-Person Interaction	Robot/Humanoid Embodiment
AMASS [4]		344, MoCap	11,265	Daily/locomotion	No	No
Human3.6M [5]		11, MoCap	∼3.6M frames	15 daily	No	No
KIT Whole-Body [6]		238, MoCap	2,972	Whole-body everyday	Some	No
3DPW [7]		7, IMU + single-view RGB	60 clips	Walking, interacting	Some	No
Harmony4D [8]		24, multi-view RGB	208	Close interactions	Yes	No
CHICO [9]		20, multi-view RGB	240 clips	7 industrial interactions	No	KUKA
H2O [10]		AMASS	∼10k	Whole-body everyday	No	Unitree H1
AgiBot World [11]		Teleop	∼3000 hours	Dexterous/loco-manip	No	AgiBot G1
CLONED [12]		AMASS + MoCap	345	Whole-body loco-manip	Some	Unitree H1
OmniH2O-6 [13]		Teleop	∼72k frames	6 everyday tasks	Some	Unitree H1
HumiMimic (Ours)		Multiple (in-the-wild)	500K frames	Diverse (sports, daily, social)	Yes	Unitree R1 Edu & HighTorque Mini-hi

HumiTransfer: Mimetic Embodiment Transfer for Human-Humanoid Interaction

As shown in Figure 2, we introduce a three-stage framework: ($i$) a framework for human–humanoid interaction with reconstructed 3D interaction motions; ($ii$) HumiTransfer, a mimetic embodiment transfer for interaction-preserving human-to-humanoid motion retargeting; and ($iii$) a post-retargeting optimization stage. After which, we obtain HumiMimic, our human-humanoid interaction dataset.

Human-Humanoid Interaction Data

1) Data Collection & Statistics: We gather high-quality videos of in-the-wild human-human interactions from YouTube, TikTok, Facebook, and other Internet sources, involving only two participant subjects. As depicted in Figure 3, our resulting dataset contains 1,200 two-person interaction videos, yielding about 500,000 processed frames (approximately 4.6 video hours). Each sequence ranges from 100 to 1,000 frames, with more than half exceeding 300 frames. Regarding diversity, our dataset covers a broad spectrum of interactions, such as co-movement (29%), social exchanges (21%), dual sports (17%), and sparring (13%).

2) Extraction of Human-Human Interaction Motion: As we noticed image-quality distortions, camera instability, and scene complexity in the collected in-the-wild videos, we perform an automatic filtering step [40] to remove low-fidelity frames and re-encapsulate high-fidelity ones, retaining approximately 73% of all frames. Next, in each re-encapsulated interaction sequence, we employ a multi-person tracker [41] to detect humans in each frame using bounding boxes. For each bounding box, a pre-trained keypoint detector [42] is used to detect the human’s key positions within the bounding box, which are subsequently used to perform SMPL reconstruction [43]. Thus, to ensure the physical plausibility of extracted poses, we optimize two complementary objectives in each video: (a) whole-body shape-fitting optimization and (b) foot-ground contact consistency fine-tuning.

a) Whole-Body Shape Fitting: The pose parameter ($\alpha$) and the body shape parameter ($\beta$) of the SMPL model [43] $\mathcal{H}_{\mathrm{SMPL}}$ are obtained by minimizing:

where $E_j$ represents the re-projection error between the detected and projected body joints, $E_{\alpha}$ and $E_{\beta}$ are pose and shape prior errors [43]; each is weighted with $\lambda_{\alpha}$ and $\lambda_{\beta}$.

b) Foot-Ground Contact Consistency Fine-tuning: To further enhance the quality of the reconstructed human model (Eq. (1)), we minimize the contact error between the fitted SMPL and the ground, computed as the foot consistency error and a Zero Moment Point (ZMP) regularizer [44]:

with the foot consistency error $E_{\mathrm{consist}}$ formulated as:

where $\mathcal{F} \subset\mathcal{H}_{\mathrm{SMPL}}$ denotes the set of foot joints of the SMPL model, $I_{i,t}$ denotes the ground-contact indicator function for foot joint $i$, $J_{i,t}$ denotes the position of joint $i$ derived from its forward kinematics at time $t$, $\mathbf{f}$ is the reference foot position, and $\mathbf{n}^{*}$ is the estimated normal of the planar ground. Meanwhile, the second term in Eq. (2) enforces the mesh-based human body’s balance while maintaining consistent motion with the ground.

By applying Eq. (1) and Eq. (2) to all detected humans, we obtain sets of SMPL models that serve as the frames $\{\mathcal{H}_{\mathrm{SMPL},t}\}_{t=0}^{T}$, forming our dataset for retargeting a humanoid’s motions interacting with a human partner.

Interaction Modeling & Mimetic Embodiment Transfer

Given the extracted sequence of human body frames $\{\mathcal{H}_{\mathrm{SMPL},t}\}_{t=0}^{T}$ and a humanoid embodiment $\mathcal{R}$, our goal is to transfer this sequence into a humanoid’s motion while preserving interaction semantics and geometric relationships.

1. Humanoid Poses Warm-Starting: As a warm start, we initialize humanoid poses $\{\mathcal{R}_{t}\}_{t=0}^{T}$ by retargeting the human body frames $\{\mathcal{H}_{\mathrm{SMPL},t}\}_{t=0}^{T}$ through inverse kinematics, denoted as $E_{\mathrm{IK}}$, similar to [23], [37]. Specifically, at each discrete timestep $t$, the human pose $\mathcal{H}_{\mathrm{SMPL},t}$ is scaled according to the humanoid’s proportion. The humanoid embodiment $\mathcal{R}$ is then initialized in its default state; thus, its joint poses are optimized to match the scaled target frames while respecting the kinematic constraints and joint limits.

2. Modeling of Human-Humanoid Interaction: To enable mimetic embodiment transfer for human-humanoid interaction, we model these interactions as two matrices: the joint-relative direction matrix and the anatomical offset matrix between the interacting subjects’ joints.

a) Joint-Relative Direction Matrix: Let $J^{\mathcal{H}}_{i,t}$, $J^{\mathcal{R}}_{i,t}$, $J^{\mathcal{P}}_{i,t}$ be the $i$-th joint in the human body frame $\mathcal{H}_{\mathrm{SMPL}}$, the humanoid embodiment $\mathcal{R}$, and the interacting human partner $\mathcal{P}$ at the discrete timestep $t$. We, therefore, introduce the joint-relative directions of the humanoid $\mathcal{R}$ and its human partner $\mathcal{P}$, written as an $n \times n$ square matrix:

where $\mathcal{R} \to\mathcal{P}$ reads as the relation of $\mathcal{R}$ to $\mathcal{P}$, and each entry in $\mathbf{J}^{\mathcal{R}\to\mathcal{P}}_{t}$ is computed as $\mathbf{r}^{\mathcal{R}\to\mathcal{P}}_{ij,t} = \mathbf{J}^{\mathcal{R}}_{i,t} - \mathbf{J}^{\mathcal{P}}_{j,t}$. Eq. (4) encodes the joint-based directional offset vectors between two interacting subjects. Similarly, $\mathbf{J}^{\mathcal{H}\to\mathcal{P}}_{t}$ can be formulated as the joint-relative direction matrix between the human body frame $\mathcal{H}_{\mathrm{SMPL}}$ and the human partner $\mathcal{P}$.

b) Anatomical Offset Matrix: To reinforce the naturalness of the interaction, we also incorporate the matrix encoding anatomical offsets among the subjects $\mathcal{R}$ and $\mathcal{P}$ at each discrete timestep $t$. Its $n \times n$ square matrix is:

where $\mathbf{a}^{\mathcal{R}\to\mathcal{P}}_{ij,t} = (\mathbf{u}^{\mathcal{R}}_{i,t})^{\mathsf{T}}(\mathbf{J}^{\mathcal{R}}_{i,t}-\mathbf{J}^{\mathcal{P}}_{j,t})$, with $\mathbf{u}^{\mathcal{R}}_{i,t}$ is the unit vector from the $i$-th joint to its immediate parent joint. Eq. (5) represents the link-based (anatomical) offsets with respect to the human partner $\mathcal{P}$ while being aware of itself $\mathcal{R}$.

c) Joint Importance Weighting: As certain parts of the body are more involved in the interactions, we propose joint importance weighting to prioritize them when retargeting.

We associate each joint $J^{\mathcal{H}}_{i}$ in the human $\mathcal{H}$ with a scalar weight using a softmax-normalized function [45] $w^{\mathcal{H}}_{\mathrm{sem},i} = \exp(\alpha_i)/\sum_{j=1}^{n}\exp(\alpha_j)$, where the cosine similarity $\alpha_i = (\mathbf{z}_{\mathrm{poses}} \cdot\mathbf{z}_{\mathrm{joint}})/(\lVert\mathbf{z}_{\mathrm{poses}}\rVert\lVert\mathbf{z}_{\mathrm{joint}}\rVert)$ with $\mathbf{z}_{\mathrm{poses}}$ and $\mathbf{z}_{\mathrm{joint}}$ respectively represent the embedding vectors of the poses throughout the interaction horizon and of the specific joint; both of which are obtained using MotionCLIP [46]. With the weight $w_{\mathrm{sem}}$ along the interaction horizon, the pairwise weight between $J^{\mathcal{H}}_{i}$ and $J^{\mathcal{P}}_{j}$ applied to all poses is $w^{\mathcal{H}\to\mathcal{P}}_{\mathrm{sem},ij} = w^{\mathcal{H}}_{\mathrm{sem},i}w^{\mathcal{P}}_{\mathrm{sem},j}$, which forms the $n \times n$ weighting matrix:

d) Joint-Relative Distance Weighting: We additionally weigh the joint-relative distance between $J^{\mathcal{H}}_{i,t}$ and $J^{\mathcal{P}}_{j,t}$ at each discrete time $t$, modeled as a modified normal distribution: $w^{\mathcal{H}\to\mathcal{P}}_{\mathrm{dis},ij,t} = \exp\left(-\lVert r^{\mathcal{H}\to\mathcal{P}}_{ij,t}\rVert^{2}/2\sigma^{2}\right)$, where $\sigma^{2}$ denotes the distribution variance. Note that, since the joint-relative distances are non-negative while the normal distribution is infinitely two-sided, joints that are closer to each other are assigned a larger weight and vice versa. Similar to Eq. (6), the time-dependent distance-weighted matrix is:

3. Mimetic Embodiment Transfer Objective: At each discrete timestep $t$, given $J^{\mathcal{H}\to\mathcal{P}}_{t}$, $A^{\mathcal{H}\to\mathcal{P}}_{t}$, $W^{\mathcal{H}\to\mathcal{P}}_{\mathrm{dis},t}$, and $W^{\mathcal{H}\to\mathcal{P}}_{\mathrm{sem}}$, we formulate the mimetic embodiment transfer term, $E_{\mathrm{interact}}(\mathcal{R}_{t}, \mathcal{P}_{t}, \mathcal{H}_{\mathrm{SMPL},t})$, to align human-humanoid interaction with its human-human interaction as:

where $\odot$ denotes element-wise multiplication. By incorporating Eq. (8) with an enforced inverse kinematic term $E_{\mathrm{IK}}$ and the Tikhonov regularization term $E_{\mathrm{smooth}}$ [47], we therefore use the mink solver [48] to solve for $\hat{\mathcal{R}}_{t}$ in Eq. (9) at each discrete timestep $t$ along the interaction horizon of $T$:

to obtain $\{\hat{\mathcal{R}}_{t}\}_{t=0}^{T}$, preserving the interaction between the target human $\{\mathcal{H}_{\mathrm{SMPL},t}\}_{t=0}^{T}$ and its human partner $\{\mathcal{P}_{t}\}_{t=0}^{T}$ along the interaction horizon.

Post-Retargeting Interaction Optimization

Yet, these newly retargeted poses face challenges in self-balancing and inter-body collisions. We mitigate these artifacts by optimizing post-retargeting interactions as we roll out their poses. For self-balancing, we follow the full-body compliant human-humanoid interaction mechanism [49]. To prevent inter-body collisions, we maintain an inter-body minimum safe distance of $d_{\min,ij}$ between $J^{R}_{i,t}$ and $J^{P}_{j,t}$ at each given discrete timestep $t$. We apply this post-retargeting optimization throughout the human-humanoid interaction horizon for all human-humanoid interaction sequences.

Lastly, we apply the entire HumiTransfer framework to all human-human interaction sequences, yielding the final HumiMimic, which is used in our experiments.

Experiment

We conduct intensive experiments to: (i) evaluate the effectiveness of HumiTransfer; (ii) demonstrate the utility of our HumiMimic dataset for a generative human-humanoid interaction task; and (iii) validate real-world deployment by using a trained policy on our HumiMimic dataset and executing it on a real humanoid robot.

Assessment of Extraction & Retargeting in HumiTransfer

First, we assess the performance of our HumiTransfer in the human-to-humanoid motion transfer task. As no training is involved, all experiments in this section run on our full HumiMimic dataset.

Physical Plausibility of Extracted Human-Human Interaction

We first evaluate the physical plausibility of extracted human-human interactions (Eq. (3)) by computing their foot-slipping errors ($E_{\mathrm{foot}}$) [50], ground penetration ($E_{\mathrm{ground}}$) [51], and static stability margin (SSM) [52]. As in [53], we compute SSM as the signed distance between the projected center of mass (CoM) and the support polygon.

As depicted in Table II, our method achieves lower error across the defined metrics compared to the traditional SMPL-based fitting [43], notably with only $E_{\mathrm{foot}}$ of 0.6765 cm/frame, $E_{\mathrm{ground}}$ of 0.9282 cm/frame, and SSM of 2.0402 cm/frame lying inside the polygon, highlighting the importance of foot-ground consistency fine-tuning.

Method	$E_{\mathrm{foot}}$ (↓)	$E_{\mathrm{ground}}$ (↓)	SSM (↑)
SMPL Fitting [43]	2.6125	36.3547	-1.9336
SMPL Fitting [43] with Foot-Ground Consistency Fine-tuning	0.6765	0.9282	2.0402

Mimetic Embodiment Transfer

We evaluate the performance of our mimetic embodiment transfer against naive methods (i.e., inverse-kinematics [25] with uniform scaling) and optimization-based fitting methods (i.e., GMR [23] and PAIR [37]) on the 23-DoF HighTorque Mini-hi and the 26-DoF Unitree R1 Edu. In this experiment, we report MPJPE in cm as the joint position error, IRD cm as the mean absolute pairwise-distance error, CF1 as the F1 score for contact preservation at a distance threshold of $d = 20$ cm.

In Table III, our method with semantic and distance-based weighting (HumiTransfer$^{+}$) achieves the best MPJPE, IRD, and CF1 results across both the Unitree R1 Edu and the HighTorque Mini-hi. Meanwhile, GMR [23], and PAIR [37], and our method without a weighting mechanism (HumiTransfer$^{-}$) result in slight inconsistencies. The qualitative results in Figure 4 confirm the overall effectiveness of our weighting strategy for preserving high-fidelity, embodiment-specific human-to-humanoid interaction retargeting.

Method	Unitree R1 Edu MPJPE (↓)	Unitree R1 Edu IRD (↓)	Unitree R1 Edu CF1 (↑)	HighTorque Mini-hi MPJPE (↓)	HighTorque Mini-hi IRD (↓)	HighTorque Mini-hi CF1 (↑)
Naive IK [25]	37.85	16.82	13.84	41.31	20.67	11.42
GMR [23]	11.91	5.73	69.12	10.49	5.30	61.97
PAIR [37]	9.37	4.16	74.54	12.79	6.24	59.41
HumiTransfer$^{-}$	10.13	4.08	80.90	11.63	6.14	65.29
HumiTransfer$^{+}$	6.31	2.84	81.97	7.58	3.67	78.15

Diversity of Retargeted Human-Humanoid Motion

We also measure the motion diversity via the diversity score proposed in [54]. As shown in Table IV, our dataset achieves higher diversity scores of 6.2427 and 6.2285 for Unitree R1 Edu and HighTorque Mini-hi, respectively, compared to AMASS [4]’s diversity scores of 5.4834 and 5.4859. These results indicate greater overall motion variability, thanks to the collection of in-the-wild human-human interaction videos. The diversity of interactions can be seen in our video. In addition, we show in Figure 5 the eight most representative human-humanoid interaction motions in HumiMimic.

Method	Unitree R1 Edu		HighTorque Mini-hi
	Diversity (↑)		Diversity (↑)
Human3.6M [5]	3.8251		3.7963
AMASS [4]	5.4834		5.4859
3DPW [7]	4.0173		3.9130
HumiMimic	6.2427		6.2285

Generative Human-Humanoid Interaction Benchmarking

Based on the concept of mimesis, we verify whether our HumiMimic dataset can serve as a benchmark for generative human-humanoid interaction tasks. In this experiment, we split our HumiMimic dataset into train and test sets in an 8 : 2 ratio to train and test existing baselines. Given the human’s motion sequence, the generative human-humanoid interaction task aims to synthesize a responsive 3D motion for the humanoid to synchronize with the partner’s motion while maintaining physical and semantic coordination, including handling occlusions, rotations, and physical contacts.

We evaluate on five architectures: ReMos [55], InterFormer [56], ComMDM [57], InterGen [58], and Duolando [59], adapted to predict follower motion conditioned on partner’s 3D poses. For Duolando, RL fine-tuning further reduces artifacts such as foot slipping and penetration. Following [59], we evaluate the motion kinematic quality through the kinematic Fréchet Inception Distance [60] ($\mathrm{FID}_{k}$) from joint trajectories and interaction quality through cross-distance FID ($\mathrm{FID}_{cd}$) between pairwise joints and frequency of contact consistency ($f_{\mathrm{contact}}$) in terms of percentages of contact frames versus non-contact frames.

Table V shows that Duolando achieves the best overall performance, with $\mathrm{FID}_{k}$ at 24.96 and 27.24, $\mathrm{FID}_{cd}$ at 9.07 and 11.53, together with the $f_{\mathrm{contact}}$ of 51.75% and 51.77%. ReMos [55] exhibits improved contact frequency at 46.79% and 47.56%. Meanwhile, InterGen [58] achieves an $\mathrm{FID}_{cd}$ of 10.86 under HighTorque Mini-hi embodiment. Figure 6 presents a challenging sports sequence with a human partner. Duolando [59] produces more appropriate and synchronized responses, while InterGen [58] generates coherent motion but often responds inappropriately, either closely mimicking the human partner or producing unnatural motions.

Method	Unitree R1 Edu			HighTorque Mini-hi
	$\mathrm{FID}_{k}$ (↓)	$\mathrm{FID}_{cd}$ (↓)	$f_{\mathrm{contact}}$ (↑)	$\mathrm{FID}_{k}$ (↓)	$\mathrm{FID}_{cd}$ (↓)	$f_{\mathrm{contact}}$ (↑)
ComMDM [57]	36.07	15.93	31.09%	38.33	18.30	29.07%
InterFormer [56]	30.16	14.66	42.97%	32.84	13.98	39.47%
InterGen [58]	34.62	17.72	48.91%	30.98	10.86	39.37%
ReMos [55]	25.49	11.66	46.79%	31.51	12.94	47.56%
Duolando [59]	24.96	9.07	51.75%	27.24	11.53	46.77%

Self-Balancing Evaluation & Real-Robot Deployment

Finally, we examine the self-balancing capability of the Unitree R1 Edu and the HighTorque Mini-hi in simulation and physically demonstrate the interaction tasks on the real Unitree R1 Edu humanoid with a human partner.

1. Self-Balancing Evaluation: We evaluate the balancing ability of the retargeted motions for both humanoid embodiments, considering their CoM heights relative to the ground’s tangential plane, and we compare these profiles with those of the retargeted human along the interaction horizon.

On the test set of HumiMimic in simulation, both humanoids, Unitree R1 Edu and HighTorque Mini-hi, maintain CoM-height differences of 2.9573 cm and 3.2021 cm, respectively, while preserving interaction-critical relationships with the human partner. Both the Unitree R1 Edu (middle) and HighTorque Mini-hi (bottom) in Figure 7 are capable of reproducing the crouching motions while maintaining the retargeted human’s CoM trajectories and remaining consistent with their respective morphologies, as shown in Figure 7 (top). These results indicate that HumiTransfer can adapt human poses to each humanoid while maintaining self-balancing capabilities, which are well-suited for real-robot deployment.

Real-Robot Deployment

To evaluate real-world executability, we deploy the retargeted motions on the 26-DoF Unitree R1 Edu. We use the sequences from HumiMimic to optimize and map to the robot’s joints, then check against the robot’s motion constraints before execution. In particular, we employ the Unitree R1 Edu motion-tracking controller in mjlab [61], thus optimizing a 26-action policy in 4096 parallel environments for approximately 30, 000 iterations on an NVIDIA RTX 3090 GPU.

The training stage samples initial motion phases, perturbs the robot’s initial pose and velocity, and randomizes physical parameters, including foot friction and joint dynamics. Following DeepMimic [33] and BeyondMimic [62], we condition the actor on this command and proprioception, rewarding the simulated robot for matching the reference. At each simulated step, a motion command samples joint positions and velocities to form the policy command, while anchor and whole-body states supply the corresponding tracking targets. The trained actor is then exported to ONNX and executed in the C++ mimic controller with ONNX Runtime. At 50 Hz, the controller reads the 26 motor states and IMU from Unitree low-state messages, reconstructs the training observation order, and finally evaluates the policy. For details, the observations contain reference joint positions and velocities, torso orientation error, IMU angular velocity, and the previous motion. The torso orientation is reconstructed from the IMU quaternion and waist joints without global position estimation. A fixed mapping converts the common 26-joint policy order to the sparse Unitree SDK motor IDs used on the robot platform. Figure 8 shows that our framework preserves the characteristic spatial and contact relationships in retargeted human-human interactions. The demonstration video is available in our supplementary materials.

Limitations & Future Work

Nevertheless, HumiTransfer remains limited by substantial morphological differences between humans and humanoids, which can make certain motions kinematically infeasible. For example, in simulation, we found that the HighTorque Mini-hi’s broad shoulders and short arms prevent it from hugging a human partner as the Unitree R1 Edu can. Furthermore, our current real-robot deployment primarily relies on predefined motion sequences and does not yet explicitly account for human motion. An interesting direction for future work is therefore to develop methods that adapt humanoid motion in response to observed human behavior, leveraging motion data from our HumiMimic dataset.

Conclusion

In this paper, we have presented HumiTransfer for mimetic embodiment transfer from in-the-wild human-human interaction videos. Our proposed method includes a data preparation pipeline that extracts physically plausible human-human interaction poses from in-the-wild videos, followed by a mimetic embodiment transfer that translates one of the observed humans into a humanoid. Our HumiMimic dataset demonstrates higher quality in avoiding ground penetration and foot slipping, while HumiTransfer allows human poses to be retargeted to different humanoids. Benchmarking on HumiMimic across motion generation methods and deployments on the Unitree R1 Edu humanoid showed that our work enables physical deployment and can serve as a benchmark for future human-humanoid interaction work.