betaFeedback

WorksComputer Science

A latent control model for realistic rodent motion

Abstract

The development of embodied models of animal behavior is a major goal of the neuro-AI community. Embodied models will enhance our ability to make direct comparisons between biological and artificial agents. To make such comparisons effective, though, we need embodied models that actually move like animals. Previous work has developed virtual animal bodies and techniques for training realistic motion through imitation. However, to date, there is no system for embodied animal modeling that produces natural movements both during trained goal-directed behavior, and during exploration driven by random generation. Here, we introduce Simulated Control via Augmented Motor Priors for Embodied Reinforcement learning (SCAMPER), a system for training an embodied virtual rat model, enabling natural movements during both goal-directed behavior and random exploration. SCAMPER uses a variational architecture with two distinct priors, a motion prior for control and a behavior prior for exploration. We show that combining both of these priors allows SCAMPER to generate realistic behavior during exploration and exploit the task structure effectively. SCAMPER can also learn from sparse rewards as a result of the diverse movements induced by the behavior prior, allowing the system to model realistic sparse-reward tasks from experimental neuroscience. Altogether, SCAMPER provides the neuro-AI community with an effective new system for training embodied models of rodents on realistic tasks.

Introduction

One of the ultimate goals of neuro-AI research is to be able to fully capture the richness of animal learning and behavior with embodied computational models [1, 2]. Previous work has developed realistic virtual bodies for a variety of model organisms commonly used in neuroscience experiments [3]. However, controlling a realistic virtual body to consistently produce natural motion is non-trivial. For example, when an artificial neural network was trained to control a virtual rat on a variety of tasks using deep reinforcement learning (RL), the model could solve the tasks, but often did so with unnatural movements that did not resemble the motion of real rats [4]. While researchers have since used imitation learning to improve the realism of virtual animal motion, these models are either limited to pure imitation [5], or struggle to transfer to downstream RL tasks [6]. Within robotics, physics-based controllers trained with RL to produce realistic humanoid motion can generalize to new tasks [7, 8, 9, 10, 11, 12]. However, most approaches have no mechanism for random generation of realistic motion. One exception is the recent Physics-based Universal motion Latent SpacE (PULSE) framework that uses residual control of a variational motion prior [13]. PULSE can sample from its motion prior to produce random, yet realistic movements. However, this sampling exhibits limited diversity: depending on initialization, the system typically settles into a single behavioral mode and stays there. This is especially problematic when PULSE is used as a low-level controller for sparse-reward tasks, which demand substantial structured exploration [14, 15]. Because most experimental tasks in neuroscience offer only sparse extrinsic reward [16, 17, 18], training embodied animal models to solve them while maintaining naturalistic motion remains an open problem.

Here, we present Simulated Control via Augmented Motor Priors for Embodied Reinforcement learning (SCAMPER), a system for training an embodied virtual rat model, enabling natural movements during both goal-directed behavior and random exploration. SCAMPER builds on the PULSE framework [13], but adds an additional behavior prior that greatly increases the diversity of randomly generated movements while preserving the realism of the motion. We provide a formal analysis demonstrating why the behavior prior increases the diversity of generated movements. We then show empirically that SCAMPER produces a diverse array of realistic behavior during random rollouts, covering the behavioral repertoire present in the training data and beyond. We also demonstrate that SCAMPER transfers effectively to a range of downstream tasks, from dense-reward locomotion and rearing to sparse-reward foraging tasks, all while maintaining naturalistic motion throughout training. Altogether, SCAMPER provides the neuro-AI community with a powerful new tool for embodied modeling of animal behavior.

Related work

Motor priors and latent skill spaces for physics-based control. Training physics-based controllers from scratch for different tasks often yields unnatural motion [19], motivating imitation-based controllers that track reference motion data [7]. Merel et al. [8] extended this approach by distilling a tracking expert into a latent skill space through an information bottleneck, and subsequent work explored adversarial alternatives [9, 10, 12] and variational formulations [20, 21, 22]. PULSE [13] distills a universal motion imitator [23] into a variational model regularized with a learnable motion prior, achieving broad coverage of human motor skills. These frameworks target humanoid characters trained on human motion capture (mocap) data; their applicability to non-human morphologies driven by species-specific behavioral data has not yet been demonstrated. Moreover, the learned motion prior in PULSE supports local motion continuation, but random rollouts tend to remain within the behavioral mode in which they are initialized, producing locally plausible but behaviorally narrow motion. Kinematic motion models, including autoregressive CVAE-based priors [24] and diffusion-based generators [25], can produce diverse long-horizon motion but do not yield a physics-based controller. Furthermore, autoregressive policies have been applied to physics-based control [26, 27], but they provide no mechanism for sampling diverse motion that respects a species-specific behavioral repertoire. Here, we extend the PULSE framework to a biomechanically realistic rodent trained on 3D kinematic recordings of freely behaving rats, and complement the motion prior with a behavior prior that induces realistic transitions across behavioral modes.

Physics-based control of virtual animals. Biomechanically realistic virtual animals have emerged as a tool for neuro-AI. Merel et al. [4] introduced a virtual rodent trained with RL, but without imitation pretraining the resulting motion was unnatural. Aldarondo et al. [5] extended this work by training a controller for the virtual rat to imitate freely moving rats and found that its internal representations matched those in the motor cortex and dorsolateral striatum of the real rats; the controller was, however, purely imitative and could not generate diverse behavior. MIMIC-MJX [6] generalizes this tracking-imitation approach across species via an open-source JAX/MJX pipeline, but supports only limited transfer learning. Parallel efforts in Drosophila [28, 29] and C. elegans [30] further extend this paradigm beyond rodents. In zebrafish, Keller et al. [31] drive naturalistic behavioral transitions via model-based intrinsic motivation against an ethological prior. Across this line of work, no existing model provides the ability to both mimic real motion and support sampling appropriate for exploration. Our work fills this gap, yielding a controller that can be both reused across tasks and sampled for diverse, naturalistic behavior.

Overview of SCAMPER

Preliminaries

We define the proprioception of the agent at a given timestep as

xt=(θt,θ˙t,τt,ht,ϕt,ptee,stkin,ct,at−1),(1)\mathbf{x}_t = (\theta_t,\dot{\theta}_t,\tau_t,h_t,\phi_t,\mathbf{p}^{ee}_t,s^{kin}_t,c_t,a_{t-1}), \tag*{(1)}

containing the joint angles (θt)(\theta_t), angular velocities (θ˙t)(\dot{\theta}_t), joint torques (τt)(\tau_t), body height (ht)(h_t), orientation relative to the world z-axis (ϕt(\phi_t, akin to a vestibular signal), egocentric end-effector positions (ptee)(\mathbf{p}^{ee}_t), kinematic sensors (accelerometer, velocimeter, gyro; stkins^{kin}_t), end-effector touch sensors (ct)(c_t), and the previous action (at−1)(a_{t-1}). During imitation tasks, the goal of the agent at a given timestep is defined as

gt=(rt+1−rt,qt+1⊖qt,θt+1−θt,pt+1seg−ptseg),(2)\mathbf{g}_t = (\mathbf{r}_{t+1}-\mathbf{r}_t,\mathbf{q}_{t+1}\ominus\mathbf{q}_t,\theta_{t+1}-\theta_t,\mathbf{p}^{seg}_{t+1}-\mathbf{p}^{seg}_t), \tag*{(2)}

and specifies the displacement the agent must achieve, computed as differences between its current state and the target state at the next timestep in the root position (rt)(\mathbf{r}_t), root orientation (qt(\mathbf{q}_t; where ⊖\ominus is egocentric quaternion difference), joint angles (θt)(\theta_t), and egocentric Cartesian body segment positions (ptseg)(\mathbf{p}^{seg}_t). Target states are provided by a mocap dataset registered to the biomechanical rat model via inverse kinematics using STAC-MJX [6]. During imitation the policy network is provided the next five goal states (gt:t+4)(\mathbf{g}_{t:t+4}). During transfer learning, the agent no longer receives a goal state, gt\mathbf{g}_t, and instead observes task-specific environmental observations ot\mathbf{o}_t (e.g., scent gradient). In this work we use the terms ‘rat’, ‘rodent’, and ‘agent’ interchangeably.

We frame each task (aside from the distillation objectives) as goal-conditioned RL, formulated as a Markov Decision Process M=⟨S,A,T,R,γ⟩\mathcal{M} = \langle\mathcal{S},\mathcal{A},\mathcal{T},\mathcal{R},\gamma\rangle. The states st∈Ss_t \in\mathcal{S} contain xt\mathbf{x}_t and either gt\mathbf{g}_t or oto_t; all values are normalized by their running mean and variance. Actions at∈Aa_t \in\mathcal{A} are torques applied to the rodent’s actuated joints. T\mathcal{T} and R\mathcal{R} are the transition dynamics (determined by the physics environment) and the reward function, respectively. We train policies with proximal policy optimization (PPO) [32] to maximize the discounted return E[∑t=1Tγt−1rt]\mathbb{E}\left[\sum_{t=1}^{T}\gamma^{t-1}r_t\right].

Training pipeline

SCAMPER uses a four-stage training pipeline (Figure 1). All networks are multi-layer perceptrons (MLPs), but in principle other architectures could be used.

Training stages of SCAMPER

Figure 1. Training stages of SCAMPER. (1) The expert imitator is trained with mocap data. (2) The latent controller is trained via online distillation. (3) The behavior prior is distilled from the frozen encoder. Finally, (4) the model can be trained with RL to perform different tasks.

(1) Expert imitator. We train an expert imitator πI(at∣xt,gt)\pi_I(a_t|x_t, g_t) with PPO to reproduce the mocap trajectories (SuppVid1; App. § 10).

(2) Online distillation. Following the PULSE framework [13], we distill the expert imitator into a latent skill controller. Similar to PULSE, we find that training the latent skill controller using supervised distillation of a pre-trained expert leads to better training stability than training the latent skill controller from scratch using RL, hence we adopt the same approach. The controller consists of three jointly trained networks: an encoder qϕ(zt∣xt,gt)q_\phi(z_t|x_t, g_t), a decoder pθ(at∣xt,zt)p_\theta(a_t|x_t, z_t), and a proprioception-conditioned motion prior pψ(zt∣xt)p_\psi(z_t|x_t) over latent codes. Training proceeds by rolling out the student controller on the imitation task in simulation, while querying the frozen expert, πI\pi_I, for target actions on visited states, following DAgger [33]. The networks are jointly trained by minimizing:

Ldistill=LDAgger+λLKL+αLAR,(3)\mathcal{L}_{\mathrm{distill}} = \mathcal{L}_{\mathrm{DAgger}} + \lambda\mathcal{L}_{\mathrm{KL}} + \alpha\mathcal{L}_{\mathrm{AR}}, \tag*{(3)}

where LDAgger=∥at−a^t∥22\mathcal{L}_{\mathrm{DAgger}} = \lVert a_t - \hat{a}_t \rVert_2^2 matches the expert’s target action ata_t to the decoder’s prediction a^t\hat{a}_t, LKL=DKL(qϕ∥pψ)\mathcal{L}_{\mathrm{KL}} = D_{\mathrm{KL}}(q_\phi\mathbin{\|} p_\psi) regularizes the encoder toward the motion prior, and LAR=∥μte−μt−1e∥22\mathcal{L}_{\mathrm{AR}} = \lVert\mu_t^e - \mu_{t-1}^e \rVert_2^2 encourages temporal smoothness in the latent space. The decoder and motion prior are retained for downstream use; the encoder is kept only for the next training stage. λ\lambda and α\alpha are hyperparameters (Table 3).

GroupHyperparameterValue
Distillation lossAR(1) smoothness weight α\alpha5×10−35 \times10^{-3}
Distillation lossGradient clip threshold (global norm)10.0
Distillation lossEncoder / prior logvar clamp[−5,0][-5, 0]
KL scheduleSchedule typelinear ramp, then hold
KL scheduleStart weight λ0\lambda_01×10−31 \times10^{-3}
KL scheduleEnd weight λ1\lambda_11×10−41 \times10^{-4}
KL scheduleRamp range (fraction of training)[0%,25%][0\%, 25\%]
Hard-negative miningReweight strength αHN\alpha_{\mathrm{HN}}0.1
Hard-negative miningMax sample weight wmaxw_{\mathrm{max}}5.0
Network architectureLatent size16
Network architectureEncoder hidden sizes[2048,1024,1024][2048, 1024, 1024]
Network architectureDecoder hidden sizes[1024,1024,512][1024, 1024, 512]
Network architectureMotion prior hidden sizes[1536,1024,512][1536, 1024, 512]
TrainingParallel environments4,096
TrainingTotal environment steps1×1091 \times10^9
TrainingBatch size1,024
TrainingMinibatches per epoch16
TrainingUpdates per batch4
TrainingLearning rate (Adam)1×10−41 \times10^{-4}
TrainingUnroll length20
TrainingAction repeat1
TrainingTrain / eval clip split0.8/0.2
TrainingEvaluation / reset interval2×1072 \times10^7 steps

Table 3. Hyperparameters used for Stage 2 (online distillation) of the SCAMPER training pipeline. Environment, termination, and domain-randomization settings are inherited from Stage 1 (Table 2).

(3) Behavior prior distillation. After online distillation, we train a second proprioception-conditioned prior pφ(zt∣xt)p_\varphi(z_t|x_t) with all other networks frozen. Rolling out the latent skill controller on the imitation task in simulation, we optimize pφp_\varphi by minimizing

Lbehavior=DKL(qϕ(zt∣xt,gt)∥pφ(zt∣xt)).(4)\mathcal{L}_{\mathrm{behavior}} = D_{\mathrm{KL}}(q_\phi(z_t|x_t, g_t) \mathbin{\|} p_\varphi(z_t|x_t)). \tag*{(4)}

By Proposition 2, this ensures pφp_\varphi captures the full goal-conditional variation in the encoder’s outputs.

(4) Transfer learning. For downstream tasks, a task policy πT(δt,βt∣xt,ot)\pi_T(\delta_t, \beta_t|x_t, o_t) is trained with PPO to produce a residual correction δt\delta_t to the motion prior’s mean μtm\mu_t^m. Importantly, we adjudicate between the two priors (motion and behavior) using a continuous gating variable βt∈[−1,1]\beta_t \in[-1, 1] that is binarized to bt∈{0,1}b_t \in\{0,1\} via hysteresis thresholding. Thus, at each timestep, btb_t selects between two latent codes:

zt={μtm+δtif bt=1,μtb+ϵσ,ϵ∼N(0,1)if bt=0,(5)z_t = \begin{cases} \mu_t^m + \delta_t & \text{if } b_t = 1, \\ \mu_t^b + \epsilon\sigma, \quad\epsilon\sim\mathcal{N}(0,1) & \text{if } b_t = 0, \end{cases} \tag*{(5)}

where μtm\mu_t^m and μtb\mu_t^b are the means of pψ(zt∣xt)p_\psi(z_t|x_t) and pφ(zt∣xt)p_\varphi(z_t|x_t), respectively, and σ\sigma is a fixed noise scale. The resulting latent code is passed through the frozen decoder pθp_\theta to produce the action. Intuitively, bt=1b_t = 1 engages goal-directed residual control of the motion prior (exploitation), while bt=0b_t = 0 samples naturalistic motion from the behavior prior (exploration). Because πT\pi_T outputs βt\beta_t, the system learns when to explore and when to exploit.

Formal analysis of the priors

Why does SCAMPER use both a motion prior, pψp_\psi, for goal-directed motion and a behavior prior, pφp_\varphi, for exploration? Although both pψp_\psi and pφp_\varphi are proprioception-conditioned distributions over the same latent space, their training objectives differ. We formalize this distinction in two propositions.

Proposition 1 (Information bottleneck under joint training). When the encoder qϕ(z∣g,x)q_\phi(z|g,x) and the state-dependent prior pψ(z∣x)p_\psi(z|x) are trained jointly, minimizing their expected KL divergence is equivalent to the following optimization:

min⁡ϕ,ψEp(x,g)[DKL(qϕ∥pψ)]=min⁡ϕ,ψ[Iϕ(Z;G∣X)+Ep(x)[DKL(mϕ∥pψ)]],(6)\min_{\phi,\psi}\mathbb{E}_{p(x,g)}\left[D_{\mathrm{KL}}(q_\phi\mathbin{\|} p_\psi)\right] = \min_{\phi,\psi}\left[I_\phi(Z;G \mid X) + \mathbb{E}_{p(x)}\left[D_{\mathrm{KL}}(m_\phi\mathbin{\|} p_\psi)\right]\right], \tag*{(6)}

where mϕ(z∣x)=Ep(g∣x)[qϕ(z∣g,x)]m_{\phi}(\mathbf{z}|\mathbf{x}) = \mathbb{E}_{p(\mathbf{g}|\mathbf{x})}[q_{\phi}(\mathbf{z}|\mathbf{g}, \mathbf{x})] is the marginal posterior induced by the encoder, which evolved during joint training.

Joint training of the encoder and the motion prior thus not only forces the motion prior to approximate the marginal posterior, but also minimizes the encoder’s conditional mutual information between the latent states and the goals. However, from equation (3) both of these terms which the KL loss decomposes into are weighted by the same hyperparameter (λ\lambda). As a result, in practice, it is difficult to obtain a favorable balance between regularization via the bottleneck and the distortion of movement imitation. We find that setting λ\lambda in order to obtain the right balance limits how closely the motion prior can approximate the marginal posterior, yielding a motion prior concentrated on local motion continuation. We also find that stochastic sampling of the motion prior with high noise can occasionally push the system to switch behavioral modes, but as we show below (Figure 2), the motion prior alone does not reproduce the rich pattern of behavioral transitions observed in the training data. Despite all this, a motion prior focused on local motion continuation yields a network favorable for residual control. The marginal posterior, by contrast, contains the more fine-grained structure of the behavioral transitions. Therefore, in order to have the practical ability to generate diverse behavior during exploration we need another prior that only approximates the marginal posterior, allowing us to decouple the training dynamics from minimizing the conditional mutual information. We can achieve this by training a new prior, the behavior prior, with the frozen encoder.

Figure 2. SCAMPER is behaviorally diverse. (a) Renderings of behavior-prior rollouts at increasing sampling noise. (b) Vendi score vs. latent sampling noise σ=exp⁡(logvar/2)\sigma= \exp(\mathrm{logvar}/2), for the behavior prior (blue), motion prior (purple), and the training data (dashed gray); data shown is bootstrapped means with 95% CIs. (c) Per-rollout net displacement across σ\sigma; violins with interquartile lines, gray violin marks training data. (d) Spatial occupancy at σ=0.22\sigma= 0.22 shown as 2D histograms of body location (log-scaled frame counts) for the training data, motion prior, and behavior prior (see Figure 9 for all noise levels).

Figure 9. Spatial occupancy of all tested sampling noise levels plotted as 2D histograms of body location (log-scaled frame counts) for the motion prior and behavior prior.

Proposition 2 (Marginal posterior approximation under frozen encoder). When the encoder qϕ(z∣g,x)q_{\phi}(\mathbf{z}|\mathbf{g}, \mathbf{x}) is frozen, minimizing the expected KL divergence between the encoder and a state-dependent prior pφ(z∣x)p_{\varphi}(\mathbf{z}|\mathbf{x}) is equivalent to

Ep(x,g)[DKL(qϕ ∥ pφ)]=Iϕ(Z;G∣X)+min⁡φEp(x)[DKL(mϕ ∥ pφ)],(7)\mathbb{E}_{p(\mathbf{x},\mathbf{g})}\left[D_{\mathrm{KL}}(q_{\phi} \,\|\, p_{\varphi})\right] = I_{\phi}(Z;G \mid X) + \min_{\varphi}\mathbb{E}_{p(\mathbf{x})}\left[D_{\mathrm{KL}}(m_{\phi} \,\|\, p_{\varphi})\right], \tag*{(7)}

where mϕ(z∣x)=Ep(g∣x)[qϕ(z∣g,x)]m_{\phi}(\mathbf{z}|\mathbf{x}) = \mathbb{E}_{p(\mathbf{g}|\mathbf{x})}[q_{\phi}(\mathbf{z}|\mathbf{g}, \mathbf{x})] is the marginal posterior induced by the frozen encoder, held fixed during optimization. The conditional mutual information, Iϕ(Z;G∣X)I_{\phi}(Z;G \mid X), does not depend on the optimization variable φ\varphi; it is therefore an additive constant that sits outside the minimization. Hence, the optimal behavior prior is the closest possible state-conditional approximation to the marginal posterior mϕm_{\phi}.

Therefore, by directly distilling the behavior prior from the frozen pre-trained encoder, we are able to lower the KL divergence between it and the goal-rich encoder to a greater extent than we could for the motion prior which was limited by the rate-distortion tradeoff. Specifically, we achieve a KL divergence of 5.88 nats between the behavior prior and the encoder, vs 14.35 nats for the motion prior and the encoder. This allows the behavior prior to better capture the behavioral transitions in the training data. However, since the behavior prior is more inclined to sample from transitions in behavior in its control signal, it is less stable when residually controlled than the motion prior (Figure 13, Table 11). This motivates the use of the dual priors for a sparse-reward exploration task in § 4.5. We provide proofs for these two propositions in App. § 8 and empirical justification in App. § 15.§ 1. By training the behavior prior in this way, we achieve a good approximation for the marginal posterior, mϕ(z∣x)m_{\phi}(\mathbf{z}|\mathbf{x}), which can then be used for effective exploration with natural movements, as we show in the subsequent experiments.

GaitMetricEmpiricalSCAMPER (MP)SCAMPER (BP)MIMIC-MJXScratchSCAMPER (MP)SCAMPER (BP)MIMIC-MJXScratch
WalkStride Duration (ms)308 ± 91332 ± 16313 ± 34374 ± 23255 ± 2930.270.060.730.58
WalkStep Frequency (Hz)3.46 ± 0.753.02 ± 0.153.22 ± 0.332.69 ± 0.264.61 ± 2.620.580.311.021.53
WalkStance LH (ms)182 ± 82191 ± 21178 ± 26221 ± 36223 ± 2950.110.050.480.50
WalkStance RH (ms)175 ± 66183 ± 9208 ± 59247 ± 3180 ± 00.120.511.091.45
WalkStance LF (ms)171 ± 71172 ± 32137 ± 11153 ± 1520 ± 100.020.480.252.12
WalkStance RF (ms)174 ± 69183 ± 6217 ± 13187 ± 160 ± 00.130.620.192.53
WalkSwing LH (ms)125 ± 20144 ± 5128 ± 7152 ± 10150 ± 450.960.111.331.23
WalkSwing RH (ms)133 ± 31148 ± 5113 ± 21123 ± 8175 ± 00.510.650.291.37
WalkSwing LF (ms)136 ± 25152 ± 7174 ± 9219 ± 22120 ± 00.631.523.330.66
WalkSwing RF (ms)133 ± 30149 ± 394 ± 5184 ± 7255 ± 00.501.301.673.99
WalkDuty Factor LH.576 ± .074.576 ± .064.568 ± .082.593 ± .095.877 ± 1.1570.000.110.234.08
WalkDuty Factor RH.562 ± .055.550 ± .027.665 ± .188.660 ± .083.314 ± .0000.211.871.774.47
WalkDuty Factor LF.544 ± .059.519 ± .098.438 ± .036.409 ± .039.079 ± .0390.441.822.317.96
WalkDuty Factor RF.558 ± .055.552 ± .020.691 ± .041.502 ± .042.000 ± .0000.112.421.0310.14
WalkTemporal Symmetry.011 ± .068.016 ± .024−.050 ± .067−.049 ± .074.000 ± .0000.080.890.880.16
WalkPhase RH rel. LH.459 ± .041.513 ± .113.539 ± .104.368 ± .094.432 ± .0001.321.962.250.67
WalkPhase LF rel. LH.333 ± .047.459 ± .094.428 ± .062.516 ± .102.334 ± .2952.692.043.930.02
WalkPhase RF rel. LH.824 ± .045.906 ± .165.755 ± .077.835 ± .206.000 ± .0001.861.530.2618.48
WalkMean (Coordination)1.491.601.834.83
WalkMean (Gait Pattern)0.951.681.686.55
WalkMean (All)0.591.011.283.44
TrotStride Duration (ms)230 ± 31267 ± 10130 ± 4285 ± 92318 ± 3911.193.241.762.84
TrotStep Frequency (Hz)4.42 ± 0.563.75 ± 0.227.71 ± 0.264.74 ± 2.814.41 ± 2.521.185.840.570.01
TrotStance LH (ms)97 ± 27135 ± 1660 ± 2181 ± 4820 ± 01.461.383.182.88
TrotStance RH (ms)97 ± 25134 ± 670 ± 387 ± 501251 ± 23211.491.100.4046.48
TrotStance LF (ms)104 ± 2393 ± 538 ± 8116 ± 1810 ± 00.512.930.504.16
TrotStance RF (ms)104 ± 24118 ± 551 ± 4155 ± 2720 ± 00.602.252.153.56
TrotSwing LH (ms)134 ± 14129 ± 370 ± 3135 ± 26110 ± 100.334.640.061.72
TrotSwing RH (ms)133 ± 15135 ± 560 ± 380 ± 14190 ± 430.104.983.673.88
TrotSwing LF (ms)126 ± 16175 ± 592 ± 9211 ± 22308 ± 03.132.185.4411.69
TrotSwing RF (ms)126 ± 16150 ± 479 ± 4173 ± 7298 ± 01.472.932.8910.62
TrotDuty Factor LH.413 ± .065.506 ± .061.460 ± .012.636 ± .169.063 ± .0001.430.723.435.42
TrotDuty Factor RH.416 ± .060.502 ± .024.537 ± .025.306 ± .1763.931 ± 7.2911.422.021.8558.61
TrotDuty Factor LF.450 ± .053.348 ± .017.292 ± .061.407 ± .064.031 ± .0001.953.010.837.96
TrotDuty Factor RF.448 ± .055.442 ± .017.391 ± .030.543 ± .094.063 ± .0000.111.021.716.95
TrotTemporal Symmetry.004 ± .058.008 ± .068−.075 ± .017.172 ± .233−.434 ± .1620.211.233.067.46
TrotPhase RH rel. LH.488 ± .060.543 ± .092.508 ± .045.474 ± .210.401 ± .3470.912.980.241.45
TrotPhase LF rel. LH.473 ± .046.620 ± .075.406 ± .044.699 ± .121.000 ± .0003.211.474.9410.35
TrotPhase RF rel. LH.975 ± .070.921 ± .173.900 ± .142.108 ± .070.126 ± .0000.771.0712.3612.11
TrotMean (Coordination)1.271.695.157.84
TrotMean (Gait Pattern)1.401.763.6214.69
TrotMean (All)1.192.502.7211.01
GallopStride Duration (ms)192 ± 19175 ± 6140 ± 5255 ± 91172 ± 1000.852.673.321.01
GallopStep Frequency (Hz)5.27 ± 0.535.71 ± 0.237.15 ± 0.255.22 ± 2.547.95 ± 3.110.833.520.095.04
GallopStance LH (ms)78 ± 1256 ± 675 ± 988 ± 5315 ± 81.830.220.815.32
GallopStance RH (ms)74 ± 1366 ± 653 ± 575 ± 5016 ± 170.621.610.024.33
GallopStance LF (ms)76 ± 1131 ± 432 ± 5138 ± 400 ± 04.003.925.496.80
GallopStance RF (ms)84 ± 1647 ± 655 ± 5141 ± 280 ± 02.301.763.575.20
GallopSwing LH (ms)114 ± 13119 ± 365 ± 8115 ± 46172 ± 880.453.760.124.52
GallopSwing RH (ms)117 ± 10108 ± 687 ± 593 ± 13140 ± 1030.862.932.402.24
GallopSwing LF (ms)115 ± 11144 ± 5108 ± 5189 ± 30172 ± 02.550.666.535.07
GallopSwing RF (ms)108 ± 9128 ± 685 ± 5187 ± 9172 ± 02.162.458.426.82
GallopDuty Factor LH.406 ± .042.321 ± .033.538 ± .061.343 ± .207.087 ± .0442.043.141.527.63
GallopDuty Factor RH.386 ± .040.377 ± .037.277 ± .034.292 ± .197.094 ± .0980.230.242.377.36
GallopDuty Factor LF.398 ± .032.179 ± .025.230 ± .037.504 ± .157.000 ± .0006.755.174.4112.28
GallopDuty Factor RF.433 ± .048.266 ± .036.394 ± .036.552 ± .110.000 ± .0003.470.822.468.99
GallopTemporal Symmetry.025 ± .059−.084 ± .060.173 ± .062.086 ± .311−.095 ± .2541.832.491.022.03
GallopPhase RH rel. LH.789 ± .073.692 ± .084.697 ± .102.446 ± .183.540 ± .2981.331.264.733.44
GallopPhase LF rel. LH.560 ± .037.486 ± .034.790 ± .047.495 ± .211.000 ± .0001.990.261.7615.20
GallopPhase RF rel. LH.188 ± .062.202 ± .044.227 ± .044.168 ± .037.000 ± .0000.230.640.323.05
GallopMean (Coordination)1.352.661.965.93
GallopMean (Gait Pattern)2.292.502.518.28
GallopMean (All)1.912.422.745.91

Table 11. Full gait metric comparison to empirical rat data. Empirical, SCAMPER (MP), SCAMPER (BP), MIMIC-MJX, and Scratch columns show mean ± std. ∣Δμ∣/σemp|\Delta\mu|/\sigma_{\mathrm{emp}} is the absolute difference in means normalized by empirical standard deviation.

Stance-phase plots for each limb across walk, trot, and gallop gaits

Figure 13. Stance-phase plots for each limb (RF, LF, RH, LH) across walk, trot, and gallop gaits, comparing empirical data (purple; [39]) to the base SCAMPER model (MP, blue), the behavior-prior-controlled SCAMPER model (BP, dark blue), MIMIC-MJX (red), and the from-scratch model (gray); bars show stance duration, error bars show variability across strides.

Experiments

The behavior prior expands behavioral diversity beyond the motion prior

We first verify Propositions 1 and 2: that sampling the behavior prior produces greater behavioral diversity than sampling the motion prior (SuppVid[2, 3, 4]; App. § 10). We initialized the rodent in a realistic pose, then iteratively sampled a latent from one prior, decoded it to an action, and stepped the simulation for 5 s of simulated time per rollout (Figure 2a). We collected 842 rollouts per noise level, comparing pure behavior-prior sampling (bt=0b_t = 0) to pure motion-prior sampling (bt=1b_t = 1); see App. § 13.§ 1.

We measured the Vendi score [34], which captures the effective number of distinct behavioral modes. The Vendi score has been used to quantify trajectory and skill diversity in RL [35, 36]. Across all noise levels, the Vendi score was significantly higher for the behavior prior than the motion prior (Figure 2b). Motion-prior sampling (similar to PULSE) produced lower Vendi scores than the training data for all σ<1\sigma< 1. At very high noise, motion-prior diversity matched the training data but at the cost of motion realism (§ 4.2). In contrast, the behavior prior matched the training data diversity at low noise and exceeded it at high noise. This provides a tunable knob for controlling the behavioral diversity of the agent.

Next, to assess whether this diversity aids exploration, we measured the net distance traveled across sampling noise levels. Mirroring the Vendi score results, motion-prior sampling produced shorter trajectories than the training data at almost all noise levels (Figure 2c). Behavior-prior sampling roughly matched the training data distance at low noise and exceeded it at high noise. We also examined the spatial occupancy in the virtual arena. In the training data, the rat was in a circular arena, and thus, the occupancy was restricted to a circular area (Figure 2d, left). Motion-prior rollouts at low noise remained largely within this same area despite no physical barrier (Figure 2d, middle), demonstrating that the motion prior samples from a restricted distribution of movements seen in the training data. The behavior prior, in contrast, explored beyond this region (Figure 2d, right).

The behavior prior preserves motion realism under stochastic sampling

To verify that the movements induced by the behavior prior during random exploration were natural we used the Kernel Inception Distance (KID), which measures the dissimilarity between real and generated probability distributions [37] and has been used to measure the realism of generative human motion [38]. We trained a variational autoencoder (VAE) on the joint angle trajectories from the training data as the inception model, then computed the KID between the VAE features of the training and generated trajectories. Across all noise levels, the KID was lower for the behavior prior than the motion prior (Figure 3a). This confirms that the behavior prior produces more natural movements during random generation than the motion prior.

Figure 3. SCAMPER preserves motion realism during random behavior. (a) Kernel Inception Distance (KID) between random rollouts and the training data behavior distributions across sampling noise σ=exp⁡(logvar⁡/2)\sigma= \exp(\operatorname{logvar}/2), for the behavior prior (blue) and motion prior (purple). Data shown is means ±1\pm1 s.d. (b) Empirical cumulative distribution function (ECDF) of root height at σ=0.22\sigma= 0.22, compared against the training data (dashed gray).

For a more concrete measure of motion realism, we examined the rearing behavior of the virtual rat by measuring the proportion of time spent at various body heights. The behavior prior more closely matched the training data’s rearing distribution, while the motion prior diverged from it (Figure 3b). This held across different levels of noise (Figure 10), and was quantitatively confirmed using the Wasserstein distance of the rearing curves between the generated data and the training data (Table 9). Together with § 4.1, these results show that the behavior prior can generate diverse and realistic rat motion during random sampling.

Figure 10. Empirical cumulative distribution function of root height at all tested sampling noise levels compared against the training data reference (dashed gray).

Prior Networksσ=0\sigma=0σ=0.14\sigma=0.14σ=0.22\sigma=0.22σ=0.37\sigma=0.37σ=0.61\sigma=0.61σ=0.78\sigma=0.78σ=1.00\sigma=1.00
Behavior Prior.0085.0054.0045.0092.0106.0129.0162
Motion Prior.0116.0121.0115.0099.0126.0171.0173

Table 9. Wasserstein distance between empirical root-height distributions of prior rollouts and the training data, across sampling noise levels σ\sigma. Corresponds to results from Figure 10.

The motion prior helps produce realistic gaits at different speeds during transfer

Having confirmed that the behavior prior enables diverse and realistic random generation, we next tested whether residual control of the motion prior enables effective transfer to RL tasks. We began with a dense-reward locomotion task: at every timestep, the agent received a reward that decreased linearly with the difference between the target and actual speed. We trained separate policy networks for each target speed. SCAMPER closely matched the target speed across a range of realistic rat locomotion speeds (Figure 4a, blue line) whereas MIMIC-MJX failed to generalize beyond speeds seen in the training data (Figure 4a, red line). A model trained from scratch (as in Merel et al. [4]) also matched target speeds but with deeply unnatural motion and higher variance in its performance (Figure 4a, gray line).

Figure 4. SCAMPER produces realistic gaits at varying speeds. (a) Actual vs. target speed for SCAMPER (blue), MIMIC-MJX (red), and the from-scratch model (gray); lines show means, shaded bands show ±1 s.d. across seeds, dashed line indicates perfect tracking. The training-data speed distribution is shown in purple on the right. (b) Renderings of SCAMPER walking, trotting, and galloping at increasing target speeds. (c) Stance-phase plots for each limb (RF, LF, RH, LH) across walk, trot, and gallop gaits, comparing empirical rat data (purple; data from [39]) to SCAMPER (blue); bars show stance duration, error bars show variability across strides.

To better understand the resulting movement we examined the gaits at different speeds. Qualitatively, SCAMPER produced distinct gaits resembling walking, trotting, and galloping in real rats (Figure 4b). Notably, the training data—collected from freely behaving rats in a circular arena—contained no sustained or high-speed locomotion, and thus no trot or gallop. The emergence of these gaits in SCAMPER suggests it generalizes out-of-distribution while maintaining natural locomotion.

To verify this, we performed a quantitative gait analysis, as is standard in behavioral experiments [40, 41], comparing SCAMPER, MIMIC-MJX, and the from-scratch model against real rat data [39]. For each gait (walk, trot, gallop), SCAMPER matched real rat profiles both qualitatively (Figure 4c) and quantitatively (Table 1). Gait profiles of MIMIC-MJX and the from-scratch model significantly diverged from the real rats (Figure 13). Notably, MIMIC-MJX could not match target speeds above 0.5 m/s, so the higher-speed gaits (trot, gallop) never emerged. Together, these results show that residual control of the motion prior enables effective task transfer with natural movement and out-of-distribution gait generalization (see SuppVid[5, 6, 7, 8, 9, 10, 11, 12, 13, 14] for videos demonstrating these results; App. § 10).

MethodWalk Coord.Walk PatternWalk AllTrot Coord.Trot PatternTrot AllGallop Coord.Gallop PatternGallop All
SCAMPER1.490.950.591.271.401.191.352.291.91
MIMIC-MJX1.831.681.285.153.622.721.962.512.74
Scratch4.836.553.447.8414.6911.015.938.285.91

Table 1. Per-metric distance to empirical rat data, computed as the absolute difference in means divided by empirical standard deviation, ∣μemp−μvirtual∣/σemp|\mu_{emp} - \mu_{virtual}|/\sigma_{emp} (↓). Values shown are the average across metrics in each category: Coordination (temporal symmetry, inter-limb phase couplings), Pattern (phase couplings, duty factors), and All (18 gait metrics). (See Table 11 for all metrics)

The motion prior expands the stability and precision of rearing behaviors

To verify that these results extend beyond locomotion, we tested a dense-reward rearing task in which the virtual rat must lift its head to a target location (Figure 5a). The reward decreases linearly with the between the skull and the target, and a one-time bonus reward is obtained for holding at the target for a full second. We set the target at the maximum rearing height observed in the training data to test each model’s ability to use the full range of motion of the virtual body.

Figure 5. SCAMPER expands the precision and stability of learnable rearing behaviors. (a) Example rollouts of a walk-then-rear policy, with the target rearing height indicated by the red box, for policies trained using SCAMPER (top), MIMIC-MJX (middle), and trained from scratch (bottom). (b) Task success rate over simulation steps for each method; lines show the mean across seeds and bands show ±1 s.d. Only the SCAMPER policy solves the task consistently within the training budget.

SCAMPER eventually achieved a high success rate (Figure 5b, blue line). In contrast, MIMIC-MJX rarely held the target height long enough (Figure 5b, red line), and the from-scratch model never reached the target, unable to use the full dynamic range of the rat body in a stable manner (Figure 5b, gray line). Thus, SCAMPER enables effective task transfer with realistic motion for both locomotion and rearing (see SuppVid[15, 16, 17] for videos demonstrating these results; App. § 10).

Adaptive prior gating balances exploration and exploitation in a sparse-reward foraging task

Finally, we tested whether SCAMPER’s gating mechanism could switch between the behavior and motion priors to balance exploration and exploitation in a sparse-reward task. We chose a foraging task relevant to behavioral neuroscience: the virtual rodent must explore a circular arena and navigate to goal locations to collect treats (Figure 6a). In each trial, the rodent spawned at the center of the arena. To force exploration, treats spawned only outside a minimum radius. Each treat emitted a Gaussian “scent” field, with the rodent receiving the local odor gradient as a state variable. Collecting a treat (head overlapping its location) yielded reward and removed both the treat and its scent.

Figure 6. SCAMPER enables efficient foraging under increasingly sparse reward. (a) Arena forage task: treats (blue circles) are spawned uniformly within an annulus bounded by the minimum spawn radius (red dashed) and the arena boundary (orange), with a scent field (colorbar) guiding the rodent from its start position (white). (b) Number of treats collected per episode as a function of the minimum spawn radius, for policies trained using SCAMPER (blue), a motion-prior-only variant (purple), MIMIC-MJX (red), and trained from scratch (gray). Lines show the mean across seeds and bands show ±1 s.d.

We compared the full SCAMPER model (Figure 6b, blue line) against a motion-prior-only variant (Figure 6b, purple line), MIMIC-MJX (Figure 6b, red line), and the from-scratch model (Figure 6b, gray line). All models found treats at small spawn radii, where little exploration was required. However, only the full SCAMPER model with the ability to switch to random generation via the behavior prior (Figure 12) was capable of solving the task when the minimum spawn radius was large and extensive exploration was necessary. This demonstrates that combining the behavior and motion priors enables SCAMPER to train on realistic sparse-reward tasks that require effective exploration (see SuppVid[18, 19, 20, 21, 22] for videos demonstrating these results; App. § 10).

Figure 12. SCAMPER’s training progression on the arena foraging task. Top-down view of trajectories the rodent takes along with the evolution of the gate logit (βt\beta_t) and its resulting binarization (btb_t). Here when a treat is collected it is still displayed in the arena for visualization purposes, but in practice it disappear upon successful collection. bt=0b_t = 0 results in stochastic sampling of the behavior prior. bt=1b_t = 1 results in residual control of the motion prior. During early training the rodent relies heavily on the behavior prior to explore its environment. During mid training, the rodent begins to gate rapidly between both priors as it begins to exploit the reward signals from the collection of treats. During late training the rodent mostly uses residual control of the motion prior to effectively exploit the task structure.

Discussion

Summary. In this paper, we introduced SCAMPER, a system for training a realistic rat body on realistic behavioral tasks. SCAMPER enables natural movement generation during both goal-directed behavior and random exploration, thanks to its dual-prior architecture. We demonstrated that the behavior prior expands the diversity of randomly sampled motion, while preserving motion realism. We also showed that through residual control of the motion prior, SCAMPER can produce realistic gaits at different speeds and fully leverage its rearing capabilities. Furthermore, we demonstrated that by combining both priors the virtual rodent can solve more difficult sparse-reward tasks that demand exploration.

Significance for neuro-AI. SCAMPER is designed primarily as a tool for neuro-AI. It builds on established techniques for imitation, latent skill learning, and residual control; its additions, a behavior prior distilled from the frozen encoder and a learned gate between the two priors, extend naturalistic motion from goal-directed control to exploration. Naturalistic exploration matters for neuroscience: most experimental tasks offer only sparse reward, so models must explore to learn them, and unnatural movement would confound comparisons with neural data. Aldarondo et al. [5] found that the internal representations of a purely imitative virtual rat matched neural activity in rat motor cortex and dorsolateral striatum. SCAMPER makes it possible to extend such comparisons to goal-directed tasks, with models and animals trained on the same task and compared throughout learning.

Limitations & next steps. There are several limitations to this work that should be recognized. First and foremost, the dataset that we used for training the model was very limited. It consisted of only 1 hour of a rodent freely behaving in an open arena. As such, it contained only simple behaviors, namely walking, rearing, sitting still, and grooming. A more extensive dataset with rats engaging in more diverse behaviors, exhibiting longer bouts of consistent locomotion at various speeds, and movement in more diverse landscapes, would be of great benefit. Another key, but missing, component in the system we presented here is the presence of rich, multi-modal sensory inputs. The current version of SCAMPER uses only proprioceptive inputs, and a simple proxy for scent, effectively making the rat blind and deaf. As the field moves towards more sophisticated embodied models and experiments, this limitation will become more pressing to address. Future work can expand SCAMPER with a larger array of realistic sensory inputs. Finally, one key difference between SCAMPER and real motion control in animals is the use of torque actuators rather than muscles. Real animals must learn to control their bodies through activation of muscle fibers, and a good embodied model would likely take this facet of animal behavior into consideration. In principle, SCAMPER is agnostic to the actuator type, so as new virtual bodies get developed, SCAMPER could be applied to muscle-based models.

Acknowledgments and Disclosure of Funding

The authors would like to thank Nanda Krishna, João Sacramento and Guillaume Lajoie for discussions, support, and feedback. AS was supported by a Fonds de recherche du Québec – Nature et technologies (FRQNT) Master’s Research Scholarship, a Natural Sciences and Engineering Research Council of Canada (NSERC) Canada Graduate Research Scholarship – Master’s (CGRS M), and a UNIQUE Excellence Scholarship. CYZ was supported by the NSF CSGrad4US Fellowship. This work was also supported by a L.I.F.E. Foundation grant to TDP, a Salk Innovation Award to TDP, an NIH BRAIN Initiative grant (U01NS136507) to BPÖ and TDP, a SFARI Investigator Award to BPÖ, and an ARNI award to BPÖ and BAR. This research was enabled in part by compute resources provided by Mila (https://mila.quebec), Calcul Québec (https://www.calculquebec.ca), and the Digital Research Alliance of Canada (https://alliancecan.ca).

Competing interests

BAR is a Research Scientist at Google and AS is a Student Researcher at Google. The authors declare no other competing interests.

Proofs

We restate each proposition from § 3.3 and provide the full proof.

Proposition 1 (Information bottleneck under joint training). When the encoder qϕ(z∣g,x)q_{\phi}(z \mid g, x) and the state-dependent prior pψ(z∣x)p_{\psi}(z \mid x) are trained jointly, minimizing their expected KL divergence decomposes as

min⁡ϕ,ψEp(x,g)[DKL(qϕ∥pψ)]=min⁡ϕ,ψ[Iϕ(Z;G∣X)+Ep(x)[DKL(mϕ∥pψ)]],(8)\min_{\phi,\psi} \mathbb{E}_{p(x,g)}\left[D_{\mathrm{KL}}(q_{\phi} \mathbin{\|} p_{\psi})\right] = \min_{\phi,\psi}\left[I_{\phi}(Z;G \mid X) + \mathbb{E}_{p(x)}\left[D_{\mathrm{KL}}(m_{\phi} \mathbin{\|} p_{\psi})\right]\right], \tag*{(8)}

where mϕ(z∣x)=Ep(g∣x)[qϕ(z∣g,x)]m_{\phi}(z \mid x) = \mathbb{E}_{p(g \mid x)}[q_{\phi}(z \mid g, x)] is the marginal posterior induced by the encoder, which evolved during joint training.

Proof. Let the joint optimization objective be the expected KL divergence over the data distribution:

L(ϕ,ψ)=min⁡ϕ,ψEp(x,g)[DKL(qϕ(z∣g,x)∥pψ(z∣x))].(9)\mathcal{L}(\phi,\psi) = \min_{\phi,\psi} \mathbb{E}_{p(x,g)}\left[D_{\mathrm{KL}}(q_{\phi}(z \mid g,x) \mathbin{\|} p_{\psi}(z \mid x))\right]. \tag*{(9)}

Expanding the expected KL divergence into its integral form:

L(ϕ,ψ)=min⁡ϕ,ψ∫p(x)∫p(g∣x)∫qϕ(z∣g,x)log⁡qϕ(z∣g,x)pψ(z∣x) dz dg dx.(10)\mathcal{L}(\phi,\psi) = \min_{\phi,\psi} \int p(x) \int p(g \mid x) \int q_{\phi}(z \mid g,x) \log\frac{q_{\phi}(z \mid g,x)}{p_{\psi}(z \mid x)} \, dz\,dg\,dx. \tag*{(10)}

Define the marginal posterior (which depends on the encoder parameters ϕ\phi):

mϕ(z∣x)≜∫qϕ(z∣g,x)p(g∣x) dg.(11)m_{\phi}(z \mid x) \triangleq\int q_{\phi}(z \mid g,x)p(g \mid x)\,dg. \tag*{(11)}

Multiplying numerator and denominator inside the logarithm by mϕ(z∣x)m_{\phi}(z \mid x):

log⁡qϕ(z∣g,x)pψ(z∣x)=log⁡qϕ(z∣g,x)pψ(z∣x)mϕ(z∣x)mϕ(z∣x)=log⁡qϕ(z∣g,x)mϕ(z∣x)+log⁡mϕ(z∣x)pψ(z∣x).(12)\log\frac{q_{\phi}(z \mid g,x)}{p_{\psi}(z \mid x)} = \log\frac{q_{\phi}(z \mid g,x)}{p_{\psi}(z \mid x)} \frac{m_{\phi}(z \mid x)}{m_{\phi}(z \mid x)} = \log\frac{q_{\phi}(z \mid g,x)}{m_{\phi}(z \mid x)} + \log\frac{m_{\phi}(z \mid x)}{p_{\psi}(z \mid x)}. \tag*{(12)}

Substituting this decomposition back into the integral yields two terms:

min⁡ϕ,ψL(ϕ,ψ)=min⁡ϕ,ψ[∫p(x)∫p(g∣x)∫qϕ(z∣g,x)log⁡qϕ(z∣g,x)mϕ(z∣x) dz dg dx⏟Term 1+∫p(x)∫p(g∣x)∫qϕ(z∣g,x)log⁡mϕ(z∣x)pψ(z∣x) dz dg dx⏟Term 2].(13)\begin{aligned} \min_{\phi,\psi}\mathcal{L}(\phi,\psi) = \min_{\phi,\psi}\Bigg[& \underbrace{\int p(x)\int p(g \mid x)\int q_{\phi}(z \mid g,x)\log\frac{q_{\phi}(z \mid g,x)}{m_{\phi}(z \mid x)}\,dz\,dg\,dx}_{\text{Term 1}}\\ &+\underbrace{\int p(x)\int p(g \mid x)\int q_{\phi}(z \mid g,x)\log\frac{m_{\phi}(z \mid x)}{p_{\psi}(z \mid x)}\,dz\,dg\,dx}_{\text{Term 2}}\Bigg]. \tag*{(13)} \end{aligned}

Term 1 is, by definition, the conditional mutual information between the latent zz and the goal gg given the state xx:

Term 1=Iϕ(Z;G∣X).(14)\text{Term 1} = I_{\phi}(Z;G \mid X). \tag*{(14)}

Term 2: Since log⁡mϕ(z∣x)pψ(z∣x)\log\frac{m_{\phi}(z \mid x)}{p_{\psi}(z \mid x)} is independent of gg, and the integrand is absolutely integrable1, we apply Fubini’s theorem to evaluate the integral over gg first:

Term 2=∫p(x)∫(∫qϕ(z∣g,x)p(g∣x) dg)log⁡mϕ(z∣x)pψ(z∣x) dz dx=Ep(x)[DKL(mϕ(z∣x)∥pψ(z∣x))].(15)\begin{aligned} \text{Term 2} &= \int p(x)\int\left(\int q_{\phi}(z \mid g,x)p(g \mid x)\,dg\right)\log\frac{m_{\phi}(z \mid x)}{p_{\psi}(z \mid x)}\,dz\,dx\\ &= \mathbb{E}_{p(x)}\left[D_{\mathrm{KL}}(m_{\phi}(z \mid x) \mathbin{\|} p_{\psi}(z \mid x))\right]. \tag*{(15)} \end{aligned}

Combining the two terms gives the desired decomposition:

min⁡ϕ,ψL(ϕ,ψ)=min⁡ϕ,ψ[Iϕ(Z;G∣X)+Ep(x)[DKL(mϕ(z∣x)∥pψ(z∣x))]].(16)\min_{\phi,\psi}\mathcal{L}(\phi,\psi) = \min_{\phi,\psi}\left[I_{\phi}(Z;G \mid X) + \mathbb{E}_{p(x)}\left[D_{\mathrm{KL}}(m_{\phi}(z \mid x) \mathbin{\|} p_{\psi}(z \mid x))\right]\right]. \tag*{(16)}

Proposition 2 (Marginal posterior approximation under frozen encoder). When the encoder qϕ(z∣g,x)q_{\phi}(z|g,x) is frozen, minimizing the expected KL divergence between the encoder and a state-dependent prior pϕ(z∣x)p_{\phi}(z|x) is equivalent to

min⁡ϕEp(x,g)[DKL(qϕ∥pϕ)]=Iϕ(Z;G∣X)+min⁡ϕEp(x)[DKL(mϕ∥pϕ)],(17)\min_{\phi}\mathbb{E}_{p(x,g)}\left[D_{\mathrm{KL}}(q_{\phi}\Vert p_{\phi})\right]=I_{\phi}(Z;G\mid X)+\min_{\phi}\mathbb{E}_{p(x)}\left[D_{\mathrm{KL}}(m_{\phi}\Vert p_{\phi})\right], \tag*{(17)}

where mϕ(z∣x)=Ep(g∣x)[qϕ(z∣g,x)]m_{\phi}(z|x)=\mathbb{E}_{p(g|x)}[q_{\phi}(z|g,x)] is the marginal posterior induced by the frozen encoder, held fixed during optimization. The conditional mutual information, Iϕ(Z;G∣X)I_{\phi}(Z;G\mid X), does not depend on the optimization variable ϕ\phi; it is therefore an additive constant that sits outside the minimization. Hence, the optimal behavior prior is the closest state-conditional approximation to the marginal posterior mϕm_{\phi}.

Proof. Let the distillation objective be the expected KL divergence over the data distribution:

min⁡ϕL(ϕ)=min⁡ϕEp(x,g)[DKL(qϕ(z∣g,x)∥pϕ(z∣x))].(18)\min_{\phi}\mathcal{L}(\phi)=\min_{\phi}\mathbb{E}_{p(x,g)}\left[D_{\mathrm{KL}}\left(q_{\phi}(z|g,x)\mathbin\Vert p_{\phi}(z|x)\right)\right]. \tag*{(18)}

Expanding the inner KL divergence as the difference between cross-entropy and entropy:

DKL(qϕ∥pϕ)=−∫qϕ(z∣g,x)log⁡pϕ(z∣x) dz⏟H(qϕ,pϕ)+∫qϕ(z∣g,x)log⁡qϕ(z∣g,x) dz⏟−H(qϕ).(19)D_{\mathrm{KL}}(q_{\phi}\mathbin\Vert p_{\phi})=-\underbrace{\int q_{\phi}(z|g,x)\log p_{\phi}(z|x)\,dz}_{H(q_{\phi},p_{\phi})}+\underbrace{\int q_{\phi}(z|g,x)\log q_{\phi}(z|g,x)\,dz}_{-H(q_{\phi})}. \tag*{(19)}

Since the encoder parameters ϕ\phi are frozen, the entropy H(qϕ)H(q_{\phi}) is constant with respect to ϕ\phi and can be pulled outside the minimization:

min⁡ϕL(ϕ)=min⁡ϕ[Ep(x,g)[H(qϕ,pϕ)]]−Ep(x,g)[H(qϕ)].(20)\min_{\phi}\mathcal{L}(\phi)=\min_{\phi}\left[\mathbb{E}_{p(x,g)}[H(q_{\phi},p_{\phi})]\right]-\mathbb{E}_{p(x,g)}[H(q_{\phi})]. \tag*{(20)}

First we focus on the first term inside the minimization and expand it out:

Ep(x,g)[H(qϕ,pϕ)]=−∫p(x)∫p(g∣x)∫qϕ(z∣g,x)log⁡pϕ(z∣x) dz dg dx.(21)\mathbb{E}_{p(x,g)}[H(q_{\phi},p_{\phi})]=-\int p(x)\int p(g|x)\int q_{\phi}(z|g,x)\log p_{\phi}(z|x)\,dz\,dg\,dx. \tag*{(21)}

Assuming absolute integrability, we apply Fubini’s theorem to swap the order of integration between gg and zz, and factor out log⁡pϕ(z∣x)\log p_{\phi}(z|x) since it is independent of gg:

Ep(x,g)[H(qϕ,pϕ)]=−∫p(x)∫log⁡pϕ(z∣x)(∫qϕ(z∣g,x)p(g∣x) dg⏟mϕ(z∣x)) dz dx=−∫p(x)∫mϕ(z∣x)log⁡pϕ(z∣x) dz dx=Ep(x)[H(mϕ,pϕ)].(22)\begin{aligned} \mathbb{E}_{p(x,g)}[H(q_{\phi},p_{\phi})] &=-\int p(x)\int\log p_{\phi}(z|x)\left(\underbrace{\int q_{\phi}(z|g,x)p(g|x)\,dg}_{m_{\phi}(z|x)}\right)\,dz\,dx \\ &=-\int p(x)\int m_{\phi}(z|x)\log p_{\phi}(z|x)\,dz\,dx \\ &=\mathbb{E}_{p(x)}[H(m_{\phi},p_{\phi})]. \tag*{(22)} \end{aligned}

This is the expectation of the cross-entropy between mϕm_{\phi} and pϕp_{\phi}. Substituting back into equation (20) yields:

min⁡ϕL(ϕ)=min⁡ϕ[Ep(x)[H(mϕ,pϕ)]]−Ep(x,g)[H(qϕ)].(23)\min_{\phi}\mathcal{L}(\phi)=\min_{\phi}\left[\mathbb{E}_{p(x)}[H(m_{\phi},p_{\phi})]\right]-\mathbb{E}_{p(x,g)}[H(q_{\phi})]. \tag*{(23)}

To relate it to the KL divergence, we add and subtract the expectation of the entropy of the marginal posterior:

min⁡ϕL(ϕ)=min⁡ϕ[Ep(x)[H(mϕ,pϕ)]−Ep(x)[H(mϕ)]]+Ep(x)[H(mϕ)]−Ep(x,g)[H(qϕ)].(24)\min_{\phi}\mathcal{L}(\phi)=\min_{\phi}\left[\mathbb{E}_{p(x)}[H(m_{\phi},p_{\phi})]-\mathbb{E}_{p(x)}[H(m_{\phi})]\right]+\mathbb{E}_{p(x)}[H(m_{\phi})]-\mathbb{E}_{p(x,g)}[H(q_{\phi})]. \tag*{(24)}

Pulling Ep(x)[H(mϕ)]\mathbb{E}_{p(x)}[H(m_{\phi})] into the minimization operation and using the identity H(mϕ,pϕ)=DKL(mϕ∥pϕ)+H(mϕ)H(m_{\phi},p_{\phi})=D_{\mathrm{KL}}(m_{\phi}\Vert p_{\phi})+H(m_{\phi}):

min⁡ϕL(ϕ)=min⁡ϕEp(x)[DKL(mϕ(z∣x)∥pϕ(z∣x))]+(Ep(x)[H(mϕ)]−Ep(x,g)[H(qϕ)]).(25)\min_{\phi}\mathcal{L}(\phi)=\min_{\phi}\mathbb{E}_{p(x)}\left[D_{\mathrm{KL}}(m_{\phi}(z|x)\mathbin\Vert p_{\phi}(z|x))\right]+\left(\mathbb{E}_{p(x)}[H(m_{\phi})]-\mathbb{E}_{p(x,g)}[H(q_{\phi})]\right). \tag*{(25)}

The bracketed difference of entropies is, by definition, the conditional mutual information between the latent and the goal given the state under the frozen encoder, therefore:

min⁡ϕL(ϕ)=min⁡ϕEp(x)[DKL(mϕ(z∣x)∥pϕ(z∣x))]+Iϕ(Z;G∣X).(26)\min_{\phi}\mathcal{L}(\phi)=\min_{\phi}\mathbb{E}_{p(x)}\left[D_{\mathrm{KL}}(m_{\phi}(z|x)\mathbin\Vert p_{\phi}(z|x))\right]+I_{\phi}(Z;G\mid X). \tag*{(26)}

□\square

Dataset

Motion capture. We use a rat motion dataset collected by Aldarondo et al. [5]. The dataset is composed of 842 5 s clips of rats freely behaving in a circular enclosure. The mocap data is characterized by 23 keypoints captured at 50 Hz using DANNCE v1.3 [42] from multicamera video. The dataset contains various kinds of walking, rearing, grooming and prone behaviors. The mocap keypoints are aligned to the virtual rodent body model through STAC-MJX [6].

Empirical rat locomotion data. The empirical data used in the gait analysis (phase differences, duty factors, and gait classifications) were obtained from Danner et al. [39], publicly available at https://github.com/dannerlab/rat-sci-locomotion under a Creative Commons Attribution–ShareAlike 4.0 International license (CC BY–SA 4.0, https://creativecommons.org/licenses/by-sa/4.0/). The data was used unmodified.

Supplementary videos

We host the supplementary videos demonstrating the rodent performing the various tasks described in this work on the project’s page (https://aidansirbu.com/SCAMPER-site/). Specific videos are also referenced throughout the text: “SuppVid<x>” refers to the corresponding video on the website and the .mp4 file in the supplementary video package.

Implementation & compute

Implementation. SCAMPER is implemented on top of the MuJoCo physics engine [43]. We accelerate simulation using MuJoCo XLA (MJX) [44], a JAX-based [45] reimplementation of MuJoCo, which enables GPU-vectorized physics rollouts and domain randomization to be applied directly to the per-environment parameters. Environments are built on top of MuJoCo Playground’s [46] base class and wrappers, which expose MJX simulation through a JAX-compatible interface and adapt our environments to the Brax [47] training API. RL is performed with Brax, whose PPO implementation and distributed training utilities we extend to support our networks and distillation objectives. Pretraining (Stages 1-3) was performed using the JAX backend of MJX and task transfer was performed using the Warp backend of MJX. MIMIC-MJX baselines were implemented using the training setup and hyperparameters described in Zhang et al. [6]. The from-scratch baseline uses position (PD) actuators, whereas all other models use torque actuators. Torque actuators are highly unstable and difficult to learn from scratch, so we give the from-scratch model PD actuators to give it a fair chance to compete.

Compute. All experiments were run on a single GPU with 4 CPU cores and 48 GB of host RAM. Pretraining used NVIDIA A40S (48 GB) GPUs: the Stage 1 expert imitation trains for 3×1093 \times10^{9} environment steps over 4096 parallel envs in ≈14\approx14 h, the Stage 2 distillation of the encoder/decoder and motion prior trains for 1×1091 \times10^{9} steps over 4096 envs in ≈12\approx12 h, and the Stage 3 behavior prior trains for 2×1092 \times10^{9} steps over 4096 envs in ≈30\approx30 h, for a total of roughly 56 L40S-hours per end-to-end pretraining run. The three task-transfer benchmarks were trained with PPO under MuJoCo Warp on NVIDIA A100 (80 GB) GPUs, with four seeds per condition. Walk-then-rear was run for the SCAMPER, MIMIC-MJX, and from-scratch models, with each job training for 1×1091 \times10^{9} steps over 512 envs in ≈11\approx11 h (12 jobs total, ≈132\approx132 A100-hours). Locomotion was run for the same three models swept over 7 target speeds, with each job training for 4×1084 \times10^{8} steps over 1024 envs in ≈10\approx10 h (84 jobs total, ≈840\approx840 A100-hours). Arena foraging was run for SCAMPER, a motion-prior-only variant, MIMIC-MJX, and the from-scratch model swept over 7 minimum spawn radii, with each job training for 2×1092 \times10^{9} steps over 1024 envs in ≈6\approx6 h (112 jobs total, ≈672\approx672 A100-hours). One full reproduction of the paper’s training pipeline therefore costs roughly 56 L40S-GPU-hours and 1,650 A100-GPU-hours.

Tasks

Stage 1: Expert imitator

Task. The expert imitator πI(at∣xt,gt)\pi_I(a_t \mid x_t, g_t) tracks 3D motion-capture trajectories of freely behaving rats using the biomechanical rat model of Merel et al. [4]. We use a ∼1\sim1 h subset of the open-source rat mocap dataset released by Aldarondo et al. [5] (842 clips of 5 s mocap), registered to the virtual rat via STAC-MJX [6]. At the start of each episode we sample a clip uniformly from the training partition (80% of the dataset; the remaining 20% is held out for evaluation rollouts) and a starting frame uniformly in [0, 50], with zero initial joint velocity. Episodes last up to 250 reference frames. At each control step the agent observes the proprioception vector xtx_t and the egocentric goal gtg_t defined in § 3.1, where gtg_t is computed from a 5-frame reference horizon, and emits torques at∈R38a_t \in\mathbb{R}^{38} applied to the rat’s actuated joints. Control runs at 100 Hz on top of a 500 Hz MuJoCo MJX physics simulation, with a 50 Hz mocap reference. The body model uses torque actuators and a 0.9× rescaling factor.

Reward. Each imitation tracking term takes the form ri=wiexp⁡(−12(ei/σi)2)r_i = w_i \exp\left(-\frac{1}{2}(e_i/\sigma_i)^2\right), a Gaussian kernel of the per-term error eie_i with characteristic scale σi\sigma_i; values of (wi,σi)(w_i, \sigma_i) are tabulated in Table 2. The four active tracking terms penalize errors in root position, root orientation (quaternion geodesic), joint angles, and egocentric Cartesian end-effector positions. Three regularization costs penalize quadratic control magnitude, quadratic control differences (jerk), and clipped torque–velocity product, and a torso-zz-range term adds an in-range bonus that rewards keeping the torso height within the healthy range [0.0325, 0.5] m.

GroupHyperparameterValue
EnvironmentSolverNewton
EnvironmentSolver iterations5
EnvironmentLine-search iterations5
EnvironmentMax contacts (naconmaxn_{\mathrm{aconmax}})256
EnvironmentMax constraint rows (njmaxn_{\mathrm{jmax}})128
EnvironmentSimulation timestep2×10−32 \times10^{-3} s (500 Hz)
EnvironmentControl timestep1×10−21 \times10^{-2} s (100 Hz)
EnvironmentMocap rate50 Hz
EnvironmentClip length250 frames
EnvironmentReference horizon5 frames
EnvironmentEpisode start-frame range[0, 50]
EnvironmentInitial joint velocityzeros
EnvironmentTrain / eval clip split0.8 / 0.2
Domain randomizationFloor frictionU(0.4,1.0)\mathcal{U}(0.4, 1.0)
Domain randomizationStatic friction scaleU(0.9,1.1)\mathcal{U}(0.9, 1.1)
Domain randomizationArmature scaleU(1.0,1.05)\mathcal{U}(1.0, 1.05)
Domain randomizationCenter-of-mass jitter (m)U(−0.05,0.05)\mathcal{U}(-0.05, 0.05)
Domain randomizationLink mass scaleU(0.9,1.1)\mathcal{U}(0.9, 1.1)
Domain randomizationTorso mass jitterU(−0.02,0.02)\mathcal{U}(-0.02, 0.02)
Domain randomizationInitial qposq_{\mathrm{pos}} jitterU(−0.05,0.05)\mathcal{U}(-0.05, 0.05)
Reward weights (ww, scale)Root positionw=1.0w=1.0, σ=0.035\sigma=0.035
Reward weights (ww, scale)Root orientation (quaternion)w=1.0w=1.0, σ=20.0\sigma=20.0
Reward weights (ww, scale)Joint anglesw=1.0w=1.0, σ=1.41\sigma=1.41
Reward weights (ww, scale)Joint velocitiesw=0.0w=0.0, σ=1.0\sigma=1.0
Reward weights (ww, scale)Body positionsw=0.0w=0.0, σ=0.25\sigma=0.25
Reward weights (ww, scale)End-effectorsw=1.0w=1.0, σ=0.03\sigma=0.03
Reward weights (ww, scale)Healthy torso zz-range (m)[0.0325,0.5][0.0325, 0.5], w=1.0w=1.0
Reward weights (ww, scale)Control costw=0.02w=0.02
Reward weights (ww, scale)Control-diff costw=0.02w=0.02
Reward weights (ww, scale)Energy cost (clipped at 50)w=0.01w=0.01
TerminationRoot-position deviation>0.1> 0.1 m
TerminationRoot-orientation deviation>60∘> 60^\circ
TerminationPose error (L2)>4.5> 4.5
Network architecturePolicy architectureMLP (no latent bottleneck)
Network architecturePolicy hidden sizes[2048, 1024, 1024, 512]
Network architectureCritic hidden sizes[2048, 1024, 1024, 1024, 1024, 512]
Network architectureObservation normalizationrunning mean/std
PPO trainingParallel environments4,096
PPO trainingTotal environment steps3×1093 \times10^9
PPO trainingBatch size1,024
PPO trainingMinibatches per epoch16
PPO trainingUpdates per batch4
PPO trainingLearning rate (Adam)1×10−41 \times10^{-4}
PPO trainingGradient clip threshold20.0
PPO trainingPPO clip range ϵ\epsilon0.2
PPO trainingDiscount factor γ\gamma0.95
PPO trainingGAE / unroll length20
PPO trainingAction repeat1
PPO trainingEntropy coefficient1×10−21 \times10^{-2}
PPO trainingValue-loss coefficient0.25
PPO trainingReward scaling1
PPO trainingEvaluation / reset interval2.5×1072.5 \times10^7 steps

Table 2. Hyperparameters used for Stage 1 (train expert imitator) of the SCAMPER training pipeline.

Termination. An episode terminates early if (i) the root position deviates from the reference by more than 0.1 m, (ii) the root orientation deviates by more than 60°, (iii) the pose L2L_2 error exceeds 4.5, or (iv) any state value becomes NaN. Episodes also terminate at the end of the reference clip.

Domain randomization. To encourage robust control across variations in the biomechanical model, at the start of each episode we sample physics parameters uniformly within fixed ranges and hold them fixed for the duration of the episode [48]. Randomized parameters include floor friction, static friction scale, joint armature scale, link mass scale, torso mass jitter, center-of-mass jitter, and an additive jitter on the initial joint configuration θ0\theta_0; full ranges are listed in Table 2.

Network architecture. The expert imitator is a multi-layer perceptron with no latent bottleneck: the concatenation of xtx_t and gtg_t, normalized by their running mean and variance, is fed directly to a [2048, 1024, 1024, 512] policy MLP that outputs the mean and log-standard-deviation of a tanh-squashed diagonal Gaussian over the 38-dimensional action, so that at∈[−1,1]38a_t \in[-1, 1]^{38}. The critic is a separate [2048, 1024, 1024, 1024, 1024, 512] MLP estimating the state-value function. Both networks use SiLU activations with LayerNorm between hidden layers, and are trained from random initialization using LeCun initialization with zero bias.

Training. We train πI\pi_I with PPO [32] for 3×1093 \times10^9 environment steps across 4,096 parallel environments. Each PPO update uses a 20-step unroll, batch size 1,024, and 4 epochs of 16 minibatches per batch, with Adam at learning rate 10−410^{-4} and gradient clipping at 20.0. We use discount γ=0.95\gamma= 0.95, PPO clip range ϵ=0.2\epsilon= 0.2, entropy coefficient 10−210^{-2}, value-loss coefficient 0.25, and reward scaling 1. Evaluation rollouts on the held-out clip partition are performed every 2.5×1072.5 \times10^7 environment steps with deterministic (mean) actions. Training is implemented in JAX [45] with MuJoCo MJX [44] and runs on a single GPU; the full list of training hyperparameters is given in Table 2.

Stage 2: Online distillation

Task. Online distillation reuses the imitation environment from Stage 1 (App. § 12.§ 1): identical clip set and train/eval split, control rate, physics solver, reward terms, termination criteria, and domain randomization (Table 2). The Stage 1 expert imitator πI\pi_I is loaded as a frozen teacher and queried for target actions ata_t on every visited state, while a student latent skill controller—the encoder qϕ(zt∣xt,gt)q_\phi(z_t \mid x_t, g_t), decoder pθ(at∣xt,zt)p_\theta(a_t \mid x_t, z_t), and proprioception-conditioned motion prior pψ(zt∣xt)p_\psi(z_t \mid x_t)—is rolled out in simulation and trained jointly to match the teacher, using a similar DAGGer [33] schedule as PULSE [13]. Episode terminations and rewards are inherited from the underlying environment but are not used as a learning signal: the student is supervised against the teacher’s actions, not the imitation reward.

Loss. The student minimizes the composite distillation objective

Ldistill=LDAgger+λLKL+αLAR,\mathcal{L}_{\mathrm{distill}} = \mathcal{L}_{\mathrm{DAgger}} + \lambda\mathcal{L}_{\mathrm{KL}} + \alpha\mathcal{L}_{\mathrm{AR}},

where LKL=DKL(qϕ∥pψ)\mathcal{L}_{\mathrm{KL}} = D_{\mathrm{KL}}(q_{\phi} \mathbin\Vert p_{\psi}) regularizes the encoder toward the motion prior and LAR=∥μte−μt−1e∥22\mathcal{L}_{\mathrm{AR}} = \lVert\mu_{t}^{e} - \mu_{t-1}^{e} \rVert_{2}^{2} promotes temporal smoothness in the encoder mean. In practice LDAgger\mathcal{L}_{\mathrm{DAgger}} is computed on the policy’s action-distribution parameters rather than on sampled actions: both the teacher MLP πI\pi_{I} and the student decoder pθp_{\theta} output the mean and log-standard-deviation of a tanh-squashed diagonal Gaussian over the action, and we take the MSE between these parameter vectors. This reduces to the action-matching form LDAgger=∥at−a^t∥22\mathcal{L}_{\mathrm{DAgger}} = \lVert a_{t} - \hat{a}_{t} \rVert_{2}^{2} given in the main text when both policies are evaluated deterministically (i.e., at their distribution means), and is a policy-distribution distillation in the spirit of DAgger [33] more generally. The KL coefficient λ\lambda is annealed linearly from 10−310^{-3} to 10−410^{-4} over the first 25% of training and held fixed thereafter; the AR(1) coefficient is fixed at α=5×10−3\alpha= 5 \times10^{-3}. To prevent posterior collapse and runaway sampling noise, the encoder and prior log-variances are clipped to [−5,0][-5, 0] (matching PULSE [13]).

Network architecture. All three networks are MLPs with SiLU activations and LayerNorm between hidden layers (matching the Stage 1 policy). The encoder maps the concatenation of xtx_{t} and gtg_{t} through hidden layers [2048,1024,1024][2048, 1024, 1024] to per-dimension means and log-variances of a 16-dimensional Gaussian latent ztz_{t}; we sample ztz_{t} via the standard reparameterization trick. The decoder maps the concatenation of ztz_{t} and xtx_{t} through [1024,1024,512][1024, 1024, 512] to an action prediction a^t\hat{a}_{t}. The proprioception-conditioned motion prior maps xtx_{t} through [1536,1024,512][1536, 1024, 512] to the mean and log-variance of pψ(zt∣xt)p_{\psi}(z_{t} \mid x_{t}). Encoder, decoder, and motion prior are randomly initialized using LeCun initialization with zero bias and trained end-to-end; the teacher πI\pi_{I} remains frozen throughout.

Training. We collect 4,096 parallel student rollouts in MuJoCo MJX with the same 20-step unroll length used in Stage 1, train for 10910^{9} environment steps with batch size 1,024 split across 16 minibatches and 4 epochs per batch, and update (ϕ,θ,ψ)(\phi, \theta, \psi) jointly with Adam at learning rate 10−410^{-4} and global gradient-norm clipping at 10.0. Observations are normalized by their running mean and variance, with statistics inherited from the Stage 1 normalizer to ensure consistency with the frozen teacher. Evaluation rollouts are run every 2×1072 \times10^{7} environment steps with the deterministic decoder mean. The complete list of distillation hyperparameters is given in Table 3.

Stage 3: Behavior prior distillation

Task. Behavior prior distillation reuses the imitation environment from Stage 1 (App. § 12.§ 1) for rollout collection, with domain randomization disabled. The Stage 2 latent skill controller—encoder qϕ(zt∣xt,gt)q_{\phi}(z_{t} \mid x_{t}, g_{t}), decoder pθ(at∣xt,zt)p_{\theta}(a_{t} \mid x_{t}, z_{t}), and motion prior pψ(zt∣xt)p_{\psi}(z_{t} \mid x_{t})—is loaded frozen and used to drive simulation: at each step we feed the encoder’s mean μte\mu_{t}^{e} as the latent into the frozen decoder and use the resulting deterministic action mean to advance the environment. The trainable behavior prior pφ(zt∣xt)p_{\varphi}(z_{t} \mid x_{t}) is optimized only on the proprioception states visited along these trajectories; loss evaluation queries the full encoder distribution (μte,σte)(\mu_{t}^{e}, \sigma_{t}^{e}) at each visited state, not a sample.

Loss. With the encoder qϕq_{\phi} held fixed, the behavior prior is trained by minimizing the expected KL divergence

Lbehavior=DKL(qϕ(zt∣xt,gt)∥pφ(zt∣xt)),\mathcal{L}_{\mathrm{behavior}} = D_{\mathrm{KL}}\bigl(q_{\phi}(z_{t} \mid x_{t}, g_{t}) \mathbin\Vert p_{\varphi}(z_{t} \mid x_{t})\bigr),

which by Proposition 2 is equivalent to matching the marginal posterior mϕ(zt∣xt)=Ep(gt∣xt)[qϕ(zt∣gt,xt)]m_{\phi}(z_{t} \mid x_{t}) = \mathbb{E}_{p(g_{t} \mid x_{t})}[q_{\phi}(z_{t} \mid g_{t}, x_{t})]. Crucially, the encoder is not modified at this stage; it provides a fixed target distribution, so the behavior prior absorbs the goal-conditional variation in qϕq_{\phi} rather than collapsing onto local motion continuation as the motion prior does (§ 3.3).

Network architecture. The behavior prior is an MLP with SiLU activations and LayerNorm between hidden layers, mirroring the motion prior. It maps xtx_{t} through hidden sizes [2048,2048,1024,512][2048, 2048, 1024, 512] to the mean and log-variance of a 16-dimensional Gaussian over ztz_{t}, matching the latent dimensionality of the frozen encoder. The behavior prior is the only trainable network in this stage; encoder, decoder, and motion prior all remain frozen. The behavior prior is initialized using LeCun initialization with zero bias.

Training. We collect 4,096 parallel rollouts in MuJoCo MJX with the same 20-step unroll length used in earlier stages, train for 2×1092 \times10^{9} environment steps with batch size 1,024 split across 16 minibatches and 4 epochs per batch, and update φ\varphi with Adam at learning rate 10−410^{-4} and global gradient-norm clipping at 10.0. Observation statistics are inherited from the Stage 2 normalizer and held fixed, so that pφp_{\varphi} sees the same input distribution as the frozen encoder. Evaluation rollouts are run every 2×1072 \times10^{7} environment steps using the deterministic decoder mean and a deterministic behavior-prior mean (with stochastic-sampling evaluations reported separately in the experiments). The complete list of behavior-prior hyperparameters is given in Table 4.

GroupHyperparameterValue
Distillation lossKL weight1.0
Distillation lossGradient clip threshold (global norm)10.0
Network architectureLatent size (inherited from Stage 2)16
Network architectureBehavior prior hidden sizes[2048,2048,1024,512][2048, 2048, 1024, 512]
TrainingParallel environments4,096
TrainingTotal environment steps2×1092 \times10^9
TrainingBatch size1,024
TrainingMinibatches per epoch16
TrainingUpdates per batch4
TrainingLearning rate (Adam)1×10−41 \times10^{-4}
TrainingUnroll length20
TrainingAction repeat1
TrainingTrain / eval clip split0.8/0.2
TrainingEvaluation / reset interval2×1072 \times10^7 steps

Table 4. Hyperparameters used for Stage 3 (behavior prior distillation) of the SCAMPER training pipeline. Environment settings are inherited from Stage 1 (Table 2); the encoder, decoder, and motion prior are loaded frozen from Stage 2 (Table 3). Only the behavior prior is updated.

Stage 4 (transfer learning): Arena foraging

Task. Stage 4 fine-tunes a task-specific policy on top of the frozen Stage 3 controller. The decoder pθp_{\theta}, motion prior pψp_{\psi}, and behavior prior pφp_{\varphi} are loaded frozen from the Stage 3 checkpoint (Table 4); only the task policy πT(δt,βt∣xt,ot)\pi_T(\delta_t, \beta_t \mid x_t, o_t) and its critic are trained. In the arena foraging task, the rat is placed at the center of a flat circular arena (Figure 11) and must navigate to and collect N=10N = 10 treats (blue spheres of radius 0.06 m at height 0.03 m). At the start of each episode the rat’s initial yaw is sampled uniformly in [0,2π)[0, 2\pi), and the NN treats are placed one per angular sector with sub-sector jitter, at uniformly random radii within an annulus of inner radius of variable size depending on the trial and outer radius rmax⁡=2.0r_{\max} = 2.0 m. Each treat emits an isotropic Gaussian scent field with bandwidth σ=0.2\sigma= 0.2 m. In addition to the proprioception vector xtx_t from § 3.1, the agent observes a task-specific oto_t comprising (i) the scalar total scent at the rat’s skull (sum of Gaussians from currently active treats) and (ii) the egocentric gradient ∇st∈R3\nabla s_t \in\mathbb{R}^{3} of the same scent field, which points toward the scent-weighted treat center.

Rendering of the arena foraging task

Figure 11. Rendering of the arena foraging task.

Reward. The task uses a sparse reward: collecting a treat (skull entering its collision sphere) yields a large, one-time reward and the treat with its scent contribution is removed for the remainder of the episode.

Termination. Each episode runs for up to 2,000 control steps (20 s at the 100 Hz control rate inherited from Stage 1), or terminates early once all NN treats have been collected.

Policy gating. The task policy outputs a 16-dimensional residual δt\delta_t to the motion prior’s mean and a scalar gating logit βt\beta_t. The binary gate btb_t is thresholded with bias 0.5 and hysteresis margin 0.2: switching btb_t to 1 requires βt>0.7\beta_t > 0.7, and switching to 0 requires βt<0.3\beta_t < 0.3. When bt=1b_t = 1, the latent is zt=μt±+δtz_t = \mu_t^{\pm} + \delta_t (motion-prior residual control). When bt=0b_t = 0, the latent is sampled stochastically from the behavior prior, zt=μtb+ϵσz_t = \mu_t^{b} + \epsilon\sigma with ϵ∼N(0,1)\epsilon\sim\mathcal{N}(0, 1) and a fixed log-variance of −2.25-2.25 (i.e., σ=0.32\sigma= 0.32), which yields random motions that fall within the range of the natural behavioral diversity of the training data (Figure 2b). The latent is passed through the frozen decoder pθp_{\theta} to produce the action torques.

Network architecture. The task policy is an MLP with hidden sizes [2048,1024,512][2048, 1024, 512] that maps the concatenation of xtx_t and oto_t (normalized by running mean and variance) to the mean and log-standard-deviation of a tanh-squashed diagonal Gaussian over the joint 17-dimensional output (δt,βt)(\delta_t, \beta_t), so that δt∈[−1,1]16\delta_t \in[-1, 1]^{16} and βt∈[−1,1]\beta_t \in[-1, 1]. The critic is a separate [2048,1024,1024,512][2048, 1024, 1024, 512] MLP estimating the state-value function. Both networks use SiLU activations with LayerNorm between hidden layers and are randomly initialized using LeCun initialization with zero bias. The decoder, motion prior, and behavior prior all remain frozen throughout this stage.

Training. We train πT\pi_T with PPO [32] for 2×1092 \times10^{9} environment steps across 1,024 parallel environments. Each PPO update uses a 20-step unroll, batch size 1,024, and 4 epochs of 16 minibatches per batch, with Adam at learning rate 10−410^{-4} and gradient clipping at maximum norm 50.0. We use discount γ=0.99\gamma= 0.99, GAE λ=0.95\lambda= 0.95, PPO clip range ε=0.3\varepsilon= 0.3, entropy coefficient 10−210^{-2}, value-loss coefficient 0.5, and reward scaling 1.0. Evaluation rollouts are performed every 1.5×1071.5 \times10^{7} environment steps. The complete list of hyperparameters is given in Table 5.

GroupHyperparameterValue
Task / environmentEpisode length2,000 control steps
Task / environmentEvaluation interval1.5×1071.5 \times10^{7} env steps
Task / environmentNumber of treats per episode10
Task / environmentTreat sphere radius0.06 m
Task / environmentTreat height0.03 m
Task / environmentMinimum spawn radiusVariable
Task / environmentMaximum spawn radius2.0 m
Task / environmentScent kernel σ\sigma0.2 m
Task / environmentTreat-collection reward500.0
Behavior–motion prior gatingGate threshold bbiasb_{\mathrm{bias}}0.5
Behavior–motion prior gatingGate hysteresis margin0.2
Behavior–motion prior gatingBehavior-prior fixed log-variance−2.25-2.25
Network architectureTask policy hidden sizes[2048, 1024, 512]
Network architectureCritic hidden sizes[2048, 1024, 1024, 512]
PPO trainingParallel environments1,024
PPO trainingTotal environment steps2×1092 \times10^{9}
PPO trainingBatch size1,024
PPO trainingMinibatches per epoch16
PPO trainingUpdates per batch4
PPO trainingLearning rate (Adam)1×10−41 \times10^{-4}
PPO trainingGradient clip threshold (max norm)50.0
PPO trainingPPO clip range ϵ\epsilon0.3
PPO trainingDiscount factor γ\gamma0.99
PPO trainingGAE λ\lambda0.95
PPO trainingEntropy coefficient1×10−21 \times10^{-2}
PPO trainingValue-loss coefficient0.5
PPO trainingReward scaling1.0
PPO trainingUnroll length20
PPO trainingAction repeat1

Table 5. Hyperparameters used to train SCAMPER on the arena foraging task (Stage 4: transfer learning). The decoder, motion prior, and behavior prior are loaded frozen from the Stage 3 checkpoint (Table 4); only the task policy and critic are trained at this stage.

Locomotion

Task. The locomotion task assesses how effectively the motion prior pψp_{\psi} supports goal-directed control via residual fine-tuning, without recourse to the behavior prior. The motion prior pψp_{\psi} and decoder pθp_{\theta} are loaded frozen from the Stage 2 checkpoint (Table 3); only the task policy and critic are trained. We do not use the behavior prior in this task: we use bt=1b_t = 1 at every step, so that the latent is always zt=μtm+δtz_t = \mu_t^{m} + \delta_t, isolating the quality of residual control of the motion prior. At the start of each episode the rat is placed at the world origin in a neutral standing pose facing the world +x+x direction (identity body quaternion, z=0z = 0), on a flat arena. The task is to maintain a fixed target forward speed v⋆v^{\star} along the world +x+x axis. In addition to the proprioception vector xtx_t from § 3.1, the agent receives a task observation oto_t comprising the previous action, and the rat’s kinematic and touch sensors. The target speed v⋆v^{\star} itself is a fixed task parameter and is not part of the observation.

Reward. The reward at each step is a piecewise-linear forward-velocity tracking signal,

rt=max⁡(0,1−∣vtx−v⋆∣/v⋆),r_t = \max(0, 1 - |v_t^x - v^{\star}|/v^{\star}),

where vtxv_t^x is the world-frame +x+x component of the torso’s linear velocity (subtree center-of-mass velocity of the torso body). The reward equals 1 when vtx=v⋆v_t^x = v^{\star} and decays linearly to 0 at vtx=0v_t^x = 0 and at vtx=2v⋆v_t^x = 2v^{\star}.

Termination. An episode terminates early if (i) the rat has fallen—defined as torso zz below 0.03 m or torso tilt from vertical exceeding 60°—or (ii) any state value becomes NaN. Episodes are otherwise truncated at a fixed length of 1,000 control steps (10 s at the 100 Hz control rate inherited from Stage 1).

Network architecture. The task policy is an MLP with hidden sizes [2048, 1024, 512] that maps the concatenation of xtx_t and oto_t (normalized by running mean and variance) to the mean and log-standard-deviation of a tanh-squashed diagonal Gaussian over the residual, so that δt∈[−1,1]16\delta_t \in[-1, 1]^{16}. The critic is a separate [2048, 1024, 1024, 512] MLP estimating the state-value function. Both networks use SiLU activations with LayerNorm between hidden layers and are randomly initialized using LeCun initialization with zero bias. The motion prior pψp_{\psi} and decoder pθp_{\theta} remain frozen throughout this task; the encoder qϕq_{\phi} is not used.

Training. We train the task policy with PPO [32] for 4×1084 \times10^{8} environment steps across 1,024 parallel environments. Each PPO update uses a 20-step unroll, batch size 1,024, and 4 epochs of 16 minibatches per batch, with Adam at learning rate 10−410^{-4} and gradient clipping at maximum norm 50.0. We use discount γ=0.99\gamma= 0.99, GAE λ=0.95\lambda= 0.95, PPO clip range ϵ=0.3\epsilon= 0.3, entropy coefficient 10−210^{-2}, value-loss coefficient 0.5, and reward scaling 1.0. Evaluation rollouts are performed every 1.5×1071.5 \times10^{7} environment steps. The complete list of hyperparameters is given in Table 6.

Table 6: Hyperparameters used to train SCAMPER on the locomotion task. The motion prior pψp_{\psi} and decoder pθp_{\theta} are loaded frozen from the Stage 2 checkpoint (Table 3); only the task policy and critic are trained at this stage. The policy outputs a residual δt\delta_t added to the (deterministic) motion-prior mean.

GroupHyperparameterValue
Task / environmentEpisode length1,000 control steps
Task / environmentEvaluation interval1.5×1071.5 \times10^{7} env steps
Task / environmentTarget forward speedVariable
Task / environmentForward-velocity reward weight1.0
Network architectureTask policy hidden sizes[2048, 1024, 512]
Network architectureCritic hidden sizes[2048, 1024, 1024, 512]
PPO trainingParallel environments1,024
PPO trainingTotal environment steps4×1084 \times10^{8}
PPO trainingBatch size1,024
PPO trainingMinibatches per epoch16
PPO trainingUpdates per batch4
PPO trainingLearning rate (Adam)1×10−41 \times10^{-4}
PPO trainingGradient clip threshold (max norm)50.0
PPO trainingPPO clip range ε\varepsilon0.3
PPO trainingDiscount factor γ\gamma0.99
PPO trainingGAE λ\lambda0.95
PPO trainingEntropy coefficient1×10−21 \times10^{-2}
PPO trainingValue-loss coefficient0.5
PPO trainingReward scaling1.0
PPO trainingUnroll length20
PPO trainingAction repeat1

Table 6.

Rearing

Task. The walk-then-rear task tests whether residual control of the motion prior can drive a navigate-and-rear sequence terminated by a sparse success bonus. As in the locomotion task, the motion prior pψp_{\psi} and decoder pθp_{\theta} are loaded frozen from the Stage 2 checkpoint (Table 3); only the task policy and critic are trained, and we use bt=1b_t = 1 at every step so that the latent is always zt=μtm+δtz_t = \mu_t^m + \delta_t. At the start of each episode the rat is placed at the world origin in a neutral standing pose facing the world +x+x direction (identity body quaternion, z=0z = 0), on a flat arena; a translucent target box is rendered at the goal location for visualization. The agent must walk to the horizontal target (x⋆,y⋆)=(1.0,0.0)(x^{\star}, y^{\star}) = (1.0, 0.0) m and rear up so that its skull reaches a target height z⋆=0.2z^{\star} = 0.2 m. A target pose is achieved when the skull is simultaneously within 0.01 m of (x⋆,y⋆)(x^{\star}, y^{\star}) horizontally and within 0.02 m of z⋆z^{\star} vertically; success requires holding the target pose continuously for 1.0 s (100 control steps at 100 Hz). In addition to the proprioception vector xtx_t from § 3.1, the agent receives a task observation oto_t comprising the previous action, the rat’s kinematic and touch sensors, the egocentric position of the world origin in the torso frame, and the egocentric position of the target in the torso frame.

Reward. The reward at each step combines a dense piecewise-linear shaping term and a sparse terminal bonus. Letting dt=∥ptskull−p⋆∥2d_t = \lVert\mathbf{p}^{\mathrm{skull}}_t - \mathbf{p}^{\star} \rVert_2 denote the Euclidean distance from the skull to the target, the dense term is

rtdist=max⁡(0,1−dt/m),r_t^{\mathrm{dist}} = \max(0, 1 - d_t/m),

with shaping margin m=1.0m = 1.0 m, so it equals 1 when the skull is at the target and decays linearly to 0 at dt=md_t = m. The sparse bonus is rtsucc=500r_t^{\mathrm{succ}} = 500 fires on the single step at which the running hold counter first reaches 100 steps, and is 0 otherwise.

Termination. An episode terminates if (i) the hold counter reaches 100 steps (success), (ii) the rat has fallen—defined as torso zz below 0.02 m or torso tilt from vertical exceeding 80°—or (iii) any state value becomes NaN. Episodes are otherwise truncated at a fixed length of 500 control steps (5 s at the 100 Hz control rate inherited from Stage 1).

Network architecture. The task policy is an MLP with hidden sizes [2048, 1024, 512] that maps the concatenation of xtx_t and oto_t (normalized by running mean and variance) to the mean and log-standard-deviation of a tanh-squashed diagonal Gaussian over the residual, so that δt∈[−1,1]16\delta_t \in[-1, 1]^{16}. The critic is a separate [2048,1024,1024,512][2048, 1024, 1024, 512] MLP estimating the state-value function. Both networks use SiLU activations with LayerNorm between hidden layers and are randomly initialized using LeCun initialization with zero bias. The motion prior pψp_{\psi} and decoder pθp_{\theta} remain frozen throughout this task; the encoder qϕq_{\phi} is not used.

Training. We train the task policy with PPO [32] for 6×1086 \times10^{8} environment steps across 1,024 parallel environments. Each PPO update uses a 20-step unroll, batch size 1,024, and 4 epochs of 16 minibatches per batch, with Adam at learning rate 5×10−55 \times10^{-5} and gradient clipping at maximum norm 50.0. We use discount γ=0.99\gamma= 0.99, GAE λ=0.95\lambda= 0.95, PPO clip range ε=0.3\varepsilon= 0.3, entropy coefficient 10−210^{-2}, value-loss coefficient 0.5, and reward scaling 1.0. Evaluation rollouts are performed every 1.5×1071.5 \times10^{7} environment steps. The complete list of hyperparameters is given in Table 7.

Table 7: Hyperparameters used to train SCAMPER on the walk-rear task. The motion prior pψp_{\psi} and decoder pθp_{\theta} are loaded frozen from the Stage 2 checkpoint (Table 3); only the task policy and critic are trained. The policy outputs a residual δt\delta_t added to the (deterministic) motion-prior mean.

GroupHyperparameterValue
Task / environmentEpisode length500 control steps
Task / environmentEvaluation interval1.5×1071.5 \times10^{7} env steps
Task / environmentTarget position (x⋆,y⋆,z⋆)(x^{\star}, y^{\star}, z^{\star})(1.0, 0.0, 0.2) m
Task / environmentxyxy-tolerance for success0.01 m
Task / environmentzz-tolerance for success0.02 m
Task / environmentRequired hold duration1.0 s
Task / environmentDistance-shaping margin1.0 m
Task / environmentDistance-to-target reward weight1.0
Task / environmentWalk-rear success reward500.0
Network architectureTask policy hidden sizes[2048, 1024, 512]
Network architectureCritic hidden sizes[2048, 1024, 1024, 512]
PPO trainingParallel environments1,024
PPO trainingTotal environment steps6×1086 \times10^{8}
PPO trainingBatch size1,024
PPO trainingMinibatches per epoch16
PPO trainingUpdates per batch4
PPO trainingLearning rate (Adam)5×10−55 \times10^{-5}
PPO trainingGradient clip threshold (max norm)50.0
PPO trainingPPO clip range ε\varepsilon0.3
PPO trainingDiscount factor γ\gamma0.99
PPO trainingGAE λ\lambda0.95
PPO trainingEntropy coefficient1×10−21 \times10^{-2}
PPO trainingValue-loss coefficient0.5
PPO trainingReward scaling1.0
PPO trainingUnroll length20
PPO trainingAction repeat1

Table 7.

Hyperparameters

Our hyperparameter choices draw on prior work where applicable, with task-specific tuning otherwise. PPO hyperparameters follow Brax’s defaults [47] with minor adjustments to maximize task reward. Physics simulation and virtual rodent parameters are taken from MIMIC-MJX [6], and online distillation hyperparameters follow PULSE [13] with small modifications identified via constrained grid search to better suit the virtual rodent setting. All remaining hyperparameters were selected through grid search.

Analysis of randomly generated behavior

Rollout collection protocol

All quantitative comparisons in this section (KID, Vendi score, root-height ECDF, spatial occupancy) are computed on rollouts collected with a single shared protocol so that the only difference between conditions is the controller that produced the trajectory.

For every condition, we start from the same set of initial states: the first frame of each of the 842 reference clips, with the rat placed in the corresponding generalized pose and zero joint velocity. This matches the initialization used during imitation training and keeps the starting distribution of body configurations identical across conditions. Each rollout is then advanced in MuJoCo MJX under the chosen controller for the same horizon as the reference clip (∼5\sim5 s of simulated time, with control at 100 Hz on top of a 500 Hz physics step); the trajectory is driven purely by stochastic sampling of the controllers. At each step we log the rat’s generalized coordinates q∈R74\mathbf{q} \in\mathbb{R}^{74} and the latent vector passed to the decoder. Clips are processed in parallel batches on a single GPU, and one randomization key is split per condition so that stochastic samples are reproducible and independent across conditions. We perform the rollouts using both the behavior prior and the motion prior and compare to the imitation rollouts as the ground truth. We test across both the deterministic sampling condition (no noise; σ=0\sigma= 0) and an array of different sampling noise levels logvar⁡={−4,−3,−2,−1,−0.5,0}\operatorname{logvar} = \{-4, -3, -2, -1, -0.5, 0\} (σ≈{0.14,0.22,0.37,0.61,0.78,1.00}\sigma\approx\{0.14, 0.22, 0.37, 0.61, 0.78, 1.00\}).

Vendi score

What the Vendi score measures. The Vendi score [34] quantifies the effective number of distinct modes in a sample under a user-specified similarity kernel. It is defined as the exponential of the von Neumann (Shannon) entropy of the eigenvalues of the normalized similarity matrix and, in contrast to a clustering-based count, requires no reference distribution, no labels, and no choice of cluster number. It reduces to the number of items when they are mutually orthogonal and to 1 when they are all identical. Higher Vendi score indicates greater behavioral diversity.

Egocentric kinematic features. Unlike for KID, where the feature extractor is a learned VAE, we measure diversity directly in an egocentric kinematic feature space so that the score is interpretable in terms of physically meaningful movement variation rather than the geometry of a learned latent space. From each rollout’s generalized coordinates q∈R74\mathbf{q} \in\mathbb{R}^{74}, we compute, per frame, the root height, root roll and pitch, body-frame linear velocity, yaw rate, and the joint angles — 74 dims/frame. The body-frame velocities are intrinsically heading-invariant and the planar root translation is dropped, so the representation is invariant to the absolute location of the rat in the arena. Frames are then stacked into sliding windows of length 30 (0.6 s at the 50 Hz control rate) with a stride of 15, yielding 2,220-dimensional feature vectors per window.

Joint PCA(50) embedding. To make the Vendi computation tractable and to apply a consistent similarity geometry across conditions, we fit a single PCA basis of dimension 50. The basis is fit jointly on a stratified subsample of 50,000 windows, drawn proportionally across every condition (each prior at every noise level, plus the training-data baseline) so that no condition dominates the principal directions. All windows are then projected into this shared 50-dimensional embedding before the Vendi score is computed.

Computing the Vendi score. Let X∈Rn×50X \in\mathbb{R}^{n \times50} be the PCA-embedded windows of a single rollout sample, and let XnX_n denote XX with rows ℓ2\ell_2-normalized. We use the cosine similarity kernel and exploit the dual identity eigs⁡(XnXn⊤/n)=eig⁡(Xn⊤Xn/n)\operatorname{eigs}(X_n X_n^\top/n) = \operatorname{eig}(X_n^\top X_n/n) (up to zero eigenvalues) to compute Vendi via the small 50×5050 \times50 Gram matrix,

M=1nXn⊤Xn,{λi}i=150=eigvalsh⁡(M).M = \frac{1}{n} X_n^\top X_n,\qquad\{\lambda_i\}_{i=1}^{50} = \operatorname{eigvalsh}(M).

which is exact for the cosine Vendi score and avoids forming the n×nn \times n kernel. The Vendi score is then the exponential of the Shannon entropy of the (positive) eigenvalues,

VS⁡(X)=exp⁡(−∑i:λi>0λilog⁡λi).\operatorname{VS}(X) = \exp\left(-\sum_{i:\lambda_i>0} \lambda_i \log\lambda_i\right).

with the convention 0log⁡0=00 \log0 = 0.

Bootstrap over clips. The independent unit of replication is the imitation clip (windows within a clip share a starting condition and are not independent), so we bootstrap at the clip level. There are 842 clips per condition (motion-prior, behavior-prior, training-data). For each condition we bootstrap by resampling the clip indices with replacement and recompute the Vendi score. We perform B=500B = 500 bootstrap iterations. The same clip indices are reused across all conditions in a given iteration, so cross-policy comparisons are exactly paired. We report bootstrap means with 95% confidence intervals.

Kernel Inception Distance

What KID measures. The Kernel Inception Distance (KID) quantifies the dissimilarity between two distributions in a learned feature space as the squared Maximum Mean Discrepancy (MMD2^2) under a polynomial kernel [37]. In contrast to a metric like Fréchet Inception Distance (FID) [49], it is an unbiased estimator and remains well-defined for moderately sized samples. KID is zero when the two distributions match.

Training the feature extractor. There is no pre-trained “Inception” network for rodent kinematics, so we train a small variational autoencoder (VAE) on the real (training-data) joint-angle trajectories and use its encoder mean μϕ(x)∈Rd\mu_\phi(x) \in\mathbb{R}^d as the feature extractor. The VAE is optimized by the standard β\beta-VAE objective

LVAE(ϕ,θ)=Ex[∥x^θ(x)−x∥22+βDKL(qϕ(z∣x)∥N(0,I))].\mathcal{L}_{\mathrm{VAE}}(\phi,\theta) = \mathbb{E}_{x}\left[\lVert\hat{x}_{\theta}(x) - x\rVert_{2}^{2} + \beta D_{\mathrm{KL}}\left(q_{\phi}(z \mid x) \mathbin\Vert\mathcal{N}(0,I)\right)\right].

with hyperparameters listed in Table 8. After training, the decoder is discarded and only μϕ\mu_\phi is retained; we repeat training under several random seeds and report the mean and standard deviation of KID across them.

GroupHyperparameterValue
ArchitectureEncoder hidden sizes[512, 256]
ArchitectureDecoder hidden sizes[256, 512]
ArchitectureLatent dimensionality dd64
ArchitectureActivationSiLU
ArchitectureLayer normalizationyes
RegularizationDropout rate0.1
RegularizationWeight decay (AdamW)1×10−51 \times10^{-5}
β\beta-VAETarget β\beta1.0
β\beta-VAEβ\beta warmup (linear)2,500 epochs
TrainingOptimizerAdamW
TrainingLearning rate1×10−31 \times10^{-3}
TrainingGradient clip norm1.0
TrainingBatch size128
TrainingTotal epochs5,000
TrainingRandom seeds{0,1,2,3}\{0, 1, 2, 3\}
KID computationPolynomial kernel degree3
KID computationNumber of subsets SS100
KID computationSubset size mmmin⁡(∣X∣,∣Y∣,1,000)\min(|X|, |Y|, 1{,}000)

Table 8. Hyperparameters for the VAE feature extractor (“inception network”) used to compute KID.

Trajectory inputs and preprocessing. Each training sample is a single 5 s trajectory of the rat’s generalized coordinates q∈R74\mathbf{q} \in\mathbb{R}^{74} — planar root position (x,y)(x,y), root height zz, root quaternion (4 dims), and 67 joint angles — drawn from the training-data imitation rollouts. We use imitation rollouts as the ground truth as opposed to the original mocap because the alignment process from keypoints to joint angle trajectories (through STAC-MJX [6]) produces jagged transition frame-to-frame. Imitation of this reference allows the simulated physics to smooth out the trajectories and results in a fairer comparison to the generated motion. Before encoding, three operations are applied so that the comparison between training and generated distributions is invariant to nuisance factors and on a consistent scale: (i) the root quaternion is replaced by a yaw-removed 6D rotation representation [50]; (ii) the planar root position (x,y)(x,y) is discarded; (iii) joint dimensions are zz-score normalized using statistics computed on the real data. Each per-step vector therefore consists of root height, root rotation, and 67 normalized joint angles for 74 features per step; the full trajectory is flattened to a single vector before being fed to the encoder. The same preprocessing is applied to every generated rollout.

Computing KID. Let X={μir}i=1mX = \{\mu_i^r\}_{i=1}^{m} and Y={μjf}j=1mY = \{\mu_j^f\}_{j=1}^{m} be the encoder features of the real and generated samples. We use the cubic polynomial kernel of Bińkowski et al. [37],

k(u,v)=(1duTv+1)3,k(\mathbf{u},\mathbf{v}) = \left(\frac{1}{d}\mathbf{u}^{\mathsf{T}}\mathbf{v} + 1\right)^3,

and define MMD^u2\widehat{\mathrm{MMD}}_u^2 as the unbiased squared-MMD estimator of Gretton et al. [51],

MMD^u2(X,Y)=1m(m−1)∑i≠jk(μir,μjr)+1m(m−1)∑i≠jk(μif,μjf)−2m2∑i,jk(μir,μjf).\widehat{\mathrm{MMD}}_u^2(X,Y) = \frac{1}{m(m-1)}\sum_{i\ne j} k(\mu_i^r,\mu_j^r) + \frac{1}{m(m-1)}\sum_{i\ne j} k(\mu_i^f,\mu_j^f) - \frac{2}{m^2}\sum_{i,j} k(\mu_i^r,\mu_j^f).

To stabilize the per-seed estimate, we average MMD^u2\widehat{\mathrm{MMD}}_u^2 over SS random subsets of size mm drawn independently from XX and YY:

KID⁡(X,Y)=1S∑s=1SMMD^u2(X(s),Y(s)).\operatorname{KID}(X,Y) = \frac{1}{S}\sum_{s=1}^{S}\widehat{\mathrm{MMD}}_u^2\left(X^{(s)},Y^{(s)}\right).

Gait analysis

We compare the gaits produced by SCAMPER, MIMIC-MJX, and the from-scratch baseline against empirical rat gait data from Danner et al. [39], at three target speeds spanning walk, trot, and gallop. The same pipeline is applied to every model rollout so that the only difference between conditions is the controller.

Rollouts. For each (model, target speed) condition we run a fixed-length deterministic rollout of the locomotion task, with the velocity setpoint chosen to elicit the corresponding gait. The simulator state, the foot-body world positions, and the four limb touch-sensor traces (palm and sole on each side) are saved for offline analysis.

Contact barcode. For each rollout we convert the four raw touch-sensor traces into a binary stance/swing barcode of shape [4,T][4, T], with rows ordered [RH, RF, LF, LH]. The pipeline is: (i) per-channel median filtering to suppress sensor noise; (ii) an optional spike-removal step that zeros out brief above-threshold transients on a sensor channel that is otherwise quiescent (so that fleeting contacts never get registered as stance bouts); (iii) per-channel thresholding to a binary stance signal, with the threshold tuned per limb to account for differing typical contact forces; and (iv) binary morphological closing on each channel to fill brief gaps in otherwise-continuous stance bouts. The four cleaned binary traces are then stacked into the barcode used for all subsequent steps.

Gait cycle detection. Gait cycles are extracted from the barcode using a chosen reference limb. We find every stance onset (a 0→10 \to1 transition on the reference channel), discard onsets that are closer together than a minimum cycle length (rejects double-detection of a single foot-strike), and define each gait cycle as the interval between two consecutive onsets that is at least the minimum cycle length. Detection is restricted to a per-condition analysis window so that the rat is at steady-state speed during the cycles being analyzed.

Per-cycle metrics. For every detected cycle we compute the standard set of behavioral-gait metrics used in rodent locomotion studies [40, 41]:

  • Stride length. Euclidean distance between the reference foot’s world position at the start and end of the cycle.

  • Mean velocity. Stride length divided by cycle duration.

  • Cadence. Stride frequency converted to steps per minute (counting both feet of a contralateral pair).

  • Stance percentage / duty factor. Fraction of the cycle during which the reference limb is in stance.

We then aggregate these metrics across all cycles in the analysis window and report mean and standard deviation per (model, gait) condition.

Swing–stance gait diagrams. The diagrams in Figure 4c are computed by isolating, on each of the four limb channels, every complete stance bout within the analysis window (partial bouts that overlap the window edges are dropped), then computing per-limb mean stance duration and standard deviation, mean stride period (median inter-onset interval averaged over limbs), and the phase offset of each limb relative to a reference hind limb. Phase offsets are taken as the mean delay between the first stance onset of each limb after a reference-limb onset, modulo the stride period. The resulting per-limb stance bars are tiled across the cycle for visualization.

Empirical reference. The empirical rat data [39] are filtered to intact-spinal-cord animals for each of the three gait categories, and we extract per-stride duty factors for the four limbs, the inter-limb phase relationships (left–right hind, homolateral, and diagonal), and the stride duration. These are converted into the same set of per-limb mean stance, phase offset, mean stride period, and duty-factor summaries used for the model rollouts, providing a directly comparable reference distribution.

Quantitative summary. For the comparison table, every metric is summarized per (gait, model) as μ±σ\mu\pm\sigma over cycles. Each model is then scored against the empirical reference as the difference in means normalized by the empirical standard deviation, ∣Δμ∣/σemp|\Delta\mu|/\sigma_{\mathrm{emp}}, which expresses the discrepancy in units of the natural across-stride variability of real rats. The metrics included are stride duration, step frequency, per-limb stance and swing duration, per-limb duty factor, temporal symmetry (left/right hind duty-factor asymmetry), and per-limb phase offset — giving a single table that aggregates the full per-limb timing pattern alongside the cycle-level summaries.

Supplementary analyses and results

Empirical tests of Propositions 1 and 2

To evaluate the encoder and the two priors on the states they actually encounter, we rolled out the reference dataset in closed loop through the frozen Stage-2 encoder–decoder and collected ∼200,000\sim200{,}000 proprioceptive states. From these we drew 20,000 anchor states. For each anchor we identified the 32 most similar proprioceptive states in a 64-dimensional principal-component space, excluding the anchor’s own clip to avoid autocorrelation, and took the goal that followed each of them as a candidate goal. Each anchor’s proprioception was then held fixed while its goal was swapped for each of these K=32K=32 candidates.

Both propositions rest on the identity E[DKL(qϕ∥p)]=Iϕ(Z;G∣X)+E[DKL(mϕ∥p)]\mathbb{E}\left[D_{\mathrm{KL}}(q_{\phi}\mathbin\Vert p)\right]=I_{\phi}(Z;G\mid X)+\mathbb{E}\left[D_{\mathrm{KL}}(m_{\phi}\mathbin\Vert p)\right], where mϕm_{\phi} is the encoder marginal. The first term depends only on the encoder, so it is identical for both priors and cancels in their difference. The comparison between priors therefore reduces to closed-form Gaussian KL divergences. The behavior prior pφp_{\varphi} is 8.46 nats closer to the encoder marginal than the motion prior pψp_{\psi} averaged across all individual states (Figure 7a).

Figure 7. (a) KL divergence to the encoder marginal for the motion and behavior priors. (b) Between-goal and within-goal shares of the encoder’s latent variance at fixed proprioception. (c) Goal decoding accuracy from the latent, compared with chance. (d) Goal sensitivity across Stage-2 training, with the KL coefficient λ\lambda on the right axis. Error bars are 95% clip-level bootstrap CIs.

To measure how much of the encoder’s behavior at a fixed body state is driven by the goal, we decompose the encoder’s latent variance at each anchor within the KK candidates into the law of total variance. The between-goal term is the variance of the encoder mean across the 32 candidate goals, Var⁡g[μϕ]\operatorname{Var}_{g}[\mu_{\phi}], and the within-goal term is the expected encoder variance, Eg[σϕ2]\mathbb{E}_{g}[\sigma_{\phi}^{2}]. With proprioception held fixed, 49.7% of the encoder’s latent variance is between-goal (Figure 7b). The regularization in Proposition 1 therefore limits the conditional mutual information Iϕ(Z;G∣X)I_{\phi}(Z;G\mid X) but does not drive it to zero. Driving it to zero would be undesirable, because the behavior prior distilled from the encoder could then no longer produce diverse motion.

Because all 32 candidates at an anchor share the same proprioceptive state, any above-chance decoding of the goal reflects information carried by the latent alone. We draw one candidate uniformly at random, sample a latent from the encoder’s distribution for that candidate, score all 32 candidates by the density their encoder Gaussians assign to the sample, and predict the highest-scoring one. Since the candidate prior is uniform and the densities are the encoder’s own, this rule is Bayes-optimal and needs no trained probe. It identifies the correct goal 69.4% of the time, against 3.1% chance (a 22-fold improvement), and ranks it in the top 5 for 94.6% of anchors, against 15.6% chance (Figure 7c).

Finally, we tested Proposition 1 directly by repeating the fixed-proprioception analysis on all 50 Stage-2 checkpoints, each with its own observation normalizer. We measure goal sensitivity as the mutual information between the latent and which of the KK candidate goals was used, which is bounded above by log⁡K\log K. The Stage-2 KL coefficient λ\lambda ramps linearly down from 10−310^{-3} to 10−410^{-4} over the first 25% of training and is then held at its floor. Goal sensitivity tracks this schedule. It falls from 1.70 to 1.33 nats while λ\lambda is near its maximum, which is the bottleneck Proposition 1 describes. It then rises monotonically to 2.52 nats as λ\lambda decays and stops changing when λ\lambda reaches its floor, varying by less than 0.03 nats over the remaining 75% of training (Figure 7d). Nothing else in the training schedule changes at that point, so the strength of the bottleneck, rather than generic training dynamics, sets how much goal information the latent carries. Proposition 1 should therefore be read as describing a pressure that acts during training rather than a property of the final model: the schedule deliberately relaxes the bottleneck to improve reconstruction, and the converged encoder is goal-rich.

Diversity and realism of prior sampling

In Figure 2 we demonstrate that when sampling random motion trajectories, we obtain greater behavioral diversity from the behavior prior than from the motion prior quantified by the resulting Vendi scores. However, the observed difference in diversity could reflect discrepancies in effective perturbation magnitude of the latent rather than qualitatively different distributions. To address this, we compare the behavioral diversity by examining the impact of noise as a function of both the scales and the jitter of the sampled latents. To disambiguate whether the scale of the latents is in fact responsible for the difference in diversity, we measure the Vendi score against the mean per-dimension standard deviation of the sampled latents for both priors across all rollouts at various sampling noise levels (Figure 8a). While we do find that the scale of the latents is different for the two priors, when accounting for this along the domain of the measured latent variance, the behavior prior is still more diverse than the motion prior. Secondly, to disambiguate whether the behavior prior’s increased diversity is a result of jitter (i.e., how quickly the latents change from timestep to timestep), we measure the Vendi score versus the one-step latent displacement (Figure 8b). While we again find that both priors tend towards varying rates of per-step latent displacement for the same sampled noise level, when accounting for this difference, the behavior prior remains more behaviorally diverse than the motion prior over the domain of the latent jitter measured.

Figure 8. Diversity at matched latent scale. Vendi score of random generations vs. the realized per-dimension latent s.d. (a) and mean one-step latent displacement (b) at various levels of sampling noise (σ\sigma). Stars: deterministic (σ=0\sigma= 0). 95% bootstrap CIs shown as shaded regions.

Foraging task

Arena foraging supplemental results

Alternatives to the adaptive binary gate

The adaptive binary gate is one way to decide when the behavior prior or the motion prior controls the latent. To test whether the specific mechanism matters, we replaced it in the foraging task with two alternatives that need no learned gate: a fixed exploration schedule and a fixed linear interpolation between the two priors. In both variants the policy outputs only the residual, and the motion prior, behavior prior and decoder stay frozen, as with the adaptive gate.

Fixed exploration schedule. The latent is constructed exactly as with the adaptive gate, zt=gt(μtMP+δt)+(1−gt)ztBPz_t = g_t(\mu_t^{\mathrm{MP}} + \delta_t) + (1-g_t)z_t^{\mathrm{BP}}, where μtMP\mu_t^{\mathrm{MP}} is the motion prior mean, δt\delta_t is the policy’s residual and ztBPz_t^{\mathrm{BP}} is a behavior prior sample. However, the binary gate gtg_t is no longer emitted by the policy. Instead, it switches stochastically, and the probability p(gt=1)p(g_t = 1) of motion-prior control follows a fixed schedule over training. It is held at 0 for an initial warmup fraction of training, ramped to 1 over a subsequent ramp fraction, and held at 1 thereafter. We vary when this switch happens.

Linear interpolation. Instead of selecting one prior at each step, the two priors are blended with a constant weight ρ∈[0,1]\rho\in[0, 1] and the residual is added on top: zt=ρμtMP+(1−ρ)ztBP+rtz_t = \rho\mu_t^{\mathrm{MP}} + (1-\rho)z_t^{\mathrm{BP}} + r_t. Here ρ\rho is a fixed hyperparameter rather than a learned quantity. The endpoints ρ=1\rho= 1 and ρ=0\rho= 0 reduce to residual control of the motion prior alone and of the behavior prior alone, respectively.

Results. Table 10 reports the mean number of treats collected in the hardest foraging condition (minimum treat radius of 1.5 m). Both alternatives can succeed, but only within a narrow range of settings. The fixed schedule trains successfully when control passes from the behavior prior to the motion prior early in training (warmup 0.05, ramp 0.25, so that the motion prior has full control by 30% of training). It fails when the switch happens in the middle or later phases of training, and training also fails if the switch never happens. Likewise, interpolation succeeds only when the blend is weighted towards the behavior prior (ρ=0.25\rho= 0.25) and fails for ρ=0.5\rho= 0.5 and ρ=0.75\rho= 0.75.

VariantSettingMean treats collected
Fixed schedule (warmup / ramp)0.05 / 0.259.94 ± 0.03
Fixed schedule (warmup / ramp)0.10 / 0.600.00 ± 0.00
Fixed schedule (warmup / ramp)0.25 / 0.650.00 ± 0.00
Linear interpolation (ρ\rho)0.259.91 ± 0.04
Linear interpolation (ρ\rho)0.500.00 ± 0.00
Linear interpolation (ρ\rho)0.750.00 ± 0.00

Table 10. Alternatives to the adaptive binary gate in the foraging task. Values are the mean number of treats collected in the hardest condition (minimum treat radius 1.5 m). Schedule fractions are of total training steps. α\alpha is the weight on the motion prior.

These results show that what matters is the right mix of the behavior and motion priors over training. The adaptive binary gate is one means of achieving that mix, not the only one. Its advantage over the alternatives is that the user does not need to hand-tune a schedule or blend weight: the gate only needs to be biased towards the behavior prior at the start of training, and the transition to the motion prior then emerges from learning (Figure 12).

Locomotion task

Here we present the full gait analysis results for the empirical rat gait data [39], the main SCAMPER model which residually controls the motion prior (MP), a control which residually controls the behavior prior (BP), MIMIC-MJX, and the from-scratch trained model. Notably, when performing residual control using the behavior prior, galloping is never discovered and the slower speed gaits also lack resemblance to the empirical data both qualitatively (Figure 13) and quantitatively (Table 11).

References

  1. [1]Melanie Segado, Felipe Parodi, Jordan K. Matelsky, Michael L. Platt, Eva B. Dyer, and Konrad P. Kording. Grounding Intelligence in Movement, July 2025. URL http://arxiv.org/abs/2507.02771. arXiv:2507.02771 [cs].
  2. [2]Anthony Zador, Sean Escola, Blake Richards, Bence Ölveczky, Yoshua Bengio, Kwabena Boahen, Matthew Botvinick, Dmitri Chklovskii, Anne Churchland, Claudia Clopath, James DiCarlo, Surya Ganguli, Jeff Hawkins, Konrad Körd­ing, Alexei Koulakov, Yann LeCun, Timothy Lillicrap, Adam M. Marblestone, Bruno Olshausen, Alexandre Pouget, Cristina Savin, Terrence Sejnowski, Eero Simoncelli, Sara Solla, David Sussillo, Andreas S. Tolias, and Doris Tsao. Catalyzing next-generation Artificial Intelligence through NeuroAI. Nature Communications, 14(1):1597, March 2023. ISSN 2041-1723. doi: 10.1038/s41467-023-37180-x. URL https://www.nature.com/articles/s41467-023-37180-x.DOI
  3. [3]Sibo Wang-Chen and Pavan Ramdya. The embodied brain: Bridging the brain, body, and behavior with neuromechanical digital twins, January 2026. URL http://arxiv.org/abs/2601.08056. arXiv:2601.08056 [q-bio].
  4. [4]Josh Merel, Diego Aldarondo, Jesse Marshall, Yuval Tassa, Greg Wayne, and Bence Ölveczky. Deep neuroethology of a virtual rodent, November 2019. URL http://arxiv.org/abs/1911.09451. Issue: arXiv:1911.09451 arXiv:1911.09451 [q-bio].
  5. [5]Diego Aldarondo, Josh Merel, Jesse D. Marshall, Leonard Hasenclever, Ugne Klibaite, Amanda Gellis, Yuval Tassa, Greg Wayne, Matthew Botvinick, and Bence P. Ölveczky. A virtual rodent predicts the structure of neural activity across behaviours. Nature, 632(8025):594–602, August 2024. ISSN 0028-0836, 1476-4687. doi: 10.1038/s41586-024-07633-4. URL https://www.nature.com/articles/s41586-024-07633-4. Number: 8025.DOI
  6. [6]Charles Y. Zhang, Yuanjia Yang, Aidan Sirbu, Elliott T. T. Abe, Emil Wärnberg, Eric J. Leonardis, Diego E. Aldarondo, Adam Lee, Aaditya Prasad, Jason Foat, Kaiwen Bian, Joshua Park, Yashwanth Bhatt, Hutton Saunders, Akira Nagamori, Ayesha R. Thanawala, Kee Wui Huang, Fabian Plum, Hendrik K. Beck, Steven W. Flavell, David Labonte, Blake A. Richards, Bingni W. Brunton, Eiman Azim, Bence P. Ölveczky, and Talmo D. Pereira. MIMIC-MJX: Neuromechanical Emulation of Animal Behavior, November 2025. URL http://arxiv.org/abs/2511.20532. arXiv:2511.20532 [q-bio].
  7. [7]Xue Bin Peng, Pieter Abbeel, Sergey Levine, and Michiel van de Panne. DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skills. ACM Transactions on Graphics, 37(4):1–14, August 2018. ISSN 0730-0301, 1557-7368. doi: 10.1145/3197517.3203111. URL http://arxiv.org/abs/1804.02717. arXiv:1804.02717 [cs].
  8. [8]Josh Merel, Leonard Hasenclever, Alexandre Galashov, Arun Ahuja, Vu Pham, Greg Wayne, Yee Whye Teh, and Nicolas Heess. Neural probabilistic motor primitives for humanoid control, January 2019. URL http://arxiv.org/abs/1811.11711. arXiv:1811.11711 [cs].
  9. [9]Xue Bin Peng, Ze Ma, Pieter Abbeel, Sergey Levine, and Angjoo Kanazawa. AMP: Adversarial Motion Priors for Stylized Physics-Based Character Control. ACM Transactions on Graphics, 40(4):1–20, August 2021. ISSN 0730-0301, 1557-7368. doi: 10.1145/3450626.3459670. URL http://arxiv.org/abs/2104.02180. arXiv:2104.02180 [cs].
  10. [10]Xue Bin Peng, Yunrong Guo, Lina Halper, Sergey Levine, and Sanja Fidler. ASE: Large-Scale Reusable Adversarial Skill Embeddings for Physically Simulated Characters. ACM Transactions on Graphics, 41(4):1–17, July 2022. ISSN 0730-0301, 1557-7368. doi: 10.1145/3528223.3530110. URL http://arxiv.org/abs/2205.01906. arXiv:2205.01906 [cs].
  11. [11]Leonard Hasenclever, Fabio Pardo, Raia Hadsell, Nicolas Heess, and Josh Merel. CoMic: Complementary Task Learning & Mimicry for Reusable Skills. In Proceedings of the 37th International Conference on Machine Learning, pages 4105–4115. PMLR, November 2020. URL https://proceedings.mlr.press/v119/hasenclever20a.html.
  12. [12]Chen Tessler, Yoni Kasten, Yunrong Guo, Shie Mannor, Gal Chechik, and Xue Bin Peng. CALM: Conditional Adversarial Latent Models for Directable Virtual Characters. In Special Interest Group on Computer Graphics and Interactive Techniques Conference Proceedings, pages 1–9, July 2023. doi: 10.1145/3588432.3591541. URL http://arxiv.org/abs/2305.02195. arXiv:2305.02195 [cs].
  13. [13]Zhengyi Luo, Jinkun Cao, Josh Merel, Alexander Winkler, Jing Huang, Kris Kitani, and Weipeng Xu. Universal Humanoid Motion Representations for Physics-Based Control, April 2024. URL http://arxiv.org/abs/2310.04582. arXiv:2310.04582 [cs].
  14. [14]Deepak Pathak, Pulkit Agrawal, Alexei A. Efros, and Trevor Darrell. Curiosity-driven Exploration by Self-supervised Prediction, May 2017. URL http://arxiv.org/abs/1705.05363. arXiv:1705.05363 [cs].
  15. [15]Adrien Ecoffet, Joost Huizinga, Joel Lehman, Kenneth O. Stanley, and Jeff Clune. First return then explore. Nature, 590(7847):580–586, February 2021. ISSN 0028-0836, 1476-4687. doi: 10.1038/s41586-020-03157-9. URL http://arxiv.org/abs/2004.12919. arXiv:2004.12919 [cs].
  16. [16]Matteo Carandini and Anne K. Churchland. Probing perceptual decisions in rodents. Nature Neuroscience, 16(7):824–831, July 2013. ISSN 1546-1726. doi: 10.1038/nn.3410. URL https://www.nature.com/articles/nn.3410.DOI
  17. [17]The International Brain Laboratory, Valeria Aguillon-Rodriguez, Dora Angelaki, Hannah Bayer, Niccolo Bonacchi, Matteo Carandini, Fanny Cazettes, Gaelle Chapuis, Anne K Churchland, Yang Dan, Eric Dewitt, Mayo Faulkner, Hamish Forrest, Laura Haetzel, Michael Häuser, Sonja B Hofer, Fei Hu, Anup Khanal, Christopher Krasniak, Ines Laranjeira, Zachary F Mainen, Guido Meijer, Nathaniel J Miska, Thomas D Mrsic-Flogel, Masayoshi Murakami, Jean-Paul Noel, Alejandro Pan-Vazquez, Cyrille Rossant, Joshua Sanders, Karolina Socha, Rebecca Terry, Anne E Urai, Hernando Vergara, Miles Wells, Christian J Wilson, Ilana B Witten, Lauren E Wool, and Anthony M Zador. Standardized and reproducible measurement of decision-making in mice. eLife, 10:e63711, May 2021. ISSN 2050-084X. doi: 10.7554/eLife.63711. URL https://doi.org/10.7554/eLife.63711.
  18. [18]Simon Musall, Matthew T. Kaufman, Ashley L. Juavinett, Steven Gluf, and Anne K. Churchland. Single-trial neural dynamics are dominated by richly varied movements. Nature Neuroscience, 22(10):1677–1686, October 2019. ISSN 1546-1726. doi: 10.1038/s41593-019-0502-4. URL https://www.nature.com/articles/s41593-019-0502-4.DOI
  19. [19]Nicolas Heess, Dhruva TB, Srinivasan Sriram, Jay Lemmon, Josh Merel, Greg Wayne, Yuval Tassa, Tom Erez, Ziyu Wang, S. M. Ali Eslami, Martin Riedmiller, and David Silver. Emergence of Locomotion Behaviours in Rich Environments, July 2017. URL http://arxiv.org/abs/1707.02286. arXiv:1707.02286 [cs].
  20. [20]Heyuan Yao, Zhenhua Song, Baoquan Chen, and Libin Liu. ControlVAE: Model-Based Learning of Generative Controllers for Physics-Based Characters. ACM Transactions on Graphics, 41(6):1–16, December 2022. ISSN 0730-0301, 1557-7368. doi: 10.1145/3550454.3555434. URL http://arxiv.org/abs/2210.06063. arXiv:2210.06063 [cs].
  21. [21]Jungdam Won, Deepak Gopinath, and Jessica Hodgins. Physics-based character controllers using conditional VAEs. ACM Trans. Graph., 41(4):96:1–96:12, July 2022. ISSN 0730-0301. doi: 10.1145/3528223.3530067. URL https://dl.acm.org/doi/10.1145/3528223.3530067.
  22. [22]Heyuan Yao, Zhenhua Song, Yuyang Zhou, Tenglong Ao, Baoquan Chen, and Libin Liu. MoConVQ: Unified Physics-Based Motion Control via Scalable Discrete Representations, December 2023. URL http://arxiv.org/abs/2310.10198. arXiv:2310.10198 [cs].
  23. [23]Zhengyi Luo, Jinkun Cao, Alexander Winkler, Kris Kitani, and Weipeng Xu. Perpetual Humanoid Control for Real-time Simulated Avatars, September 2023. URL http://arxiv.org/abs/2305.06456. arXiv:2305.06456 [cs].
  24. [24]Davis Rempe, Tolga Birdal, Aaron Hertzmann, Jimei Yang, Srinath Sridhar, and Leonidas J. Guibas. HuMoR: 3D Human Motion Model for Robust Pose Estimation, August 2021. URL http://arxiv.org/abs/2105.04668. arXiv:2105.04668 [cs].
  25. [25]Guy Tevet, Sigal Raab, Brian Gordon, Yonatan Shafir, Daniel Cohen-Or, and Amit H. Bermano. Human Motion Diffusion Model, October 2022. URL http://arxiv.org/abs/2209.14916. arXiv:2209.14916 [cs].
  26. [26]Nolan Wagener, Andrey Kolobov, Felipe Vieira Frujeri, Ricky Loynd, Ching-An Cheng, and Matthew Hausknecht. MoCapAct: A Multi-Task Dataset for Simulated Humanoid Control, January 2023. URL http://arxiv.org/abs/2208.07363. arXiv:2208.07363 [cs].
  27. [27]Ilija Radosavovic, Bike Zhang, Baifeng Shi, Jathushan Rajasegaran, Sarthak Kamat, Trevor Darrell, Koushil Sreenath, and Jitendra Malik. Humanoid Locomotion as Next Token Prediction, February 2024. URL http://arxiv.org/abs/2402.19469. arXiv:2402.19469 [cs].
  28. [28]Roman Vaxenburg, Igor Siwanowicz, Josh Merel, Alice A. Robie, Carmen Morrow, Guido Novati, Zinovia Stefani, Gert-Jan Both, Gwyneth M. Card, Michael B. Reiser, Matthew M. Botvinick, Kristin M. Branson, Yuval Tassa, and Srinivas C. Turaga. Whole-body physics simulation of fruit fly locomotion. Nature, pages 1–3, April 2025. ISSN 1476-4687. doi: 10.1038/s41586-025-09029-4. URL https://www.nature.com/articles/s41586-025-09029-4.DOI
  29. [29]Sibo Wang-Chen, Victor Alfred Stimpfling, Thomas Ka Chung Lam, Pembe Gizem Öz dil, Louise Genoud, Femke Hurtak, and Pavan Ramdya. NeuroMechFly v2: simulating embodied sensorimotor control in adult Drosophila. Nature Methods, 21(12):2353–2362, December 2024. ISSN 1548-7105. doi: 10.1038/s41592-024-02497-y. URL https://www.nature.com/articles/s41592-024-02497-y.
  30. [30]Mengdi Zhao, Ning Wang, Xinrui Jiang, Xiaoyang Ma, Haixin Ma, Gan He, Kai Du, Lei Ma, and Tiejun Huang. An integrative data-driven model simulating C. elegans brain, body and environment interactions. Nature Computational Science, 4(12):978–990, December 2024. ISSN 2662-8457. doi: 10.1038/s43588-024-00738-w. URL https://www.nature.com/articles/s43588-024-00738-w.DOI
  31. [31]Reece Keller, Alyn Tornell, Felix Pei, Xaq Pitkow, Leo Kozachkov, and Aran Nayebi. Autonomous Behavior and Whole-Brain Dynamics Emerge in Embodied Zebrafish Agents with Model-based Intrinsic Motivation, May 2025. URL http://arxiv.org/abs/2506.00138. arXiv:2506.00138 [q-bio].
  32. [32]John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal Policy Optimization Algorithms, August 2017. URL http://arxiv.org/abs/1707.06347. arXiv:1707.06347 [cs].
  33. [33]Stephane Ross, Geoffrey J. Gordon, and J. Andrew Bagnell. A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning, March 2011. URL http://arxiv.org/abs/1011.0686. arXiv:1011.0686 [cs].
  34. [34]Dan Friedman and Adji Bousso Dieng. The Vendi Score: A Diversity Evaluation Metric for Machine Learning, July 2023. URL http://arxiv.org/abs/2210.02410. arXiv:2210.02410 [cs].
  35. [35]Tom Stanic, Marco Jiralerspong, Xiaofeng Zhang, Danilo Vucetic, and Gauthier Gidel. Beyond reward maximization: Evaluating the diversity of trajectories in reinforcement learning with temporal vendi score. 2026.
  36. [36]Erik M. Lintunen. VendiRL: A Framework for Self-Supervised Reinforcement Learning of Diversely Diverse Skills, October 2025. URL http://arxiv.org/abs/2509.02930. arXiv:2509.02930 [cs].
  37. [37]Mikołaj Bińkowski, Danica J. Sutherland, Michael Arbel, and Arthur Gretton. Demystifying MMD GANs, January 2021. URL http://arxiv.org/abs/1801.01401. arXiv:1801.01401 [stat].
  38. [38]Sigal Raab, Inbal Leibovitch, Peizhuo Li, Kfir Aberman, Olga Sorkine-Hornung, and Daniel Cohen-Or. MoDi: Unconditional Motion Synthesis from Diverse Data, December 2022. URL http://arxiv.org/abs/2206.08010. arXiv:2206.08010 [cs].
  39. [39]Simon Danner, Courtney Shepard, Casey Hainline, Natalia Shevtsova, Ilya Rybak, and David Magnuson. Spinal control of locomotion before and after spinal cord injury. June 2023. doi: 10.1101/2023.03.22.533794. Journal Abbreviation: bioRxiv : the preprint server for biology Publication Title: bioRxiv : the preprint server for biology.DOI
  40. [40]Frank P.T. Hamers, Alex J. Lankhorst, Teus Jan van Laar, Wouter B. Veldhuis, and Willem Hendrik Gispen. Automated Quantitative Gait Analysis During Overground Locomotion in the Rat: Its Application to Spinal Cord Contusion and Transection Injuries. Journal of Neurotrauma, 18(2):187–201, February 2001. ISSN 0897-7151. doi: 10.1089/08977150150502613. URL https://journals.sagepub.com/action/showAbstract.DOI
  41. [41]César S. Mendes, Imre Bartos, Zsuzsanna Márka, Turgay Akay, Szabolcs Márka, and Richard S. Mann. Quantification of gait parameters in freely walking rodents. BMC Biology, 13(1):50, July 2015. ISSN 1741-7007. doi: 10.1186/s12915-015-0154-0. URL https://doi.org/10.1186/s12915-015-0154-0.
  42. [42]Timothy W Dunn, Jesse D Marshall, Kyle S Severson, Diego E Aldarondo, David G C Hildebrand, Selmaan N Chettih, William L Wang, Amanda J Gellis, David E Carlson, Dmitry Aronov, Winrich A Freiwald, Fan Wang, and Bence P Ölveczky. Geometric deep learning enables 3D kinematic profiling across species and environments. Nat. Methods, 18(5):564–573, May 2021.DOI
  43. [43]Emanuel Todorov, Tom Erez, and Yuval Tassa. MuJoCo: A physics engine for model-based control. 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 5026–5033, October 2012.DOI
  44. [44]MuJoCo XLA Authors. MuJoCo XLA (MJX). https://mujoco.readthedocs.io/en/stable/mjx.html.
  45. [45]James Bradbury, Roy Frostig, Peter Hawkins, Matthew James Johnson, Chris Leary, Dougal Maclaurin, George Necula, Adam Paszke, Jake VanderPlas, Skye Wanderman-Milne, and Qiao Zhang. JAX: composable transformations of Python+NumPy programs, 2018.
  46. [46]Kevin Zakka, Baruch Tabanpour, Qia yuan Liao, Mustafa Haiderbhai, Samuel Holt, Jing Yuan Luo, Arthur Allshire, Erik Frey, Koushil Sreenath, Lueder A Kahrs, Carmelo Sferrazza, Yuval Tassa, and Pieter Abbeel. MuJoCo playground. arXiv [cs.RO], February 2025.arxiv.org/abs/2502.08844
  47. [47]C Daniel Freeman, Erik Frey, Anton Raichuk, Sertan Girgin, Igor Mordatch, and Olivier Bachem. Brax – a differentiable physics engine for large scale rigid body simulation. arXiv [cs.RO], June 2021.arxiv.org/abs/2106.13281
  48. [48]Xue Bin Peng, Marcin Andrychowicz, Wojciech Zaremba, and Pieter Abbeel. Sim-to-Real Transfer of Robotic Control with Dynamics Randomization. In 2018 IEEE International Conference on Robotics and Automation (ICRA), pages 3803–3810, May 2018. doi: 10.1109/ICRA.2018.8460528. URL http://arxiv.org/abs/1710.06537. arXiv:1710.06537 [cs].
  49. [49]Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium, January 2018. URL http://arxiv.org/abs/1706.08500. arXiv:1706.08500 [cs].
  50. [50]Yi Zhou, Connelly B. Jones, Jianing Lu, Jimei Yang, and Hao Li. On the Continuity of Rotation Representations in Neural Networks, June 2020. URL http://arxiv.org/abs/1812.07035. arXiv:1812.07035 [cs].
  51. [51]Arthur Gretton, Karsten M. Borgwardt, Malte J. Rasch, Bernhard Schölkopf, and Alexander Smola. A Kernel Two-Sample Test. Journal of Machine Learning Research, 13(25):723–773, 2012. ISSN 1533-7928. URL http://jmlr.org/papers/v13/gretton12a.html.DOI

Paper details

vlnc:W_zussgsvm
Version
v2 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 16

downloads · 3

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents