WorksComputer Science
CFGComposer: Compositional Audio-Text-to-Motion Generation via Copula Score Correction
Abstract
Generating whole-body gestures that are simultaneously rhythmically synchronized with speech audio and semantically aligned with text descriptions is a fundamental challenge in embodied AI and social robotics. We tackle this Audio-Text-to-Motion (AT2M) problem by composing two independently-trained diffusion models at inference time and refine bias in joint inference by modeling the copula score . A rigorous theoretical analysis reveals that naive epsilon blending corresponds to a Product-of-Experts formulation that omits the copula score : the cross-modal interaction between audio and text that neither marginal model can capture. We propose CFGComposer: a jointly-trained AT2M diffusion model that estimates via a CFG analogy and injects it as an -space correction. Experiments on BEAT demonstrate best-in-class semantic fidelity (SRGR , FID ) with GT-level beat alignment (BA vs. GT ), and 76-83% win rate in blind pairwise human evaluation against all prior methods. Physical deployment on a Unitree G1 humanoid confirms direct sim-to-real transfer.