betaFeedback

WorksComputer Science

CFGComposer: Compositional Audio-Text-to-Motion Generation via Copula Score Correction

Abstract

Generating whole-body gestures that are simultaneously rhythmically synchronized with speech audio and semantically aligned with text descriptions is a fundamental challenge in embodied AI and social robotics. We tackle this Audio-Text-to-Motion (AT2M) problem by composing two independently-trained diffusion models at inference time and refine bias in joint inference by modeling the copula score sIs_I. A rigorous theoretical analysis reveals that naive epsilon blending corresponds to a Product-of-Experts formulation that omits the copula score sIs_I: the cross-modal interaction between audio and text that neither marginal model can capture. We propose CFGComposer: a jointly-trained AT2M diffusion model that estimates sIs_I via a CFG analogy and injects it as an εε-space correction. Experiments on BEAT demonstrate best-in-class semantic fidelity (SRGR 0.99160.9916, FID 27.1827.18) with GT-level beat alignment (BA 0.18260.1826 vs. GT 0.17430.1743), and 76-83% win rate in blind pairwise human evaluation against all prior methods. Physical deployment on a Unitree G1 humanoid confirms direct sim-to-real transfer.

Paper details

vlnc:W_4fzjkebv
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
arXiv nonexclusive license
Usage

views · 8

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…