GMD: Controllable Human Motion Synthesis via Guided Diffusion Models

by   Korrawe Karunratanakul, et al.

Denoising diffusion models have shown great promise in human motion synthesis conditioned on natural language descriptions. However, it remains a challenge to integrate spatial constraints, such as pre-defined motion trajectories and obstacles, which is essential for bridging the gap between isolated human motion and its surrounding environment. To address this issue, we propose Guided Motion Diffusion (GMD), a method that incorporates spatial constraints into the motion generation process. Specifically, we propose an effective feature projection scheme that largely enhances the coherency between spatial information and local poses. Together with a new imputation formulation, the generated motion can reliably conform to spatial constraints such as global motion trajectories. Furthermore, given sparse spatial constraints (e.g. sparse keyframes), we introduce a new dense guidance approach that utilizes the denoiser of diffusion models to turn a sparse signal into denser signals, effectively guiding the generation motion to the given constraints. The extensive experiments justify the development of GMD, which achieves a significant improvement over state-of-the-art methods in text-based motion generation while being able to control the synthesized motions with spatial constraints.


page 1

page 2

page 7

page 8


InterGen: Diffusion-based Multi-human Motion Generation under Complex Interactions

We have recently seen tremendous progress in diffusion advances for gene...

Diffusion Motion: Generate Text-Guided 3D Human Motion by Diffusion Model

We propose a simple and novel method for generating 3D human motion from...

Make-An-Animation: Large-Scale Text-conditional 3D Human Motion Generation

Text-guided human motion generation has drawn significant interest becau...

FLAME: Free-form Language-based Motion Synthesis Editing

Text-based motion generation models are drawing a surge of interest for ...

Bipartite Graph Diffusion Model for Human Interaction Generation

The generation of natural human motion interactions is a hot topic in co...

Understanding Text-driven Motion Synthesis with Keyframe Collaboration via Diffusion Models

The emergence of text-driven motion synthesis technique provides animato...

TEDi: Temporally-Entangled Diffusion for Long-Term Motion Synthesis

The gradual nature of a diffusion process that synthesizes samples in sm...

Please sign up or login with your details

Forgot password? Click here to reset