ChoreoNet: Towards Music to Dance Synthesis with Choreographic Action Unit

09/16/2020
by   Zijie Ye, et al.
7

Dance and music are two highly correlated artistic forms. Synthesizing dance motions has attracted much attention recently. Most previous works conduct music-to-dance synthesis via directly music to human skeleton keypoints mapping. Meanwhile, human choreographers design dance motions from music in a two-stage manner: they firstly devise multiple choreographic dance units (CAUs), each with a series of dance motions, and then arrange the CAU sequence according to the rhythm, melody and emotion of the music. Inspired by these, we systematically study such two-stage choreography approach and construct a dataset to incorporate such choreography knowledge. Based on the constructed dataset, we design a two-stage music-to-dance synthesis framework ChoreoNet to imitate human choreography procedure. Our framework firstly devises a CAU prediction model to learn the mapping relationship between music and CAU sequences. Afterwards, we devise a spatial-temporal inpainting model to convert the CAU sequence into continuous dance motions. Experimental results demonstrate that the proposed ChoreoNet outperforms baseline methods (0.622 in terms of CAU BLEU score and 1.59 in terms of user study score).

READ FULL TEXT

Authors

page 1

page 2

page 3

page 4

12/13/2019

Music-oriented Dance Video Synthesis with Pose Perceptual Loss

We present a learning-based approach with pose perceptual loss for autom...
02/02/2020

Music2Dance: DanceNet for Music-driven Dance Generation

Synthesize human motions from music, i.e., music to dance, is appealing ...
04/30/2021

Dance Generation with Style Embedding: Learning and Transferring Latent Representations of Dance Styles

Choreography refers to creation of dance steps and motions for dances ac...
11/23/2021

Rhythm is a Dancer: Music-Driven Motion Synthesis with Global Structure

Synthesizing human motion with a global structure, such as a choreograph...
11/05/2019

Dancing to Music

Dancing to music is an instinctive move by humans. Learning to model the...
09/17/2020

DanceIt: Music-inspired Dancing Video Synthesis

Close your eyes and listen to music, one can easily imagine an actor dan...
07/02/2018

An energy-based generative sequence model for testing sensory theories of Western harmony

The relationship between sensory consonance and Western harmony is an im...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.