DeepAI AI Chat
Log In Sign Up

Dance Style Transfer with Cross-modal Transformer

08/19/2022
by   Wenjie Yin, et al.
0

We present CycleDance, a dance style transfer system to transform an existing motion clip in one dance style to a motion clip in another dance style while attempting to preserve motion context of the dance. Our method extends an existing CycleGAN architecture for modeling audio sequences and integrates multimodal transformer encoders to account for music context. We adopt sequence length-based curriculum learning to stabilize training. Our approach captures rich and long-term intra-relations between motion frames, which is a common challenge in motion transfer and synthesis work. We further introduce new metrics for gauging transfer strength and content preservation in the context of dance movements. We perform an extensive ablation study as well as a human study including 30 participants with 5 or more years of dance experience. The results demonstrate that CycleDance generates realistic movements with the target style, significantly outperforming the baseline CycleGAN on naturalness, transfer strength, and content preservation.

READ FULL TEXT

page 1

page 6

page 7

page 8

05/14/2019

Style Transformer: Unpaired Text Style Transfer without Disentangled Latent Representation

Disentangling the content and style in the latent space is prevalent in ...
06/11/2020

Dance Revolution: Long Sequence Dance Generation with Music via Curriculum Learning

Dancing to music is one of human's innate abilities since ancient times....
04/30/2020

Contextual Text Style Transfer

We introduce a new task, Contextual Text Style Transfer - translating a ...
04/15/2022

Human Judgement as a Compass to Navigate Automatic Metrics for Formality Transfer

Although text style transfer has witnessed rapid development in recent y...
05/11/2020

Reinforced Rewards Framework for Text Style Transfer

Style transfer deals with the algorithms to transfer the stylistic prope...
04/08/2020

Conditional Rap Lyrics Generation with Denoising Autoencoders

We develop a method for automatically synthesizing a rap verse given an ...
01/12/2022

Real-Time Style Modelling of Human Locomotion via Feature-Wise Transformations and Local Motion Phases

Controlling the manner in which a character moves in a real-time animati...