Behavior-Driven Synthesis of Human Dynamics

03/08/2021
by   Andreas Blattmann, et al.
0

Generating and representing human behavior are of major importance for various computer vision applications. Commonly, human video synthesis represents behavior as sequences of postures while directly predicting their likely progressions or merely changing the appearance of the depicted persons, thus not being able to exercise control over their actual behavior during the synthesis process. In contrast, controlled behavior synthesis and transfer across individuals requires a deep understanding of body dynamics and calls for a representation of behavior that is independent of appearance and also of specific postures. In this work, we present a model for human behavior synthesis which learns a dedicated representation of human dynamics independent of postures. Using this representation, we are able to change the behavior of a person depicted in an arbitrary posture, or to even directly transfer behavior observed in a given video sequence. To this end, we propose a conditional variational framework which explicitly disentangles posture from behavior. We demonstrate the effectiveness of our approach on this novel task, evaluating capturing, transferring, and sampling fine-grained, diverse behavior, both quantitatively and qualitatively. Project page is available at https://cutt.ly/5l7rXEp

READ FULL TEXT

page 2

page 5

page 13

page 17

page 19

page 21

page 22

page 23

research
05/10/2021

Stochastic Image-to-Video Synthesis using cINNs

Video understanding calls for a model to learn the characteristic interp...
research
10/21/2021

LARNet: Latent Action Representation for Human Action Synthesis

We present LARNet, a novel end-to-end approach for generating human acti...
research
06/21/2021

Understanding Object Dynamics for Interactive Image-to-Video Synthesis

What would be the effect of locally poking a static scene? We present an...
research
11/25/2019

Appearance Composing GAN: A General Method for Appearance-Controllable Human Video Motion Transfer

Due to the rapid development of GANs, there has been significant progres...
research
12/10/2017

Dynamics Transfer GAN: Generating Video by Transferring Arbitrary Temporal Dynamics from a Source Video to a Single Target Image

In this paper, we propose Dynamics Transfer GAN; a new method for genera...
research
12/16/2020

uBAM: Unsupervised Behavior Analysis and Magnification using Deep Learning

Motor behavior analysis is essential to biomedical research and clinical...
research
06/30/2023

DisCo: Disentangled Control for Referring Human Dance Generation in Real World

Generative AI has made significant strides in computer vision, particula...

Please sign up or login with your details

Forgot password? Click here to reset