Copy Motion From One to Another: Fake Motion Video Generation

05/03/2022
by   Zhenguang Liu, et al.
0

One compelling application of artificial intelligence is to generate a video of a target person performing arbitrary desired motion (from a source person). While the state-of-the-art methods are able to synthesize a video demonstrating similar broad stroke motion details, they are generally lacking in texture details. A pertinent manifestation appears as distorted face, feet, and hands, and such flaws are very sensitively perceived by human observers. Furthermore, current methods typically employ GANs with a L2 loss to assess the authenticity of the generated videos, inherently requiring a large amount of training samples to learn the texture details for adequate video generation. In this work, we tackle these challenges from three aspects: 1) We disentangle each video frame into foreground (the person) and background, focusing on generating the foreground to reduce the underlying dimension of the network output. 2) We propose a theoretically motivated Gromov-Wasserstein loss that facilitates learning the mapping from a pose to a foreground image. 3) To enhance texture details, we encode facial features with geometric guidance and employ local GANs to refine the face, feet, and hands. Extensive experiments show that our method is able to generate realistic target person videos, faithfully copying complex motions from a source person. Our code and datasets are released at https://github.com/Sifann/FakeMotion

READ FULL TEXT

page 3

page 6

research
08/12/2019

Multi-Frame Content Integration with a Spatio-Temporal Attention Mechanism for Person Video Motion Transfer

Existing person video generation methods either lack the flexibility in ...
research
09/01/2022

REMOT: A Region-to-Whole Framework for Realistic Human Motion Transfer

Human Video Motion Transfer (HVMT) aims to, given an image of a source p...
research
10/27/2021

Image Comes Dancing with Collaborative Parsing-Flow Video Synthesis

Transferring human motion from a source to a target person poses great p...
research
03/30/2019

Dance Dance Generation: Motion Transfer for Internet Videos

This work presents computational methods for transferring body movements...
research
06/27/2021

Robust Pose Transfer with Dynamic Details using Neural Video Rendering

Pose transfer of human videos aims to generate a high fidelity video of ...
research
12/02/2020

Single-Shot Freestyle Dance Reenactment

The task of motion transfer between a source dancer and a target person ...
research
12/15/2021

Vision Transformer Based Video Hashing Retrieval for Tracing the Source of Fake Videos

Conventional fake video detection methods outputs a possibility value or...

Please sign up or login with your details

Forgot password? Click here to reset