DeepAI
Log In Sign Up

Cross-identity Video Motion Retargeting with Joint Transformation and Synthesis

10/02/2022
by   Haomiao Ni, et al.
0

In this paper, we propose a novel dual-branch Transformation-Synthesis network (TS-Net), for video motion retargeting. Given one subject video and one driving video, TS-Net can produce a new plausible video with the subject appearance of the subject video and motion pattern of the driving video. TS-Net consists of a warp-based transformation branch and a warp-free synthesis branch. The novel design of dual branches combines the strengths of deformation-grid-based transformation and warp-free generation for better identity preservation and robustness to occlusion in the synthesized videos. A mask-aware similarity module is further introduced to the transformation branch to reduce computational overhead. Experimental results on face and dance datasets show that TS-Net achieves better performance in video motion retargeting than several state-of-the-art models as well as its single-branch variants. Our code is available at https://github.com/nihaomiao/WACV23_TSNet.

READ FULL TEXT

page 1

page 3

page 5

page 6

page 7

page 8

04/04/2019

Spatiotemporal CNN for Video Object Segmentation

In this paper, we present a unified, end-to-end trainable spatiotemporal...
03/26/2022

V3GAN: Decomposing Background, Foreground and Motion for Video Generation

Video generation is a challenging task that requires modeling plausible ...
04/14/2022

Human Identity-Preserved Motion Retargeting in Video Synthesis by Feature Disentanglement

Most motion retargeting methods in human action video synthesis decompos...
11/30/2020

One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencing

We propose a neural talking-head video synthesis model and demonstrate i...
08/21/2018

Deep Video-Based Performance Cloning

We present a new video-based performance cloning technique. After traini...
11/30/2022

BEVPoolv2: A Cutting-edge Implementation of BEVDet Toward Deployment

We release a new codebase version of the BEVDet, dubbed branch dev2.0. W...
08/27/2022

LAB-Net: LAB Color-Space Oriented Lightweight Network for Shadow Removal

This paper focuses on the limitations of current over-parameterized shad...