Log In Sign Up

Cross-identity Video Motion Retargeting with Joint Transformation and Synthesis

by   Haomiao Ni, et al.

In this paper, we propose a novel dual-branch Transformation-Synthesis network (TS-Net), for video motion retargeting. Given one subject video and one driving video, TS-Net can produce a new plausible video with the subject appearance of the subject video and motion pattern of the driving video. TS-Net consists of a warp-based transformation branch and a warp-free synthesis branch. The novel design of dual branches combines the strengths of deformation-grid-based transformation and warp-free generation for better identity preservation and robustness to occlusion in the synthesized videos. A mask-aware similarity module is further introduced to the transformation branch to reduce computational overhead. Experimental results on face and dance datasets show that TS-Net achieves better performance in video motion retargeting than several state-of-the-art models as well as its single-branch variants. Our code is available at


page 1

page 3

page 5

page 6

page 7

page 8


Spatiotemporal CNN for Video Object Segmentation

In this paper, we present a unified, end-to-end trainable spatiotemporal...

V3GAN: Decomposing Background, Foreground and Motion for Video Generation

Video generation is a challenging task that requires modeling plausible ...

Human Identity-Preserved Motion Retargeting in Video Synthesis by Feature Disentanglement

Most motion retargeting methods in human action video synthesis decompos...

One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencing

We propose a neural talking-head video synthesis model and demonstrate i...

Deep Video-Based Performance Cloning

We present a new video-based performance cloning technique. After traini...

BEVPoolv2: A Cutting-edge Implementation of BEVDet Toward Deployment

We release a new codebase version of the BEVDet, dubbed branch dev2.0. W...

LAB-Net: LAB Color-Space Oriented Lightweight Network for Shadow Removal

This paper focuses on the limitations of current over-parameterized shad...