Image Animation with Perturbed Masks

11/13/2020
by   Yoav Shalev, et al.
0

We present a novel approach for image-animation of a source image by a driving video, both depicting the same type of object. We do not assume the existence of pose models and our method is able to animate arbitrary objects without knowledge of the object's structure. Furthermore, both the driving video and the source image are only seen during test-time. Our method is based on a shared mask generator, which separates the foreground object from its background, and captures the object's general pose and shape. A mask-refinement module then replaces, in the mask extracted from the driver image, the identity of the driver with the identity of the source. Conditioned on the source image, the transformed mask is then decoded by a multi-scale generator that renders a realistic image, in which the content of the source frame is animated by the pose in the driving video. Due to lack of fully supervised data, we train on the task of reconstructing frames from the same video the source image is taken from. In order to control the source of the identity of the output frame, we employ during training perturbations that remove the unwanted identity information. Our method is shown to greatly outperform the state of the art methods on multiple benchmarks. Our code and samples are available at https://github.com/itsyoavshalev/Image-Animation-with-Perturbed-Masks

READ FULL TEXT

page 4

page 7

page 8

research
07/26/2023

Tracking Anything in High Quality

Visual object tracking is a fundamental video task in computer vision. R...
research
12/03/2020

Make One-Shot Video Object Segmentation Efficient Again

Video object segmentation (VOS) describes the task of segmenting a set o...
research
08/22/2022

FurryGAN: High Quality Foreground-aware Image Synthesis

Foreground-aware image synthesis aims to generate images as well as thei...
research
05/22/2023

UVOSAM: A Mask-free Paradigm for Unsupervised Video Object Segmentation via Segment Anything Model

Unsupervised video object segmentation has made significant progress in ...
research
12/20/2021

Image Animation with Keypoint Mask

Motion transfer is the task of synthesizing future video frames of a sin...
research
07/19/2023

Implicit Identity Representation Conditioned Memory Compensation Network for Talking Head video Generation

Talking head video generation aims to animate a human face in a still im...
research
07/17/2020

Cross-Identity Motion Transfer for Arbitrary Objects through Pose-Attentive Video Reassembling

We propose an attention-based networks for transferring motions between ...

Please sign up or login with your details

Forgot password? Click here to reset