MetaPix: Few-Shot Video Retargeting

10/10/2019
by   Jessica Lee, et al.
8

We address the task of unsupervised retargeting of human actions from one video to another. We consider the challenging setting where only a few frames of the target is available. The core of our approach is a conditional generative model that can transcode input skeletal poses (automatically extracted with an off-the-shelf pose estimator) to output target frames. However, it is challenging to build a universal transcoder because humans can appear wildly different due to clothing and background scene geometry. Instead, we learn to adapt - or personalize - a universal generator to the particular human and background in the target. To do so, we make use of meta-learning to discover effective strategies for on-the-fly personalization. One significant benefit of meta-learning is that the personalized transcoder naturally enforces temporal coherence across its generated frames; all frames contain consistent clothing and background geometry of the target. We experiment on in-the-wild internet videos and images and show our approach improves over widely-used baselines for the task.

READ FULL TEXT

page 2

page 5

page 7

page 9

research
01/13/2022

MetaDance: Few-shot Dancing Video Retargeting via Temporal-aware Meta-learning

Dancing video retargeting aims to synthesize a video that transfers the ...
research
02/20/2020

A Structured Prediction Approach for Conditional Meta-Learning

Optimization-based meta-learning algorithms are a powerful class of meth...
research
02/05/2018

One-Shot Imitation from Observing Humans via Domain-Adaptive Meta-Learning

Humans and animals are capable of learning a new behavior by observing o...
research
02/22/2023

Vid2Avatar: 3D Avatar Reconstruction from Videos in the Wild via Self-supervised Scene Decomposition

We present Vid2Avatar, a method to learn human avatars from monocular in...
research
03/27/2022

Temporal Transductive Inference for Few-Shot Video Object Segmentation

Few-shot video object segmentation (FS-VOS) aims at segmenting video fra...
research
04/02/2020

Scene-Adaptive Video Frame Interpolation via Meta-Learning

Video frame interpolation is a challenging problem because there are dif...
research
11/20/2015

Personalizing Human Video Pose Estimation

We propose a personalized ConvNet pose estimator that automatically adap...

Please sign up or login with your details

Forgot password? Click here to reset