Unsupervised Volumetric Animation

01/26/2023
by   Aliaksandr Siarohin, et al.
0

We propose a novel approach for unsupervised 3D animation of non-rigid deformable objects. Our method learns the 3D structure and dynamics of objects solely from single-view RGB videos, and can decompose them into semantically meaningful parts that can be tracked and animated. Using a 3D autodecoder framework, paired with a keypoint estimator via a differentiable PnP algorithm, our model learns the underlying object geometry and parts decomposition in an entirely unsupervised manner. This allows it to perform 3D segmentation, 3D keypoint estimation, novel view synthesis, and animation. We primarily evaluate the framework on two video datasets: VoxCeleb 256^2 and TEDXPeople 256^2. In addition, on the Cats 256^2 image dataset, we show it even learns compelling 3D geometry from still images. Finally, we show our model can obtain animatable 3D objects from a single or few images. Code and visual results available on our project website, see https://snap-research.github.io/unsupervised-volumetric-animation .

READ FULL TEXT

page 1

page 4

page 7

page 8

page 9

page 17

research
03/30/2023

Few-shot Geometry-Aware Keypoint Localization

Supervised keypoint localization methods rely on large manually labeled ...
research
02/18/2022

KINet: Keypoint Interaction Networks for Unsupervised Forward Modeling

Object-centric representation is an essential abstraction for physical r...
research
11/24/2020

UKPGAN: Unsupervised KeyPoint GANeration

Keypoint detection is an essential component for the object registration...
research
04/15/2021

GANcraft: Unsupervised 3D Neural Rendering of Minecraft Worlds

We present GANcraft, an unsupervised neural rendering framework for gene...
research
11/11/2021

Unsupervised Part Discovery from Contrastive Reconstruction

The goal of self-supervised visual representation learning is to learn s...
research
03/12/2019

Unsupervised Discovery of Parts, Structure, and Dynamics

Humans easily recognize object parts and their hierarchical structure by...
research
12/01/2022

NeuWigs: A Neural Dynamic Model for Volumetric Hair Capture and Animation

The capture and animation of human hair are two of the major challenges ...

Please sign up or login with your details

Forgot password? Click here to reset