Reconstructing Animatable Categories from Videos

05/10/2023
by   Gengshan Yang, et al.
0

Building animatable 3D models is challenging due to the need for 3D scans, laborious registration, and manual rigging, which are difficult to scale to arbitrary categories. Recently, differentiable rendering provides a pathway to obtain high-quality 3D models from monocular videos, but these are limited to rigid categories or single instances. We present RAC that builds category 3D models from monocular videos while disentangling variations over instances and motion over time. Three key ideas are introduced to solve this problem: (1) specializing a skeleton to instances via optimization, (2) a method for latent space regularization that encourages shared structure across a category while maintaining instance details, and (3) using 3D background models to disentangle objects from the background. We show that 3D models of humans, cats, and dogs can be learned from 50-100 internet videos.

READ FULL TEXT

page 1

page 5

page 6

page 8

research
04/14/2023

CAMM: Building Category-Agnostic and Animatable 3D Models from Monocular Videos

Animating an object in 3D often requires an articulated structure, e.g. ...
research
02/22/2023

Vid2Avatar: 3D Avatar Reconstruction from Videos in the Wild via Self-supervised Scene Decomposition

We present Vid2Avatar, a method to learn human avatars from monocular in...
research
08/19/2023

Root Pose Decomposition Towards Generic Non-rigid 3D Reconstruction with Monocular Videos

This work focuses on the 3D reconstruction of non-rigid objects based on...
research
09/14/2018

Transferring Grasping Skills to Novel Instances by Latent Space Non-Rigid Registration

Robots acting in open environments need to be able to handle novel objec...
research
02/13/2020

Summarizing the performances of a background subtraction algorithm measured on several videos

There exist many background subtraction algorithms to detect motion in v...
research
12/04/2020

Scale-aware Insertion of Virtual Objects in Monocular Videos

In this paper, we propose a scale-aware method for inserting virtual obj...
research
07/07/2023

AutoDecoding Latent 3D Diffusion Models

We present a novel approach to the generation of static and articulated ...

Please sign up or login with your details

Forgot password? Click here to reset