MoDA: Modeling Deformable 3D Objects from Casual Videos

04/17/2023
by   Chaoyue Song, et al.
0

In this paper, we focus on the challenges of modeling deformable 3D objects from casual videos. With the popularity of neural radiance fields (NeRF), many works extend it to dynamic scenes with a canonical NeRF and a deformation model that achieves 3D point transformation between the observation space and the canonical space. Recent works rely on linear blend skinning (LBS) to achieve the canonical-observation transformation. However, the linearly weighted combination of rigid transformation matrices is not guaranteed to be rigid. As a matter of fact, unexpected scale and shear factors often appear. In practice, using LBS as the deformation model can always lead to skin-collapsing artifacts for bending or twisting motions. To solve this problem, we propose neural dual quaternion blend skinning (NeuDBS) to achieve 3D point deformation, which can perform rigid transformation without skin-collapsing artifacts. Besides, we introduce a texture filtering approach for texture rendering that effectively minimizes the impact of noisy colors outside target deformable objects. Extensive experiments on real and synthetic datasets show that our approach can reconstruct 3D models for humans and animals with better qualitative and quantitative performance than state-of-the-art methods.

READ FULL TEXT

page 1

page 3

page 4

page 7

page 8

research
05/06/2021

Animatable Neural Radiance Fields for Human Body Modeling

This paper addresses the challenge of reconstructing an animatable human...
research
07/19/2022

NDF: Neural Deformable Fields for Dynamic Human Modelling

We propose Neural Deformable Fields (NDF), a new representation for dyna...
research
03/30/2022

CaDeX: Learning Canonical Deformation Coordinate Space for Dynamic Surface Representation via Neural Homeomorphism

While neural representations for static 3D shapes are widely studied, re...
research
05/24/2019

DEMEA: Deep Mesh Autoencoders for Non-Rigidly Deforming Objects

Mesh autoencoders are commonly used for dimensionality reduction, sampli...
research
11/26/2022

Reduced Representation of Deformation Fields for Effective Non-rigid Shape Matching

In this work we present a novel approach for computing correspondences b...
research
07/23/2023

TransHuman: A Transformer-based Human Representation for Generalizable Neural Human Rendering

In this paper, we focus on the task of generalizable neural human render...
research
06/24/2021

HyperNeRF: A Higher-Dimensional Representation for Topologically Varying Neural Radiance Fields

Neural Radiance Fields (NeRF) are able to reconstruct scenes with unprec...

Please sign up or login with your details

Forgot password? Click here to reset