3D Surface Reconstruction in the Wild by Deforming Shape Priors from Synthetic Data

by   Nicolai Häni, et al.

Reconstructing the underlying 3D surface of an object from a single image is a challenging problem that has received extensive attention from the computer vision community. Many learning-based approaches tackle this problem by learning a 3D shape prior from either ground truth 3D data or multi-view observations. To achieve state-of-the-art results, these methods assume that the objects are specified with respect to a fixed canonical coordinate frame, where instances of the same category are perfectly aligned. In this work, we present a new method for joint category-specific 3D reconstruction and object pose estimation from a single image. We show that one can leverage shape priors learned on purely synthetic 3D data together with a point cloud pose canonicalization method to achieve high-quality 3D reconstruction in the wild. Given a single depth image at test time, we first transform this partial point cloud into a learned canonical frame. Then, we use a neural deformation field to reconstruct the 3D surface of the object. Finally, we jointly optimize object pose and 3D shape to fit the partial depth observation. Our approach achieves state-of-the-art reconstruction performance across several real-world datasets, even when trained only on synthetic data. We further show that our method generalizes to different input modalities, from dense depth images to sparse and noisy LIDAR scans.


page 1

page 5

page 6

page 7

page 13

page 14


Weakly-supervised 3D Shape Completion in the Wild

3D shape completion for real data is important but challenging, since pa...

Reconstruct, Rasterize and Backprop: Dense shape and pose estimation from a single image

This paper presents a new system to obtain dense object reconstructions ...

Fully Understanding Generic Objects: Modeling, Segmentation, and Reconstruction

Inferring 3D structure of a generic object from a 2D image is a long-sta...

Joint Pose and Shape Estimation of Vehicles from LiDAR Data

We address the problem of estimating the pose and shape of vehicles from...

SE(3)-Equivariant Reconstruction from Light Field

Recent progress in geometric computer vision has shown significant advan...

N-SfC: Robust and Fast Shape Estimation from Caustic Images

This paper deals with the highly challenging problem of reconstructing t...

Secrets of 3D Implicit Object Shape Reconstruction in the Wild

Reconstructing high-fidelity 3D objects from sparse, partial observation...

Please sign up or login with your details

Forgot password? Click here to reset