Perspective Transformer Nets: Learning Single-View 3D Object Reconstruction without 3D Supervision

12/01/2016
by   Xinchen Yan, et al.
0

Understanding the 3D world is a fundamental problem in computer vision. However, learning a good representation of 3D objects is still an open problem due to the high dimensionality of the data and many factors of variation involved. In this work, we investigate the task of single-view 3D object reconstruction from a learning agent's perspective. We formulate the learning process as an interaction between 3D and 2D representations and propose an encoder-decoder network with a novel projection loss defined by the perspective transformation. More importantly, the projection loss enables the unsupervised learning using 2D observation without explicit 3D supervision. We demonstrate the ability of the model in generating 3D volume from a single 2D image with three sets of experiments: (1) learning from single-class objects; (2) learning from multi-class objects and (3) testing on novel object classes. Results show superior performance and better generalization ability for 3D object reconstruction when the projection loss is involved.

READ FULL TEXT

page 12

page 13

page 14

research
09/06/2021

Toward Realistic Single-View 3D Object Reconstruction with Unsupervised Learning from Multiple Images

Recovering the 3D structure of an object from a single image is a challe...
research
01/05/2016

Weakly-supervised Disentangling with Recurrent Transformations for 3D View Synthesis

An important problem for both graphics and vision is to synthesize novel...
research
04/14/2019

Conditional Single-view Shape Generation for Multi-view Stereo Reconstruction

In this paper, we present a new perspective towards image-based shape ge...
research
08/04/2022

ZeroMesh: Zero-shot Single-view 3D Mesh Reconstruction

Single-view 3D object reconstruction is a fundamental and challenging co...
research
04/27/2020

CoReNet: Coherent 3D scene reconstruction from a single RGB image

Advances in deep learning techniques have allowed recent work to reconst...
research
08/18/2020

Contact Area Detector using Cross View Projection Consistency for COVID-19 Projects

The ability to determine what parts of objects and surfaces people touch...
research
07/10/2006

An active curve approach for tomographic reconstruction of binary radially symmetric objects

This paper deals with a method of tomographic reconstruction of radially...

Please sign up or login with your details

Forgot password? Click here to reset