DeepAI AI Chat
Log In Sign Up

Weakly-supervised Disentangling with Recurrent Transformations for 3D View Synthesis

by   Jimei Yang, et al.
University of Michigan
University of California, Merced

An important problem for both graphics and vision is to synthesize novel views of a 3D object from a single image. This is particularly challenging due to the partial observability inherent in projecting a 3D object onto the image space, and the ill-posedness of inferring object shape and pose. However, we can train a neural network to address the problem if we restrict our attention to specific object categories (in our case faces and chairs) for which we can gather ample training data. In this paper, we propose a novel recurrent convolutional encoder-decoder network that is trained end-to-end on the task of rendering rotated objects starting from a single image. The recurrent structure allows our model to capture long-term dependencies along a sequence of transformations. We demonstrate the quality of its predictions for human faces on the Multi-PIE dataset and for a dataset of 3D chair models, and also show its ability to disentangle latent factors of variation (e.g., identity and pose) without using full supervision.


Unsupervised Novel View Synthesis from a Single Image

Novel view synthesis from a single image aims at generating novel views ...

Weakly-supervised Single-view Image Relighting

We present a learning-based approach to relight a single image of Lamber...

Perspective Transformer Nets: Learning Single-View 3D Object Reconstruction without 3D Supervision

Understanding the 3D world is a fundamental problem in computer vision. ...

A Self-Supervised Bootstrap Method for Single-Image 3D Face Reconstruction

State-of-the-art methods for 3D reconstruction of faces from a single im...

View Extrapolation of Human Body from a Single Image

We study how to synthesize novel views of human body from a single image...

Im2Struct: Recovering 3D Shape Structure from a Single RGB Image

We propose to recover 3D shape structures from single RGB images, where ...

Transformable Bottleneck Networks

We propose a novel approach to performing fine-grained 3D manipulation o...