Pix2Vex: Image-to-Geometry Reconstruction using a Smooth Differentiable Renderer

03/26/2019
by   Felix Petersen, et al.
0

We present a novel approach to 3D object reconstruction from its 2D projections. Our unique, GAN-inspired system employs a novel C^∞ smooth differentiable renderer. Unlike the state-of-the-art, our renderer does not display any discontinuities at occlusions and dis-occlusions, facilitating training without 3D supervision and only minimal 2D supervision. Through domain adaptation and a novel training scheme, our network, the Reconstructive Adversarial Network (RAN), is able to train on different types of images. In contrast, previous work can only train on images of a similar appearance to those rendered by a differentiable renderer. We validate our reconstruction method through three shape classes from ShapeNet, and demonstrate that our method is robust to perturbations in view directions, different lighting conditions, and levels of texture details.

READ FULL TEXT

page 2

page 4

page 5

page 7

page 8

research
10/20/2021

Style Agnostic 3D Reconstruction via Adversarial Style Transfer

Reconstructing the 3D geometry of an object from an image is a major cha...
research
11/08/2017

MarrNet: 3D Shape Reconstruction via 2.5D Sketches

3D object reconstruction from a single image is a highly under-determine...
research
12/21/2020

Unsupervised Domain Adaptation with Temporal-Consistent Self-Training for 3D Hand-Object Joint Reconstruction

Deep learning-solutions for hand-object 3D pose and shape estimation are...
research
09/27/2018

Geometry-Aware Network for Non-Rigid Shape Prediction from a Single View

We propose a method for predicting the 3D shape of a deformable surface ...
research
07/19/2022

VoloGAN: Adversarial Domain Adaptation for Synthetic Depth Data

We present VoloGAN, an adversarial domain adaptation network that transl...
research
01/03/2013

A Self-Organizing Neural Scheme for Door Detection in Different Environments

Doors are important landmarks for indoor mobile robot navigation and als...
research
01/19/2019

Learning single-image 3D reconstruction by generative modelling of shape, pose and shading

We present a unified framework tackling two problems: class-specific 3D ...

Please sign up or login with your details

Forgot password? Click here to reset