Multi-view Consistency as Supervisory Signal for Learning Shape and Pose Prediction

01/11/2018
by   Shubham Tulsiani, et al.
0

We present a framework for learning single-view shape and pose prediction without using direct supervision for either. Our approach allows leveraging multi-view observations from unknown poses as supervisory signal during training. Our proposed training setup enforces geometric consistency between the independently predicted shape and pose from two views of the same instance. We consequently learn to predict shape in an emergent canonical (view-agnostic) frame along with a corresponding pose predictor. We show empirical and qualitative results using the ShapeNet dataset and observe encouragingly competitive performance to previous techniques which rely on stronger forms of supervision. We also demonstrate the applicability of our framework in a realistic setting which is beyond the scope of existing techniques: using a training dataset comprised of online product images where the underlying shape and pose are unknown.

READ FULL TEXT

page 1

page 2

page 6

page 8

page 12

research
05/05/2020

From Image Collections to Point Clouds with Self-supervised Shape and Pose Networks

Reconstructing 3D models from 2D images is one of the fundamental proble...
research
04/03/2020

Learning Pose-invariant 3D Object Reconstruction from Single-view Images

Learning to reconstruct 3D shapes using 2D images is an active research ...
research
05/07/2019

Learning Unsupervised Multi-View Stereopsis via Robust Photometric Consistency

We present a learning based approach for multi-view stereopsis (MVS). Wh...
research
09/10/2019

FreiHAND: A Dataset for Markerless Capture of Hand Pose and Shape from Single RGB Images

Estimating 3D hand pose from single RGB images is a highly ambiguous pro...
research
04/21/2022

Share With Thy Neighbors: Single-View Reconstruction by Cross-Instance Consistency

Approaches for single-view reconstruction typically rely on viewpoint an...
research
02/06/2018

Toward Marker-free 3D Pose Estimation in Lifting: A Deep Multi-view Solution

Lifting is a common manual material handling task performed in the workp...
research
08/03/2016

Detailed Garment Recovery from a Single-View Image

Most recent garment capturing techniques rely on acquiring multiple view...

Please sign up or login with your details

Forgot password? Click here to reset