Reconstructing Vechicles from a Single Image: Shape Priors for Road Scene Understanding

09/29/2016
by   J. Krishna Murthy, et al.
0

We present an approach for reconstructing vehicles from a single (RGB) image, in the context of autonomous driving. Though the problem appears to be ill-posed, we demonstrate that prior knowledge about how 3D shapes of vehicles project to an image can be used to reason about the reverse process, i.e., how shapes (back-)project from 2D to 3D. We encode this knowledge in shape priors, which are learnt over a small keypoint-annotated dataset. We then formulate a shape-aware adjustment problem that uses the learnt shape priors to recover the 3D pose and shape of a query object from an image. For shape representation and inference, we leverage recent successes of Convolutional Neural Networks (CNNs) for the task of object and keypoint localization, and train a novel cascaded fully-convolutional architecture to localize vehicle keypoints in images. The shape-aware adjustment then robustly recovers shape (3D locations of the detected keypoints) while simultaneously filling in occluded keypoints. To tackle estimation errors incurred due to erroneously detected keypoints, we use an Iteratively Re-weighted Least Squares (IRLS) scheme for robust optimization, and as a by-product characterize noise models for each predicted keypoint. We evaluate our approach on autonomous driving benchmarks, and present superior results to existing monocular, as well as stereo approaches.

READ FULL TEXT

page 1

page 2

page 7

research
02/10/2018

Vehicle Three-Dimensional Pose and Shape Estimation from Multiple Monocular Vision

This paper proposes an accurate approach to estimate vehicles' 3D pose a...
research
04/22/2019

DirectShape: Photometric Alignment of Shape Priors for Visual Vehicle Pose and Shape Estimation

3D scene understanding from images is a challenging problem which is enc...
research
01/11/2019

Mono3D++: Monocular 3D Vehicle Detection with Two-Scale 3D Hypotheses and Task Priors

We present a method to infer 3D pose and shape of vehicles from a single...
research
11/04/2017

Object-Centric Photometric Bundle Adjustment with Deep Shape Prior

Reconstructing 3D shapes from a sequence of images has long been a probl...
research
07/22/2021

Pose Estimation and 3D Reconstruction of Vehicles from Stereo-Images Using a Subcategory-Aware Shape Prior

The 3D reconstruction of objects is a prerequisite for many highly relev...
research
02/21/2021

Probabilistic Vehicle Reconstruction Using a Multi-Task CNN

The retrieval of the 3D pose and shape of objects from images is an ill-...
research
04/29/2021

HandsFormer: Keypoint Transformer for Monocular 3D Pose Estimation ofHands and Object in Interaction

We propose a robust and accurate method for estimating the 3D poses of t...

Please sign up or login with your details

Forgot password? Click here to reset