PerMO: Perceiving More at Once from a Single Image for Autonomous Driving

07/16/2020
by   Feixiang Lu, et al.
6

We present a novel approach to detect, segment, and reconstruct complete textured 3D models of vehicles from a single image for autonomous driving. Our approach combines the strengths of deep learning and the elegance of traditional techniques from part-based deformable model representation to produce high-quality 3D models in the presence of severe occlusions. We present a new part-based deformable vehicle model that is used for instance segmentation and automatically generate a dataset that contains dense correspondences between 2D images and 3D models. We also present a novel end-to-end deep neural network to predict dense 2D/3D mapping and highlight its benefits. Based on the dense mapping, we are able to compute precise 6-DoF poses and 3D reconstruction results at almost interactive rates on a commodity GPU. We have integrated these algorithms with an autonomous driving system. In practice, our method outperforms the state-of-the-art methods for all major vehicle parsing tasks: 2D instance segmentation by 4.4 points (mAP), 6-DoF pose estimation by 9.11 points, and 3D detection by 1.37. Moreover, we have released all of the source code, dataset, and the trained model on Github.

READ FULL TEXT

page 2

page 3

page 4

page 8

page 11

page 12

page 13

page 15

research
03/11/2021

Robust 2D/3D Vehicle Parsing in CVIS

We present a novel approach to robustly detect and perceive vehicles in ...
research
12/15/2020

Fine-Grained Vehicle Perception via 3D Part-Guided Visual Data Augmentation

Holistically understanding an object and its 3D movable parts through vi...
research
11/04/2021

Towards Panoptic 3D Parsing for Single Image in the Wild

Performing single image holistic understanding and 3D reconstruction is ...
research
01/01/2020

Multi-lane Detection Using Instance Segmentation and Attentive Voting

Autonomous driving is becoming one of the leading industrial research ar...
research
08/24/2023

Panoptic-Depth Color Map for Combination of Depth and Image Segmentation

Image segmentation and depth estimation are crucial tasks in computer vi...
research
07/07/2021

Learning Stixel-based Instance Segmentation

Stixels have been successfully applied to a wide range of vision tasks i...
research
03/15/2019

BLVD: Building A Large-scale 5D Semantics Benchmark for Autonomous Driving

In autonomous driving community, numerous benchmarks have been establish...

Please sign up or login with your details

Forgot password? Click here to reset