Learning to Look around Objects for Top-View Representations of Outdoor Scenes

03/28/2018
by   Samuel Schulter, et al.
0

Given a single RGB image of a complex outdoor road scene in the perspective view, we address the novel problem of estimating an occlusion-reasoned semantic scene layout in the top-view. This challenging problem not only requires an accurate understanding of both the 3D geometry and the semantics of the visible scene, but also of occluded areas. We propose a convolutional neural network that learns to predict occluded portions of the scene layout by looking around foreground objects like cars or pedestrians. But instead of hallucinating RGB values, we show that directly predicting the semantics and depths in the occluded areas enables a better transformation into the top-view. We further show that this initial top-view representation can be significantly enhanced by learning priors and rules about typical road layouts from simulated or, if available, map data. Crucially, training our model does not require costly or subjective human annotations for occluded areas or the top-view, but rather uses readily available annotations for standard semantic segmentation. We extensively evaluate and analyze our approach on the KITTI and Cityscapes data sets.

READ FULL TEXT

page 10

page 13

page 21

page 23

page 24

page 25

page 26

page 27

research
02/19/2020

MonoLayout: Amodal scene layout from a single image

In this paper, we address the novel, highly challenging problem of estim...
research
04/14/2021

Weakly But Deeply Supervised Occlusion-Reasoned Parametric Layouts

We propose an end-to-end network that takes a single perspective RGB ima...
research
12/14/2018

A Parametric Top-View Representation of Complex Road Scenes

In this paper, we address the problem of inferring the layout of complex...
research
07/23/2018

Peeking Behind Objects: Layered Depth Prediction from a Single Image

While conventional depth estimation can infer the geometry of a scene fr...
research
08/26/2019

Object-Driven Multi-Layer Scene Decomposition From a Single Image

We present a method that tackles the challenge of predicting color and d...
research
07/30/2017

Occlusion Handling using Semantic Segmentation and Visibility-Based Rendering for Mixed Reality

Real-time occlusion handling is a major problem in outdoor mixed reality...
research
05/29/2018

Semantic Road Layout Understanding by Generative Adversarial Inpainting

Autonomous driving is becoming a reality, yet vehicles still need to rel...

Please sign up or login with your details

Forgot password? Click here to reset