Scene as Occupancy

by   Wenwen Tong, et al.

Human driver can easily describe the complex traffic scene by visual system. Such an ability of precise perception is essential for driver's planning. To achieve this, a geometry-aware representation that quantizes the physical 3D scene into structured grid map with semantic labels per cell, termed as 3D Occupancy, would be desirable. Compared to the form of bounding box, a key insight behind occupancy is that it could capture the fine-grained details of critical obstacles in the scene, and thereby facilitate subsequent tasks. Prior or concurrent literature mainly concentrate on a single scene completion task, where we might argue that the potential of this occupancy representation might obsess broader impact. In this paper, we propose OccNet, a multi-view vision-centric pipeline with a cascade and temporal voxel decoder to reconstruct 3D occupancy. At the core of OccNet is a general occupancy embedding to represent 3D physical world. Such a descriptor could be applied towards a wide span of driving tasks, including detection, segmentation and planning. To validate the effectiveness of this new representation and our proposed algorithm, we propose OpenOcc, the first dense high-quality 3D occupancy benchmark built on top of nuScenes. Empirical experiments show that there are evident performance gain across multiple tasks, e.g., motion planning could witness a collision rate reduction by 15 superiority of our method.


page 1

page 3

page 5

page 6

page 8

page 12

page 14

page 15


Semantic Scene Completion via Integrating Instances and Scene in-the-Loop

Semantic Scene Completion aims at reconstructing a complete 3D scene wit...

Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3D

The goal of perception for autonomous vehicles is to extract semantic re...

Voxurf: Voxel-based Efficient and Accurate Neural Surface Reconstruction

Neural surface reconstruction aims to reconstruct accurate 3D surfaces b...

Efficient View Path Planning for Autonomous Implicit Reconstruction

Implicit neural representations have shown promising potential for the 3...

PENet: A Joint Panoptic Edge Detection Network

In recent years, compact and efficient scene understanding representatio...

DADA: A Large-scale Benchmark and Model for Driver Attention Prediction in Accidental Scenarios

Driver attention prediction has recently absorbed increasing attention i...

3D Sketch-aware Semantic Scene Completion via Semi-supervised Structure Prior

The goal of the Semantic Scene Completion (SSC) task is to simultaneousl...

Please sign up or login with your details

Forgot password? Click here to reset