MV-Map: Offboard HD-Map Generation with Multi-view Consistency

05/15/2023
by   Ziyang Xie, et al.
0

While bird's-eye-view (BEV) perception models can be useful for building high-definition maps (HD-Maps) with less human labor, their results are often unreliable and demonstrate noticeable inconsistencies in the predicted HD-Maps from different viewpoints. This is because BEV perception is typically set up in an 'onboard' manner, which restricts the computation and consequently prevents algorithms from reasoning multiple views simultaneously. This paper overcomes these limitations and advocates a more practical 'offboard' HD-Map generation setup that removes the computation constraints, based on the fact that HD-Maps are commonly reusable infrastructures built offline in data centers. To this end, we propose a novel offboard pipeline called MV-Map that capitalizes multi-view consistency and can handle an arbitrary number of frames with the key design of a 'region-centric' framework. In MV-Map, the target HD-Maps are created by aggregating all the frames of onboard predictions, weighted by the confidence scores assigned by an 'uncertainty network'. To further enhance multi-view consistency, we augment the uncertainty network with the global 3D structure optimized by a voxelized neural radiance field (Voxel-NeRF). Extensive experiments on nuScenes show that our MV-Map significantly improves the quality of HD-Maps, further highlighting the importance of offboard methods for HD-Map generation.

READ FULL TEXT

page 4

page 7

page 8

page 14

research
11/09/2021

PREMA: Part-based REcurrent Multi-view Aggregation Network for 3D Shape Retrieval

We propose the Part-based Recurrent Multi-view Aggregation network(PREMA...
research
04/17/2019

Render4Completion: Synthesizing Multi-view Depth Maps for 3D Shape Completion

We propose a novel approach for 3D shape completion by synthesizing mult...
research
03/18/2020

3D Crowd Counting via Multi-View Fusion with 3D Gaussian Kernels

Crowd counting has been studied for decades and a lot of works have achi...
research
01/15/2022

Uncertainty-Aware Multi-View Representation Learning

Learning from different data views by exploring the underlying complemen...
research
10/21/2022

Multi-View Reasoning: Consistent Contrastive Learning for Math Word Problem

Math word problem solver requires both precise relation reasoning about ...
research
07/27/2023

MapNeRF: Incorporating Map Priors into Neural Radiance Fields for Driving View Simulation

Simulating camera sensors is a crucial task in autonomous driving. Altho...
research
06/07/2023

NeMO: Neural Map Growing System for Spatiotemporal Fusion in Bird's-Eye-View and BDD-Map Benchmark

Vision-centric Bird's-Eye View (BEV) representation is essential for aut...

Please sign up or login with your details

Forgot password? Click here to reset