Translating Images into Maps

10/03/2021
by   Avishkar Saha, et al.
0

We approach instantaneous mapping, converting images to a top-down view of the world, as a translation problem. We show how a novel form of transformer network can be used to map from images and video directly to an overhead map or bird's-eye-view (BEV) of the world, in a single end-to-end network. We assume a 1-1 correspondence between a vertical scanline in the image, and rays passing through the camera location in an overhead map. This lets us formulate map generation from an image as a set of sequence-to-sequence translations. Posing the problem as translation allows the network to use the context of the image when interpreting the role of each pixel. This constrained formulation, based upon a strong physical grounding of the problem, leads to a restricted transformer network that is convolutional in the horizontal direction only. The structure allows us to make efficient use of data when training, and obtains state-of-the-art results for instantaneous mapping of three large-scale datasets, including a 15 performing methods on the nuScenes and Argoverse datasets, respectively. We make our code available on https://github.com/avishkarsaha/translating-images-into-maps.

READ FULL TEXT
research
05/05/2022

Cross-view Transformers for real-time Map-view Semantic Segmentation

We present cross-view transformers, an efficient attention-based model f...
research
12/14/2022

Trust, but Verify: Cross-Modality Fusion for HD Map Change Detection

High-definition (HD) map change detection is the task of determining whe...
research
03/07/2023

F2BEV: Bird's Eye View Generation from Surround-View Fisheye Camera Images for Automated Driving

Bird's Eye View (BEV) representations are tremendously useful for percep...
research
11/27/2022

BEV-Locator: An End-to-end Visual Semantic Localization Network Using Multi-View Images

Accurate localization ability is fundamental in autonomous driving. Trad...
research
08/06/2021

Bird's-Eye-View Panoptic Segmentation Using Monocular Frontal View Images

Bird's-Eye-View (BEV) maps have emerged as one of the most powerful repr...
research
07/13/2022

Trans4Map: Revisiting Holistic Top-down Mapping from Egocentric Images to Allocentric Semantics with Vision Transformers

Humans have an innate ability to sense their surroundings, as they can e...
research
12/03/2021

An Automatic Approach for Generating Rich, Linked Geo-Metadata from Historical Map Images

Historical maps contain detailed geographic information difficult to fin...

Please sign up or login with your details

Forgot password? Click here to reset