MinkLoc++: Lidar and Monocular Image Fusion for Place Recognition

04/12/2021
by   Jacek Komorowski, et al.
0

We introduce a discriminative multimodal descriptor based on a pair of sensor readings: a point cloud from a LiDAR and an image from an RGB camera. Our descriptor, named MinkLoc++, can be used for place recognition, re-localization and loop closure purposes in robotics or autonomous vehicles applications. We use late fusion approach, where each modality is processed separately and fused in the final part of the processing pipeline. The proposed method achieves state-of-the-art performance on standard place recognition benchmarks. We also identify dominating modality problem when training a multimodal descriptor. The problem manifests itself when the network focuses on a modality with a larger overfit to the training data. This drives the loss down during the training but leads to suboptimal performance on the evaluation set. In this work we describe how to detect and mitigate such risk when using a deep metric learning approach to train a multimodal neural network. Our code is publicly available on the project website: https://github.com/jac99/MinkLocMultimodal.

READ FULL TEXT

page 1

page 6

research
11/09/2020

MinkLoc3D: Point Cloud Based Large-Scale Place Recognition

The paper presents a learning-based method for computing a discriminativ...
research
03/01/2023

ORCHNet: A Robust Global Feature Aggregation approach for 3D LiDAR-based Place recognition in Orchards

Robust and reliable place recognition and loop closure detection in agri...
research
06/25/2022

FreSCo: Frequency-Domain Scan Context for LiDAR-based Place Recognition with Translation and Rotation Invariance

Place recognition plays a crucial role in re-localization and loop closu...
research
09/16/2022

SeqOT: A Spatial-Temporal Transformer Network for Place Recognition Using Sequential LiDAR Data

Place recognition is an important component for autonomous vehicles to a...
research
06/17/2021

AttDLNet: Attention-based DL Network for 3D LiDAR Place Recognition

Deep networks have been progressively adapted to new sensor modalities, ...
research
10/04/2022

Uncertainty-Aware Lidar Place Recognition in Novel Environments

State-of-the-art approaches to lidar place recognition degrade significa...
research
02/03/2023

CVTNet: A Cross-View Transformer Network for Place Recognition Using LiDAR Data

LiDAR-based place recognition (LPR) is one of the most crucial component...

Please sign up or login with your details

Forgot password? Click here to reset