2D3D-MatchNet: Learning to Match Keypoints Across 2D Image and 3D Point Cloud

04/22/2019
by   Mengdan Feng, et al.
0

Large-scale point cloud generated from 3D sensors is more accurate than its image-based counterpart. However, it is seldom used in visual pose estimation due to the difficulty in obtaining 2D-3D image to point cloud correspondences. In this paper, we propose the 2D3D-MatchNet - an end-to-end deep network architecture to jointly learn the descriptors for 2D and 3D keypoint from image and point cloud, respectively. As a result, we are able to directly match and establish 2D-3D correspondences from the query image and 3D point cloud reference map for visual pose estimation. We create our Oxford 2D-3D Patches dataset from the Oxford Robotcar dataset with the ground truth camera poses and 2D-3D image to point cloud correspondences for training and testing the deep network. Experimental results verify the feasibility of our approach.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
04/10/2018

PointNetVLAD: Deep Point Cloud Based Retrieval for Large-Scale Place Recognition

Unlike its image based counterpart, point cloud based retrieval for plac...
research
09/14/2023

EP2P-Loc: End-to-End 3D Point to 2D Pixel Localization for Large-Scale Visual Localization

Visual localization is the task of estimating a 6-DoF camera pose of a q...
research
11/19/2019

Single-Stage 6D Object Pose Estimation

Most recent 6D pose estimation frameworks first rely on a deep network t...
research
08/14/2021

PICCOLO: Point Cloud-Centric Omnidirectional Localization

We present PICCOLO, a simple and efficient algorithm for omnidirectional...
research
11/08/2016

Deep Convolutional Neural Network for 6-DOF Image Localization

We present an accurate and robust method for six degree of freedom image...
research
03/28/2022

REGTR: End-to-end Point Cloud Correspondences with Transformers

Despite recent success in incorporating learning into point cloud regist...
research
07/15/2021

Unsupervised 3D Human Mesh Recovery from Noisy Point Clouds

This paper presents a novel unsupervised approach to reconstruct human s...

Please sign up or login with your details

Forgot password? Click here to reset