Self-supervised Geometric Perception

03/04/2021
by   Heng Yang, et al.
16

We present self-supervised geometric perception (SGP), the first general framework to learn a feature descriptor for correspondence matching without any ground-truth geometric model labels (e.g., camera poses, rigid transformations). Our first contribution is to formulate geometric perception as an optimization problem that jointly optimizes the feature descriptor and the geometric models given a large corpus of visual measurements (e.g., images, point clouds). Under this optimization formulation, we show that two important streams of research in vision, namely robust model fitting and deep feature learning, correspond to optimizing one block of the unknown variables while fixing the other block. This analysis naturally leads to our second contribution – the SGP algorithm that performs alternating minimization to solve the joint optimization. SGP iteratively executes two meta-algorithms: a teacher that performs robust model fitting given learned features to generate geometric pseudo-labels, and a student that performs deep feature learning under noisy supervision of the pseudo-labels. As a third contribution, we apply SGP to two perception problems on large-scale real datasets, namely relative camera pose estimation on MegaDepth and point cloud registration on 3DMatch. We demonstrate that SGP achieves state-of-the-art performance that is on-par or superior to the supervised oracles trained using ground-truth labels.

READ FULL TEXT

page 7

page 8

page 16

page 17

research
04/28/2020

Learning Feature Descriptors using Camera Pose Supervision

Recent research on learned visual descriptors has shown promising improv...
research
06/01/2021

Bootstrap Your Own Correspondences

Geometric feature extraction is a crucial component of point cloud regis...
research
03/02/2023

Neural Intrinsic Embedding for Non-rigid Point Cloud Matching

As a primitive 3D data representation, point clouds are prevailing in 3D...
research
05/18/2021

Self-Point-Flow: Self-Supervised Scene Flow Estimation from Point Clouds with Optimal Transport and Random Walk

Due to the scarcity of annotated scene flow data, self-supervised scene ...
research
04/05/2022

SNUG: Self-Supervised Neural Dynamic Garments

We present a self-supervised method to learn dynamic 3D deformations of ...
research
07/09/2019

UnsuperPoint: End-to-end Unsupervised Interest Point Detector and Descriptor

It is hard to create consistent ground truth data for interest points in...
research
03/10/2021

Dynamical Pose Estimation

We study the problem of aligning two sets of 3D geometric primitives giv...

Please sign up or login with your details

Forgot password? Click here to reset