Adversarial Self-Supervised Scene Flow Estimation

11/01/2020
by   Victor Zuanazzi, et al.
9

This work proposes a metric learning approach for self-supervised scene flow estimation. Scene flow estimation is the task of estimating 3D flow vectors for consecutive 3D point clouds. Such flow vectors are fruitful, for recognizing actions, or avoiding collisions. Training a neural network via supervised learning for scene flow is impractical, as this requires manual annotations for each 3D point at each new timestamp for each scene. To that end, we seek for a self-supervised approach, where a network learns a latent metric to distinguish between points translated by flow estimations and the target point cloud. Our adversarial metric learning includes a multi-scale triplet loss on sequences of two-point clouds as well as a cycle consistency loss. Furthermore, we outline a benchmark for self-supervised scene flow estimation: the Scene Flow Sandbox. The benchmark consists of five datasets designed to study individual aspects of flow estimation in progressive order of complexity, from a moving object to real-world scenes. Experimental evaluation on the benchmark shows that our approach obtains state-of-the-art self-supervised scene flow results, outperforming recent neighbor-based approaches. We use our proposed benchmark to expose shortcomings and draw insights on various training setups. We find that our setup captures motion coherence and preserves local geometries. Dealing with occlusions, on the other hand, is still an open challenge.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/31/2020

Scene Flow from Point Clouds with or without Learning

Scene flow is the three-dimensional (3D) motion field of a scene. It pro...
research
09/22/2020

Self-Supervised Learning of Non-Rigid Residual Flow and Ego-Motion

Most of the current scene flow methods choose to model scene flow as a p...
research
11/25/2022

SCOOP: Self-Supervised Correspondence and Optimization-Based Scene Flow

Scene flow estimation is a long-standing problem in computer vision, whe...
research
04/10/2021

Occlusion Guided Self-supervised Scene Flow Estimation on 3D Point Clouds

Understanding the flow in 3D space of sparsely sampled points between tw...
research
05/04/2023

Self-Supervised 3D Scene Flow Estimation Guided by Superpoints

3D scene flow estimation aims to estimate point-wise motions between two...
research
05/18/2021

Self-Point-Flow: Self-Supervised Scene Flow Estimation from Point Clouds with Optimal Transport and Random Walk

Due to the scarcity of annotated scene flow data, self-supervised scene ...
research
12/03/2019

FlowNet3D++: Geometric Losses For Deep Scene Flow Estimation

We present FlowNet3D++, a deep scene flow estimation network. Inspired b...

Please sign up or login with your details

Forgot password? Click here to reset