Adversarial Self-Supervised Scene Flow Estimation

by   Victor Zuanazzi, et al.

This work proposes a metric learning approach for self-supervised scene flow estimation. Scene flow estimation is the task of estimating 3D flow vectors for consecutive 3D point clouds. Such flow vectors are fruitful, for recognizing actions, or avoiding collisions. Training a neural network via supervised learning for scene flow is impractical, as this requires manual annotations for each 3D point at each new timestamp for each scene. To that end, we seek for a self-supervised approach, where a network learns a latent metric to distinguish between points translated by flow estimations and the target point cloud. Our adversarial metric learning includes a multi-scale triplet loss on sequences of two-point clouds as well as a cycle consistency loss. Furthermore, we outline a benchmark for self-supervised scene flow estimation: the Scene Flow Sandbox. The benchmark consists of five datasets designed to study individual aspects of flow estimation in progressive order of complexity, from a moving object to real-world scenes. Experimental evaluation on the benchmark shows that our approach obtains state-of-the-art self-supervised scene flow results, outperforming recent neighbor-based approaches. We use our proposed benchmark to expose shortcomings and draw insights on various training setups. We find that our setup captures motion coherence and preserves local geometries. Dealing with occlusions, on the other hand, is still an open challenge.



There are no comments yet.


page 1

page 2

page 3

page 4


Scene Flow from Point Clouds with or without Learning

Scene flow is the three-dimensional (3D) motion field of a scene. It pro...

Self-Supervised Learning of Non-Rigid Residual Flow and Ego-Motion

Most of the current scene flow methods choose to model scene flow as a p...

Occlusion Guided Self-supervised Scene Flow Estimation on 3D Point Clouds

Understanding the flow in 3D space of sparsely sampled points between tw...

PointPWC-Net: A Coarse-to-Fine Network for Supervised and Self-Supervised Scene Flow Estimation on 3D Point Clouds

We propose a novel end-to-end deep scene flow model, called PointPWC-Net...

FlowNet3D++: Geometric Losses For Deep Scene Flow Estimation

We present FlowNet3D++, a deep scene flow estimation network. Inspired b...

Self-Point-Flow: Self-Supervised Scene Flow Estimation from Point Clouds with Optimal Transport and Random Walk

Due to the scarcity of annotated scene flow data, self-supervised scene ...

FlowStep3D: Model Unrolling for Self-Supervised Scene Flow Estimation

Estimating the 3D motion of points in a scene, known as scene flow, is a...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.