Unsupervised Learning of 3D Scene Flow with 3D Odometry Assistance

09/11/2022
by   Guangming Wang, et al.
0

Scene flow represents the 3D motion of each point in the scene, which explicitly describes the distance and the direction of each point's movement. Scene flow estimation is used in various applications such as autonomous driving fields, activity recognition, and virtual reality fields. As it is challenging to annotate scene flow with ground truth for real-world data, this leaves no real-world dataset available to provide a large amount of data with ground truth for scene flow estimation. Therefore, many works use synthesized data to pre-train their network and real-world LiDAR data to finetune. Unlike the previous unsupervised learning of scene flow in point clouds, we propose to use odometry information to assist the unsupervised learning of scene flow and use real-world LiDAR data to train our network. Supervised odometry provides more accurate shared cost volume for scene flow. In addition, the proposed network has mask-weighted warp layers to get a more accurate predicted point cloud. The warp operation means applying an estimated pose transformation or scene flow to a source point cloud to obtain a predicted point cloud and is the key to refining scene flow from coarse to fine. When performing warp operations, the points in different states use different weights for the pose transformation and scene flow transformation. We classify the states of points as static, dynamic, and occluded, where the static masks are used to divide static and dynamic points, and the occlusion masks are used to divide occluded points. The mask-weighted warp layer indicates that static masks and occlusion masks are used as weights when performing warp operations. Our designs are proved to be effective in ablation experiments. The experiment results show the promising prospect of an odometry-assisted unsupervised learning method for 3D scene flow in real-world data.

READ FULL TEXT

page 1

page 3

page 12

research
06/08/2022

Unsupervised Learning of 3D Scene Flow from Monocular Camera

Scene flow represents the motion of points in the 3D space, which is the...
research
09/27/2022

3D Scene Flow Estimation on Pseudo-LiDAR: Bridging the Gap on Estimating Point Motion

3D scene flow characterizes how the points at the current time flow to t...
research
12/02/2020

PWCLO-Net: Deep LiDAR Odometry in 3D Point Clouds Using Hierarchical Embedding Mask Optimization

A novel 3D point cloud learning model for deep LiDAR odometry, named PWC...
research
10/12/2020

Hierarchical Attention Learning of Scene Flow in 3D Point Clouds

Scene flow represents the 3D motion of every point in the dynamic enviro...
research
10/07/2022

GMA3D: Local-Global Attention Learning to Estimate Occluded Motions of Scene Flow

Scene flow is the collection of each point motion information in the 3D ...
research
03/01/2021

Scalable Scene Flow from Point Clouds in the Real World

Autonomous vehicles operate in highly dynamic environments necessitating...
research
11/01/2021

Neural Scene Flow Prior

Before the deep learning revolution, many perception algorithms were bas...

Please sign up or login with your details

Forgot password? Click here to reset