RAFT-3D: Scene Flow using Rigid-Motion Embeddings

12/01/2020
by   Zachary Teed, et al.
0

We address the problem of scene flow: given a pair of stereo or RGB-D video frames, estimate pixelwise 3D motion. We introduce RAFT-3D, a new deep architecture for scene flow. RAFT-3D is based on the RAFT model developed for optical flow but iteratively updates a dense field of pixelwise SE3 motion instead of 2D motion. A key innovation of RAFT-3D is rigid-motion embeddings, which represent a soft grouping of pixels into rigid objects. Integral to rigid-motion embeddings is Dense-SE3, a differentiable layer that enforces geometric consistency of the embeddings. Experiments show that RAFT-3D achieves state-of-the-art performance. On FlyingThings3D, under the two-view evaluation, we improved the best published accuracy (d < 0.05) from 30.33 KITTI, we achieve an error of 5.77, outperforming the best published method (6.31), despite using no object instance supervision.

READ FULL TEXT

page 6

page 7

page 9

research
01/11/2021

Learning to Segment Rigid Motions from Two Frames

Appearance-based detectors achieve remarkable performance on common scen...
research
10/05/2017

Multiframe Scene Flow with Piecewise Rigid Motion

We introduce a novel multiframe scene flow approach that jointly optimiz...
research
07/26/2017

Cascaded Scene Flow Prediction using Semantic Segmentation

Given two consecutive frames from a pair of stereo cameras, 3D scene flo...
research
04/18/2019

Deep Rigid Instance Scene Flow

In this paper we tackle the problem of scene flow estimation in the cont...
research
02/17/2021

Weakly Supervised Learning of Rigid 3D Scene Flow

We propose a data-driven scene flow estimation algorithm exploiting the ...
research
09/07/2016

Dense Motion Estimation for Smoke

Motion estimation for highly dynamic phenomena such as smoke is an open ...
research
07/26/2019

Differential Scene Flow from Light Field Gradients

This paper presents novel techniques for recovering 3D dense scene flow,...

Please sign up or login with your details

Forgot password? Click here to reset