Fast Multi-frame Stereo Scene Flow with Motion Segmentation

07/05/2017
by   Tatsunori Taniai, et al.
0

We propose a new multi-frame method for efficiently computing scene flow (dense depth and optical flow) and camera ego-motion for a dynamic scene observed from a moving stereo camera rig. Our technique also segments out moving objects from the rigid scene. In our method, we first estimate the disparity map and the 6-DOF camera motion using stereo matching and visual odometry. We then identify regions inconsistent with the estimated camera motion and compute per-pixel optical flow only at these regions. This flow proposal is fused with the camera motion-based flow proposal using fusion moves to obtain the final optical flow and motion segmentation. This unified framework benefits all four tasks - stereo, optical flow, visual odometry and motion segmentation leading to overall higher accuracy and efficiency. Our method is currently ranked third on the KITTI 2015 scene flow benchmark. Furthermore, our CPU implementation runs in 2-3 seconds per frame which is 1-3 orders of magnitude faster than the top six methods. We also report a thorough evaluation on challenging Sintel sequences with fast camera and object motion, where our method consistently outperforms OSF [Menze and Geiger, 2015], which is currently ranked second on the KITTI benchmark.

READ FULL TEXT

page 1

page 2

page 4

page 5

page 6

page 7

page 8

page 13

research
09/16/2019

Learning Residual Flow as Dynamic Motion from Stereo Videos

We present a method for decomposing the 3D scene flow observed from a mo...
research
04/14/2021

VOLDOR: Visual Odometry from Log-logistic Dense Optical flow Residuals

We propose a dense indirect visual odometry method taking as input exter...
research
08/02/2021

Neural Image Representations for Multi-Image Fusion and Layer Separation

We propose a framework for aligning and fusing multiple images into a si...
research
07/12/2022

M-FUSE: Multi-frame Fusion for Scene Flow Estimation

Recently, neural network for scene flow estimation show impressive resul...
research
11/16/2020

EffiScene: Efficient Per-Pixel Rigidity Inference for Unsupervised Joint Learning of Optical Flow, Depth, Camera Pose and Motion Segmentation

This paper addresses the challenging unsupervised scene flow estimation ...
research
07/26/2017

Cascaded Scene Flow Prediction using Semantic Segmentation

Given two consecutive frames from a pair of stereo cameras, 3D scene flo...
research
10/14/2020

Semantic Flow-guided Motion Removal Method for Robust Mapping

Moving objects in scenes are still a severe challenge for the SLAM syste...

Please sign up or login with your details

Forgot password? Click here to reset