Motion Estimation for Large Displacements and Deformations

06/24/2022
by   Qiao Chen, et al.
33

Large displacement optical flow is an integral part of many computer vision tasks. Variational optical flow techniques based on a coarse-to-fine scheme interpolate sparse matches and locally optimize an energy model conditioned on colour, gradient and smoothness, making them sensitive to noise in the sparse matches, deformations, and arbitrarily large displacements. This paper addresses this problem and presents HybridFlow, a variational motion estimation framework for large displacements and deformations. A multi-scale hybrid matching approach is performed on the image pairs. Coarse-scale clusters formed by classifying pixels according to their feature descriptors are matched using the clusters' context descriptors. We apply a multi-scale graph matching on the finer-scale superpixels contained within each matched pair of coarse-scale clusters. Small clusters that cannot be further subdivided are matched using localized feature matching. Together, these initial matches form the flow, which is propagated by an edge-preserving interpolation and variational refinement. Our approach does not require training and is robust to substantial displacements and rigid and non-rigid transformations due to motion in the scene, making it ideal for large-scale imagery such as Wide-Area Motion Imagery (WAMI). More notably, HybridFlow works on directed graphs of arbitrary topology representing perceptual groups, which improves motion estimation in the presence of significant deformations. We demonstrate HybridFlow's superior performance to state-of-the-art variational techniques on two benchmark datasets and report comparable results with state-of-the-art deep-learning-based techniques.

READ FULL TEXT

page 1

page 2

page 7

page 8

page 9

page 11

page 12

research
02/29/2016

FALDOI: A new minimization strategy for large displacement variational optical flow

We propose a large displacement optical flow method that introduces a ne...
research
01/12/2015

EpicFlow: Edge-Preserving Interpolation of Correspondences for Optical Flow

We propose a novel approach for optical flow estimation , targeted at la...
research
10/27/2017

SceneFlowFields: Dense Interpolation of Sparse Scene Flow Correspondences

While most scene flow methods use either variational optimization or a s...
research
05/28/2022

DeepRM: Deep Recurrent Matching for 6D Pose Refinement

Precise 6D pose estimation of rigid objects from RGB images is a critica...
research
07/23/2017

Deep Optical Flow Estimation Via Multi-Scale Correspondence Structure Learning

As an important and challenging problem in computer vision, learning bas...
research
07/25/2022

Multi-Scale RAFT: Combining Hierarchical Concepts for Learning-based Optical FLow Estimation

Many classical and learning-based optical flow methods rely on hierarchi...
research
01/31/2021

Nonlinear Evolutionary PDE-Based Refinement of Optical Flow

The goal of this paper is propose a mathematical framework for optical f...

Please sign up or login with your details

Forgot password? Click here to reset