Beyond Brightness Constancy: Learning Noise Models for Optical Flow

by   Dan Rosenbaum, et al.

Optical flow is typically estimated by minimizing a "data cost" and an optional regularizer. While there has been much work on different regularizers many modern algorithms still use a data cost that is not very different from the ones used over 30 years ago: a robust version of brightness constancy or gradient constancy. In this paper we leverage the recent availability of ground-truth optical flow databases in order to learn a data cost. Specifically we take a generative approach in which the data cost models the distribution of noise after warping an image according to the flow and we measure the "goodness" of a data cost by how well it matches the true distribution of flow warp error. Consistent with current practice, we find that robust versions of gradient constancy are better models than simple brightness constancy but a learned GMM that models the density of patches of warp error gives a much better fit than any existing assumption of constancy. This significant advantage of the GMM is due to an explicit modeling of the spatial structure of warp errors, a feature which is missing from almost all existing data costs in optical flow. Finally, we show how a good density model of warp error patches can be used for optical flow estimation on whole images. We replace the data cost by the expected patch log-likelihood (EPLL), and show how this cost can be optimized iteratively using an additional step of denoising the warp error image. The results of our experiments are promising and show that patch models with higher likelihood lead to better optical flow estimation.


page 2

page 5

page 6

page 8


Pyramidal Gradient Matching for Optical Flow Estimation

Initializing optical flow field by either sparse descriptor matching or ...

Optical Flow Super-Resolution Based on Image Guidence Using Convolutional Neural Network

The convolutional neural network model for optical flow estimation usual...

What Causes Optical Flow Networks to be Vulnerable to Physical Adversarial Attacks

Recent work demonstrated the lack of robustness of optical flow networks...

FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks

The FlowNet demonstrated that optical flow estimation can be cast as a l...

Bregman Iteration for Correspondence Problems: A Study of Optical Flow

Bregman iterations are known to yield excellent results for denoising, d...

Estimating motion with principal component regression strategies

In this paper, two simple principal component regression methods for est...

A qualitative investigation of optical flow algorithms for video denoising

A good optical flow estimation is crucial in many video analysis and res...

Please sign up or login with your details

Forgot password? Click here to reset