Learning optical flow from still images

04/08/2021
by   Filippo Aleotti, et al.
2

This paper deals with the scarcity of data for training optical flow networks, highlighting the limitations of existing sources such as labeled synthetic datasets or unlabeled real videos. Specifically, we introduce a framework to generate accurate ground-truth optical flow annotations quickly and in large amounts from any readily available single real picture. Given an image, we use an off-the-shelf monocular depth estimation network to build a plausible point cloud for the observed scene. Then, we virtually move the camera in the reconstructed environment with known motion vectors and rotation angles, allowing us to synthesize both a novel view and the corresponding optical flow field connecting each pixel in the input image to the one in the new frame. When trained with our data, state-of-the-art optical flow networks achieve superior generalization to unseen real data compared to the same models trained either on annotated synthetic datasets or unlabeled videos, and better specialization if combined with synthetic images.

READ FULL TEXT

page 1

page 3

page 4

page 7

research
12/12/2016

Hybrid Learning of Optical Flow and Next Frame Prediction to Boost Optical Flow in the Wild

CNN-based optical flow estimation has attracted attention recently, main...
research
12/01/2018

From Third Person to First Person: Dataset and Baselines for Synthesis and Retrieval

First-person (egocentric) and third person (exocentric) videos are drast...
research
04/02/2021

Optical Flow Dataset Synthesis from Unpaired Images

The estimation of optical flow is an ambiguous task due to the lack of c...
research
03/31/2020

Distilled Semantics for Comprehensive Scene Understanding from Videos

Whole understanding of the surroundings is paramount to autonomous syste...
research
12/06/2021

Controllable Animation of Fluid Elements in Still Images

We propose a method to interactively control the animation of fluid elem...
research
08/20/2018

FusionNet and AugmentedFlowNet: Selective Proxy Ground Truth for Training on Unlabeled Images

Recent work has shown that convolutional neural networks (CNNs) can be u...
research
03/13/2011

SO(3)-invariant asymptotic observers for dense depth field estimation based on visual data and known camera motion

In this paper, we use known camera motion associated to a video sequence...

Please sign up or login with your details

Forgot password? Click here to reset