DeepAI
Log In Sign Up

Comparing Correspondences: Video Prediction with Correspondence-wise Losses

04/19/2021
by   Daniel Geng, et al.
0

Today's image prediction methods struggle to change the locations of objects in a scene, producing blurry images that average over the many positions they might occupy. In this paper, we propose a simple change to existing image similarity metrics that makes them more robust to positional errors: we match the images using optical flow, then measure the visual similarity of corresponding pixels. This change leads to crisper and more perceptually accurate predictions, and can be used with any image prediction network. We apply our method to predicting future frames of a video, where it obtains strong performance with simple, off-the-shelf architectures.

READ FULL TEXT

page 1

page 4

page 6

page 7

page 14

05/30/2018

Novel Video Prediction for Large-scale Scene using Optical Flow

Making predictions of future frames is a critical challenge in autonomou...
11/09/2017

Predicting Scene Parsing and Motion Dynamics in the Future

The ability of predicting the future is important for intelligent system...
02/08/2022

Learning Optical Flow with Adaptive Graph Reasoning

Estimating per-pixel motion between video frames, known as optical flow,...
11/10/2017

Predicting Chroma from Luma in AV1

Chroma from luma (CfL) prediction is a new and promising chroma-only int...
12/07/2017

Multi-Scale Video Frame-Synthesis Network with Transitive Consistency Loss

Traditional approaches to interpolating/extrapolating frames in a video ...
02/02/2015

Learning the Matching Function

The matching function for the problem of stereo reconstruction or optica...
10/28/2020

GloFlow: Global Image Alignment for Creation of Whole Slide Images for Pathology from Video

The application of deep learning to pathology assumes the existence of d...