Log In Sign Up

Comparing Correspondences: Video Prediction with Correspondence-wise Losses

by   Daniel Geng, et al.

Today's image prediction methods struggle to change the locations of objects in a scene, producing blurry images that average over the many positions they might occupy. In this paper, we propose a simple change to existing image similarity metrics that makes them more robust to positional errors: we match the images using optical flow, then measure the visual similarity of corresponding pixels. This change leads to crisper and more perceptually accurate predictions, and can be used with any image prediction network. We apply our method to predicting future frames of a video, where it obtains strong performance with simple, off-the-shelf architectures.


page 1

page 4

page 6

page 7

page 14


Novel Video Prediction for Large-scale Scene using Optical Flow

Making predictions of future frames is a critical challenge in autonomou...

Predicting Scene Parsing and Motion Dynamics in the Future

The ability of predicting the future is important for intelligent system...

Learning Optical Flow with Adaptive Graph Reasoning

Estimating per-pixel motion between video frames, known as optical flow,...

Predicting Chroma from Luma in AV1

Chroma from luma (CfL) prediction is a new and promising chroma-only int...

Multi-Scale Video Frame-Synthesis Network with Transitive Consistency Loss

Traditional approaches to interpolating/extrapolating frames in a video ...

Learning the Matching Function

The matching function for the problem of stereo reconstruction or optica...

GloFlow: Global Image Alignment for Creation of Whole Slide Images for Pathology from Video

The application of deep learning to pathology assumes the existence of d...