Frequency Domain Transformer Networks for Video Prediction

03/01/2019
by   Hafez Farazi, et al.
0

The task of video prediction is forecasting the next frames given some previous frames. Despite much recent progress, this task is still challenging mainly due to high nonlinearity in the spatial domain. To address this issue, we propose a novel architecture, Frequency Domain Transformer Network (FDTN), which is an end-to-end learnable model that estimates and uses the transformations of the signal in the frequency domain. Experimental evaluations show that this approach can outperform some widely used video prediction methods like Video Ladder Network (VLN) and Predictive Gated Pyramids (PGP).

READ FULL TEXT

page 1

page 2

page 3

page 4

research
04/18/2020

Motion Segmentation using Frequency Domain Transformer Networks

Self-supervised prediction is a powerful mechanism to learn representati...
research
07/18/2019

Video Prediction for Precipitation Nowcasting

Video prediction, which aims to synthesize new consecutive frames subseq...
research
05/10/2021

Local Frequency Domain Transformer Networks for Video Prediction

Video prediction is commonly referred to as forecasting future frames of...
research
10/06/2021

Semantic Prediction: Which One Should Come First, Recognition or Prediction?

The ultimate goal of video prediction is not forecasting future pixel-va...
research
10/09/2020

Video Quality Enhancement Using Deep Learning-Based Prediction Models for Quantized DCT Coefficients in MPEG I-frames

Recent works have successfully applied some types of Convolutional Neura...
research
08/07/2023

A Hybrid CNN-Transformer Architecture with Frequency Domain Contrastive Learning for Image Deraining

Image deraining is a challenging task that involves restoring degraded i...
research
07/27/2022

One-Trimap Video Matting

Recent studies made great progress in video matting by extending the suc...

Please sign up or login with your details

Forgot password? Click here to reset