PreCNet: Next Frame Video Prediction Based on Predictive Coding

04/30/2020
by   Zdenek Straka, et al.
0

Predictive coding, currently a highly influential theory in neuroscience, has not been widely adopted in machine learning yet. In this work, we transform the seminal model of Rao and Ballard (1999) into a modern deep learning framework while remaining maximally faithful to the original schema. The resulting network we propose (PreCNet) is tested on a widely used next frame video prediction benchmark, which consists of images from an urban environment recorded from a car-mounted camera. On this benchmark (training: 41k images from KITTI dataset; testing: Caltech Pedestrian dataset), we achieve to our knowledge the best performance to date when measured with the Structural Similarity Index (SSIM). On two other common measures, MSE and PSNR, the model ranked third and fourth, respectively. Performance was further improved when a larger training set (2M images from BDD100k), pointing to the limitations of the KITTI training set. This work demonstrates that an architecture carefully based in a neuroscience model, without being explicitly tailored to the task at hand, can exhibit unprecedented performance.

READ FULL TEXT

page 5

page 8

page 9

page 10

page 11

page 15

page 16

page 17

research
10/16/2018

Reduced-Gate Convolutional LSTM Using Predictive Coding for Spatiotemporal Prediction

Spatiotemporal sequence prediction is an important problem in deep learn...
research
07/27/2021

Predictive Coding: a Theoretical and Experimental Review

Predictive coding offers a potentially unifying account of cortical func...
research
06/24/2020

PredNet and Predictive Coding: A Critical Review

PredNet, a deep predictive coding network developed by Lotter et al., co...
research
07/09/2023

Predictive Coding For Animation-Based Video Compression

We address the problem of efficiently compressing video for conferencing...
research
05/07/2020

Hierarchical Predictive Coding Models in a Deep-Learning Framework

Bayesian predictive coding is a putative neuromorphic method for acquiri...
research
06/12/2019

Privacy-Preserving Deep Visual Recognition: An Adversarial Learning Framework and A New Dataset

This paper aims to boost privacy-preserving visual recognition, an incre...
research
06/30/2021

A Structured Analysis of the Video Degradation Effects on the Performance of a Machine Learning-enabled Pedestrian Detector

ML-enabled software systems have been incorporated in many public demons...

Please sign up or login with your details

Forgot password? Click here to reset