3DSRnet: Video Super-resolution using 3D Convolutional Neural Networks

12/21/2018
by   Soo Ye Kim, et al.
10

In video super-resolution, the spatio-temporal coherence between, and among the frames must be exploited appropriately for accurate prediction of the high resolution frames. Although 2D convolutional neural networks (CNNs) are powerful in modelling images, 3D-CNNs are more suitable for spatio-temporal feature extraction as they can preserve temporal information. To this end, we propose an effective 3D-CNN for video super-resolution, called the 3DSRnet that does not require motion alignment as preprocessing. Our 3DSRnet maintains the temporal depth of spatio-temporal feature maps to maximally capture the temporally nonlinear characteristics between low and high resolution frames, and adopts residual learning in conjunction with the sub-pixel outputs. It outperforms the most state-of-the-art method with average 0.45 and 0.36 dB higher in PSNR for scales 3 and 4, respectively, in the Vidset4 benchmark. Our 3DSRnet first deals with the performance drop due to scene change, which is important in practice but has not been previously considered.

READ FULL TEXT

page 7

page 8

page 9

research
11/16/2016

Real-Time Video Super-Resolution with Spatio-Temporal Networks and Motion Compensation

Convolutional neural networks have enabled accurate image super-resoluti...
research
03/02/2019

Spatio-Temporal Vegetation Pixel Classification By Using Convolutional Networks

Plant phenology studies rely on long-term monitoring of life cycles of p...
research
01/28/2022

Deep Networks for Image and Video Super-Resolution

Efficiency of gradient propagation in intermediate layers of convolution...
research
10/24/2019

Spatiotemporal Tile-based Attention-guided LSTMs for Traffic Video Prediction

This extended abstract describes our solution for the Traffic4Cast Chall...
research
05/01/2020

MeshfreeFlowNet: A Physics-Constrained Deep Continuous Space-Time Super-Resolution Framework

We propose MeshfreeFlowNet, a novel deep learning-based super-resolution...
research
04/14/2017

CBinfer: Change-Based Inference for Convolutional Neural Networks on Video Data

Extracting per-frame features using convolutional neural networks for re...
research
12/07/2022

RainUNet for Super-Resolution Rain Movie Prediction under Spatio-temporal Shifts

This paper presents a solution to the Weather4cast 2022 Challenge Stage ...

Please sign up or login with your details

Forgot password? Click here to reset