Deformable 3D Convolution for Video Super-Resolution

04/06/2020
by   Xinyi Ying, et al.
Sohu
National University of Defense Technology
0

The spatio-temporal information among video sequences is significant for video super-resolution (SR). However, the spatio-temporal information cannot be fully used by existing video SR methods since spatial feature extraction and temporal motion compensation are usually performed sequentially. In this paper, we propose a deformable 3D convolution network (D3Dnet) to incorporate spatio-temporal information from both spatial and temporal dimensions for video SR. Specifically, we introduce deformable 3D convolutions (D3D) to integrate 2D spatial deformable convolutions with 3D convolutions (C3D), obtaining both superior spatio-temporal modeling capability and motion-aware modeling flexibility. Extensive experiments have demonstrated the effectiveness of our proposed D3D in exploiting spatio-temporal information. Comparative results show that our network outperforms the state-of-the-art methods. Code is available at: https://github.com/XinyiYing/D3Dnet.

READ FULL TEXT

page 3

page 4

01/04/2022

MoCoPnet: Exploring Local Motion and Contrast Priors for Infrared Small Target Super-Resolution

Infrared small target super-resolution (SR) aims to recover reliable and...
04/05/2019

Fast Spatio-Temporal Residual Network for Video Super-Resolution

Recently, deep learning based video super-resolution (SR) methods have a...
05/27/2021

Blind Motion Deblurring Super-Resolution: When Dynamic Spatio-Temporal Learning Meets Static Image Understanding

Single-image super-resolution (SR) and multi-frame SR are two ways to su...
11/15/2021

D^2Conv3D: Dynamic Dilated Convolutions for Object Segmentation in Videos

Despite receiving significant attention from the research community, the...
01/17/2020

Temporal Interlacing Network

For a long time, the vision community tries to learn the spatio-temporal...
07/01/2022

Motion Compensated Frequency Selective Extrapolation for Error Concealment in Video Coding

Although wireless and IP-based access to video content gives a new degre...
06/25/2020

SmallBigNet: Integrating Core and Contextual Views for Video Classification

Temporal convolution has been widely used for video classification. Howe...

Code Repositories

D3Dnet

Repository for "Deformable 3D Convolution for Video Super-Resolution", arXiv, 2020


view repo

Please sign up or login with your details

Forgot password? Click here to reset