DeepAI
Log In Sign Up

A New Dataset and Transformer for Stereoscopic Video Super-Resolution

04/21/2022
by   Hassan Imani, et al.
0

Stereo video super-resolution (SVSR) aims to enhance the spatial resolution of the low-resolution video by reconstructing the high-resolution video. The key challenges in SVSR are preserving the stereo-consistency and temporal-consistency, without which viewers may experience 3D fatigue. There are several notable works on stereoscopic image super-resolution, but there is little research on stereo video super-resolution. In this paper, we propose a novel Transformer-based model for SVSR, namely Trans-SVSR. Trans-SVSR comprises two key novel components: a spatio-temporal convolutional self-attention layer and an optical flow-based feed-forward layer that discovers the correlation across different video frames and aligns the features. The parallax attention mechanism (PAM) that uses the cross-view information to consider the significant disparities is used to fuse the stereo views. Due to the lack of a benchmark dataset suitable for the SVSR task, we collected a new stereoscopic video dataset, SVSR-Set, containing 71 full high-definition (HD) stereo videos captured using a professional stereo camera. Extensive experiments on the collected dataset, along with two other datasets, demonstrate that the Trans-SVSR can achieve competitive performance compared to the state-of-the-art methods. Project code and additional results are available at https://github.com/H-deep/Trans-SVSR/

READ FULL TEXT

page 3

page 8

06/12/2021

Video Super-Resolution Transformer

Video super-resolution (VSR), with the aim to restore a high-resolution ...
11/21/2022

FlowLens: Seeing Beyond the FoV via Flow-guided Clip-Recurrent Transformer

Limited by hardware cost and system size, camera's Field-of-View (FoV) i...
08/04/2022

H2-Stereo: High-Speed, High-Resolution Stereoscopic Video System

High-speed, high-resolution stereoscopic (H2-Stereo) video allows us to ...
05/20/2022

Combining Contrastive and Supervised Learning for Video Super-Resolution Detection

Upscaled video detection is a helpful tool in multimedia forensics, but ...
03/26/2020

Holopix50k: A Large-Scale In-the-wild Stereo Image Dataset

With the mass-market adoption of dual-camera mobile phones, leveraging s...
07/18/2022

Boosting Video Super Resolution with Patch-Based Temporal Redundancy Optimization

The success of existing video super-resolution (VSR) algorithms stems ma...
07/12/2021

Real-Time Super-Resolution System of 4K-Video Based on Deep Learning

Video super-resolution (VSR) technology excels in reconstructing low-qua...