Foveation-based Deep Video Compression without Motion Search

03/30/2022
by   Meixu Chen, et al.
0

The requirements of much larger file sizes, different storage formats, and immersive viewing conditions of VR pose significant challenges to the goals of acquiring, transmitting, compressing, and displaying high-quality VR content. At the same time, the great potential of deep learning to advance progress on the video compression problem has driven a significant research effort. Because of the high bandwidth requirements of VR, there has also been significant interest in the use of space-variant, foveated compression protocols. We have integrated these techniques to create an end-to-end deep learning video compression framework. A feature of our new compression model is that it dispenses with the need for expensive search-based motion prediction computations. This is accomplished by exploiting statistical regularities inherent in video motion expressed by displaced frame differences. Foveation protocols are desirable since only a small portion of a video viewed in VR may be visible as a user gazes in any given direction. Moreover, even within a current field of view (FOV), the resolution of retinal neurons rapidly decreases with distance (eccentricity) from the projected point of gaze. In our learning based approach, we implement foveation by introducing a Foveation Generator Unit (FGU) that generates foveation masks which direct the allocation of bits, significantly increasing compression efficiency while making it possible to retain an impression of little to no additional visual loss given an appropriate viewing geometry. Our experiment results reveal that our new compression model, which we call the Foveated MOtionless VIdeo Codec (Foveated MOVI-Codec), is able to efficiently compress videos without computing motion, while outperforming foveated version of both H.264 and H.265 on the widely used UVG dataset and on the HEVC Standard Class B Test Sequences.

READ FULL TEXT

page 1

page 6

page 8

page 9

page 10

page 11

page 12

research
09/29/2020

Learning to Compress Videos without Computing Motion

With the development of higher resolution contents and displays, its sig...
research
08/28/2021

Towards Retina-Quality VR Video Streaming: 15ms Could Save You 80 Your Bandwidth

Virtual reality systems today cannot yet stream immersive, retina-qualit...
research
11/30/2018

DVC: An End-to-end Deep Video Compression Framework

Conventional video compression approaches use the predictive coding arch...
research
12/12/2017

Learning Compressible 360° Video Isomers

Standard video encoders developed for conventional narrow field-of-view ...
research
04/11/2018

Measurement of exceptional motion in VR video contents for VR sickness assessment using deep convolutional autoencoder

This paper proposes a new objective metric of exceptional motion in VR v...
research
12/01/2020

Ultra-low bitrate video conferencing using deep image animation

In this work we propose a novel deep learning approach for ultra-low bit...
research
06/22/2023

Relevance-Based Compression of Cataract Surgery Videos

In the last decade, the need for storing videos from cataract surgery ha...

Please sign up or login with your details

Forgot password? Click here to reset