FVC: A New Framework towards Deep Video Compression in Feature Space

05/20/2021
by   Zhihao Hu, et al.
0

Learning based video compression attracts increasing attention in the past few years. The previous hybrid coding approaches rely on pixel space operations to reduce spatial and temporal redundancy, which may suffer from inaccurate motion estimation or less effective motion compensation. In this work, we propose a feature-space video coding network (FVC) by performing all major operations (i.e., motion estimation, motion compression, motion compensation and residual compression) in the feature space. Specifically, in the proposed deformable compensation module, we first apply motion estimation in the feature space to produce motion information (i.e., the offset maps), which will be compressed by using the auto-encoder style network. Then we perform motion compensation by using deformable convolution and generate the predicted feature. After that, we compress the residual feature between the feature from the current frame and the predicted feature from our deformable compensation module. For better frame reconstruction, the reference features from multiple previous reconstructed frames are also fused by using the non-local attention mechanism in the multi-frame feature fusion module. Comprehensive experimental results demonstrate that the proposed framework achieves the state-of-the-art performance on four benchmark datasets including HEVC, UVG, VTL and MCL-JCV.

READ FULL TEXT

page 3

page 8

research
09/21/2023

Spatial-Temporal Transformer based Video Compression Framework

Learned video compression (LVC) has witnessed remarkable advancements in...
research
07/11/2022

Learned Video Compression via Heterogeneous Deformable Compensation Network

Learned video compression has recently emerged as an essential research ...
research
05/26/2021

DFPN: Deformable Frame Prediction Network

Learned frame prediction is a current problem of interest in computer vi...
research
07/12/2012

Supervised Texture Classification Using a Novel Compression-Based Similarity Measure

Supervised pixel-based texture classification is usually performed in th...
research
06/15/2022

Coarse-to-fine Deep Video Coding with Hyperprior-guided Mode Prediction

The previous deep video compression approaches only use the single scale...
research
04/13/2021

Spatiotemporal Entropy Model is All You Need for Learned Video Compression

The framework of dominant learned video compression methods is usually c...
research
05/02/2022

D-DPCC: Deep Dynamic Point Cloud Compression via 3D Motion Prediction

The non-uniformly distributed nature of the 3D dynamic point cloud (DPC)...

Please sign up or login with your details

Forgot password? Click here to reset