MMVC: Learned Multi-Mode Video Compression with Block-based Prediction Mode Selection and Density-Adaptive Entropy Coding

04/05/2023
by   Bowen Liu, et al.
0

Learning-based video compression has been extensively studied over the past years, but it still has limitations in adapting to various motion patterns and entropy models. In this paper, we propose multi-mode video compression (MMVC), a block wise mode ensemble deep video compression framework that selects the optimal mode for feature domain prediction adapting to different motion patterns. Proposed multi-modes include ConvLSTM-based feature domain prediction, optical flow conditioned feature domain prediction, and feature propagation to address a wide range of cases from static scenes without apparent motions to dynamic scenes with a moving camera. We partition the feature space into blocks for temporal prediction in spatial block-based representations. For entropy coding, we consider both dense and sparse post-quantization residual blocks, and apply optional run-length coding to sparse residuals to improve the compression rate. In this sense, our method uses a dual-mode entropy coding scheme guided by a binary density map, which offers significant rate reduction surpassing the extra cost of transmitting the binary selection map. We validate our scheme with some of the most popular benchmarking datasets. Compared with state-of-the-art video compression schemes and standard codecs, our method yields better or competitive results measured with PSNR and MS-SSIM.

READ FULL TEXT

page 2

page 5

page 7

research
06/15/2022

Coarse-to-fine Deep Video Coding with Hyperprior-guided Mode Prediction

The previous deep video compression approaches only use the single scale...
research
09/13/2020

Improving Deep Video Compression by Resolution-adaptive Flow Coding

In the learning based video compression approaches, it is an essential i...
research
07/06/2020

ModeNet: Mode Selection Network For Learned Video Coding

In this paper, a mode selection network (ModeNet) is proposed to enhance...
research
09/10/2020

Key-Point Sequence Lossless Compression for Intelligent Video Analysis

Feature coding has been recently considered to facilitate intelligent vi...
research
09/27/2017

Fast Convolutional Sparse Coding in the Dual Domain

Convolutional sparse coding (CSC) is an important building block of many...
research
12/26/2021

Learning Cross-Scale Prediction for Efficient Neural Video Compression

In this paper, we present the first neural video codec that can compete ...
research
08/06/2020

Optical Flow and Mode Selection for Learning-based Video Coding

This paper introduces a new method for inter-frame coding based on two c...

Please sign up or login with your details

Forgot password? Click here to reset