Convolutional Block Design for Learned Fractional Downsampling

05/20/2021
by   Li-Heng Chen, et al.
0

The layers of convolutional neural networks (CNNs) can be used to alter the resolution of their inputs, but the scaling factors are limited to integer values. However, in many image and video processing applications, the ability to resize by a fractional factor would be advantageous. One example is conversion between resolutions standardized for video compression, such as from 1080p to 720p. To solve this problem, we propose an alternative building block, formulated as a conventional convolutional layer followed by a differentiable resizer. More concretely, the convolutional layer preserves the resolution of the input, while the resizing operation is fully handled by the resizer. In this way, any CNN architecture can be adapted for non-integer resizing. As an application, we replace the resizing convolutional layer of a modern deep downsampling model by the proposed building block, and apply it to an adaptive bitrate video streaming scenario. Our experimental results show that an improvement in coding efficiency over the conventional Lanczos algorithm is attained, in terms of PSNR, SSIM, and VMAF on test videos.

READ FULL TEXT
research
03/20/2022

Fully Convolutional Fractional Scaling

We introduce a fully convolutional fractional scaling component, FCFS. F...
research
03/30/2020

BVI-DVC: A Training Database for Deep Video Compression

Deep learning methods are increasingly being applied in the optimisation...
research
06/07/2023

Video Compression with Arbitrary Rescaling Network

Most video platforms provide video streaming services with different qua...
research
02/01/2022

Fractional Motion Estimation for Point Cloud Compression

Motivated by the success of fractional pixel motion in video coding, we ...
research
08/30/2023

Neural Video Compression with Temporal Layer-Adaptive Hierarchical B-frame Coding

Neural video compression (NVC) is a rapidly evolving video coding resear...
research
06/19/2020

From Discrete to Continuous Convolution Layers

A basic operation in Convolutional Neural Networks (CNNs) is spatial res...
research
08/18/2020

PRNU Estimation from Encoded Videos Using Block-Based Weighting

Estimating the photo-response non-uniformity (PRNU) of an imaging sensor...

Please sign up or login with your details

Forgot password? Click here to reset