On the Impact of Lossy Image and Video Compression on the Performance of Deep Convolutional Neural Network Architectures

07/28/2020
by   Matt Poyser, et al.
12

Recent advances in generalized image understanding have seen a surge in the use of deep convolutional neural networks (CNN) across a broad range of image-based detection, classification and prediction tasks. Whilst the reported performance of these approaches is impressive, this study investigates the hitherto unapproached question of the impact of commonplace image and video compression techniques on the performance of such deep learning architectures. Focusing on the JPEG and H.264 (MPEG-4 AVC) as a representative proxy for contemporary lossy image/video compression techniques that are in common use within network-connected image/video devices and infrastructure, we examine the impact on performance across five discrete tasks: human pose estimation, semantic segmentation, object detection, action recognition, and monocular depth estimation. As such, within this study we include a variety of network architectures and domains spanning end-to-end convolution, encoder-decoder, region-based CNN (R-CNN), dual-stream, and generative adversarial networks (GAN). Our results show a non-linear and non-uniform relationship between network performance and the level of lossy compression applied. Notably, performance decreases significantly below a JPEG quality (quantization) level of 15 architectures on pre-compressed imagery conversely recovers network performance by up to 78.4 architectures employing an encoder-decoder pipeline and those that demonstrate resilience to lossy image compression. The characteristics of the relationship between input compression to output task performance can be used to inform design decisions within future image/video devices and infrastructure.

READ FULL TEXT

page 1

page 3

page 4

page 5

research
10/10/2021

Operationalizing Convolutional Neural Network Architectures for Prohibited Object Detection in X-Ray Imagery

The recent advancement in deep Convolutional Neural Network (CNN) has br...
research
01/26/2020

Deep Learning-based Image Compression with Trellis Coded Quantization

Recently many works attempt to develop image compression models based on...
research
01/27/2022

Neural JPEG: End-to-End Image Compression Leveraging a Standard JPEG Encoder-Decoder

Recent advances in deep learning have led to superhuman performance acro...
research
05/16/2022

Lost in Compression: the Impact of Lossy Image Compression on Variable Size Object Detection within Infrared Imagery

Lossy image compression strategies allow for more efficient storage and ...
research
03/22/2022

End-to-End Learned Block-Based Image Compression with Block-Level Masked Convolutions and Asymptotic Closed Loop Training

Learned image compression research has achieved state-of-the-art compres...
research
05/24/2022

Wavelet Feature Maps Compression for Image-to-Image CNNs

Convolutional Neural Networks (CNNs) are known for requiring extensive c...
research
11/26/2018

Adversarial Video Compression Guided by Soft Edge Detection

We propose a video compression framework using conditional Generative Ad...

Please sign up or login with your details

Forgot password? Click here to reset