End-to-end optimized image compression for machines, a study

11/10/2020
by   Lahiru D. Chamain, et al.
0

An increasing share of image and video content is analyzed by machines rather than viewed by humans, and therefore it becomes relevant to optimize codecs for such applications where the analysis is performed remotely. Unfortunately, conventional coding tools are challenging to specialize for machine tasks as they were originally designed for human perception. However, neural network based codecs can be jointly trained end-to-end with any convolutional neural network (CNN)-based task model. In this paper, we propose to study an end-to-end framework enabling efficient image compression for remote machine task analysis, using a chain composed of a compression module and a task algorithm that can be optimized end-to-end. We show that it is possible to significantly improve the task accuracy when fine-tuning jointly the codec and the task networks, especially at low bit-rates. Depending on training or deployment constraints, selective fine-tuning can be applied only on the encoder, decoder or task network and still achieve rate-accuracy improvements over an off-the-shelf codec and task network. Our results also demonstrate the flexibility of end-to-end pipelines for practical applications.

READ FULL TEXT
research
03/06/2021

End-to-end optimized image compression for multiple machine tasks

An increasing share of captured images and videos are transmitted for st...
research
07/28/2020

Efficient Adaptation of Neural Network Filter for Video Compression

We present an efficient finetuning methodology for neural-network filter...
research
04/04/2019

An End-to-End Baseline for Video Captioning

Building correspondences across different modalities, such as video and ...
research
12/04/2018

Content Authentication for Neural Imaging Pipelines: End-to-end Optimization of Photo Provenance in Complex Distribution Channels

Forensic analysis of digital photo provenance relies on intrinsic traces...
research
03/30/2017

Learning Convolutional Networks for Content-weighted Image Compression

Lossy image compression is generally formulated as a joint rate-distorti...
research
02/10/2020

Learning End-to-End Lossy Image Compression: A Benchmark

Image compression is one of the most fundamental techniques and commonly...
research
02/27/2019

Neural Imaging Pipelines - the Scourge or Hope of Forensics?

Forensic analysis of digital photographs relies on intrinsic statistical...

Please sign up or login with your details

Forgot password? Click here to reset