DeepAI AI Chat
Log In Sign Up

JPAD-SE: High-Level Semantics for Joint Perception-Accuracy-Distortion Enhancement in Image Compression

by   Shiyu Duan, et al.

While humans can effortlessly transform complex visual scenes into simple words and the other way around by leveraging their high-level understanding of the content, conventional or the more recent learned image compression codecs do not seem to utilize the semantic meanings of visual content to its full potential. Moreover, they focus mostly on rate-distortion and tend to underperform in perception quality especially in low bitrate regime, and often disregard the performance of downstream computer vision algorithms, which is a fast-growing consumer group of compressed images in addition to human viewers. In this paper, we (1) present a generic framework that can enable any image codec to leverage high-level semantics, and (2) study the joint optimization of perception quality, accuracy of downstream computer vision task, and distortion. Our idea is that given any codec, we utilize high-level semantics to augment the low-level visual features extracted by it and produce essentially a new, semantic-aware codec. And we argue that semantic enhancement implicitly optimizes rate-perception-accuracy-distortion (R-PAD) performance. To validate our claim, we perform extensive empirical evaluations and provide both quantitative and qualitative results.


page 24

page 25

page 26

page 27

page 28

page 29

page 31

page 32


Learned Image Compression for Machine Perception

Recent work has shown that learned image compression strategies can outp...

Discernible Compressed Images via Deep Perception Consistency

Image compression, as one of the fundamental low-level image processing ...

TACTIC: Joint Rate-Distortion-Accuracy Optimisation for Low Bitrate Compression

We present TACTIC: Task-Aware Compression Through Intelligent Coding. Ou...

On The Classification-Distortion-Perception Tradeoff

Signal degradation is ubiquitous and computational restoration of degrad...

Mid-Level Visual Representations Improve Generalization and Sample Efficiency for Learning Active Tasks

One of the ultimate promises of computer vision is to help robotic agent...

FishEyeRecNet: A Multi-Context Collaborative Deep Network for Fisheye Image Rectification

Images captured by fisheye lenses violate the pinhole camera assumption ...

SLLEN: Semantic-aware Low-light Image Enhancement Network

How to effectively explore semantic feature is vital for low-light image...