Multi-Modality Deep Network for JPEG Artifacts Reduction

05/04/2023
by   Xuhao Jiang, et al.
0

In recent years, many convolutional neural network-based models are designed for JPEG artifacts reduction, and have achieved notable progress. However, few methods are suitable for extreme low-bitrate image compression artifacts reduction. The main challenge is that the highly compressed image loses too much information, resulting in reconstructing high-quality image difficultly. To address this issue, we propose a multimodal fusion learning method for text-guided JPEG artifacts reduction, in which the corresponding text description not only provides the potential prior information of the highly compressed image, but also serves as supplementary information to assist in image deblocking. We fuse image features and text semantic features from the global and local perspectives respectively, and design a contrastive loss built upon contrastive learning to produce visually pleasing results. Extensive experiments, including a user study, prove that our method can obtain better deblocking results compared to the state-of-the-art methods.

READ FULL TEXT

page 5

page 12

page 13

page 14

page 15

page 16

page 17

page 18

research
04/26/2023

Multi-Modality Deep Network for Extreme Learned Image Compression

Image-based single-modality compression learning approaches have demonst...
research
11/15/2016

One-to-Many Network for Visually Pleasing Compression Artifacts Reduction

We consider the compression artifacts reduction problem, where a compres...
research
12/22/2022

Restoring Vision in Hazy Weather with Hierarchical Contrastive Learning

Image restoration under hazy weather condition, which is called single i...
research
04/12/2022

CLMLF:A Contrastive Learning and Multi-Layer Fusion Method for Multimodal Sentiment Detection

Compared with unimodal data, multimodal data can provide more features t...
research
11/14/2022

Extreme Generative Image Compression by Learning Text Embedding from Diffusion Models

Transferring large amount of high resolution images over limited bandwid...
research
10/06/2022

Prompt Compression and Contrastive Conditioning for Controllability and Toxicity Reduction in Language Models

We explore the idea of compressing the prompts used to condition languag...
research
11/21/2022

High-Perceptual Quality JPEG Decoding via Posterior Sampling

JPEG is arguably the most popular image coding format, achieving high co...

Please sign up or login with your details

Forgot password? Click here to reset