Image Deblurring by Exploring In-depth Properties of Transformer

03/24/2023
by   Pengwei Liang, et al.
0

Image deblurring continues to achieve impressive performance with the development of generative models. Nonetheless, there still remains a displeasing problem if one wants to improve perceptual quality and quantitative scores of recovered image at the same time. In this study, drawing inspiration from the research of transformer properties, we introduce the pretrained transformers to address this problem. In particular, we leverage deep features extracted from a pretrained vision transformer (ViT) to encourage recovered images to be sharp without sacrificing the performance measured by the quantitative metrics. The pretrained transformer can capture the global topological relations (i.e., self-similarity) of image, and we observe that the captured topological relations about the sharp image will change when blur occurs. By comparing the transformer features between recovered image and target one, the pretrained transformer provides high-resolution blur-sensitive semantic information, which is critical in measuring the sharpness of the deblurred image. On the basis of the advantages, we present two types of novel perceptual losses to guide image deblurring. One regards the features as vectors and computes the discrepancy between representations extracted from recovered image and target one in Euclidean space. The other type considers the features extracted from an image as a distribution and compares the distribution discrepancy between recovered image and target one. We demonstrate the effectiveness of transformer properties in improving the perceptual quality while not sacrificing the quantitative scores (PSNR) over the most competitive models, such as Uformer, Restormer, and NAFNet, on defocus deblurring and motion deblurring tasks.

READ FULL TEXT

page 1

page 3

page 4

page 5

page 8

page 9

research
04/26/2021

Clean Images are Hard to Reblur: A New Clue for Deblurring

The goal of dynamic scene deblurring is to remove the motion blur presen...
research
04/04/2020

Deblurring by Realistic Blurring

Existing deep learning methods for image deblurring typically train mode...
research
06/25/2023

Adaptive Window Pruning for Efficient Local Motion Deblurring

Local motion blur commonly occurs in real-world photography due to the m...
research
08/01/2023

ViT2EEG: Leveraging Hybrid Pretrained Vision Transformers for EEG Data

In this study, we demonstrate the application of a hybrid Vision Transfo...
research
08/16/2021

No-Reference Image Quality Assessment via Transformers, Relative Ranking, and Self-Consistency

The goal of No-Reference Image Quality Assessment (NR-IQA) is to estimat...
research
07/20/2018

Improving Image Clustering With Multiple Pretrained CNN Feature Extractors

For many image clustering problems, replacing raw image data with featur...

Please sign up or login with your details

Forgot password? Click here to reset