Reference-based Image Super-Resolution with Deformable Attention Transformer

07/25/2022
by   Jiezhang Cao, et al.
0

Reference-based image super-resolution (RefSR) aims to exploit auxiliary reference (Ref) images to super-resolve low-resolution (LR) images. Recently, RefSR has been attracting great attention as it provides an alternative way to surpass single image SR. However, addressing the RefSR problem has two critical challenges: (i) It is difficult to match the correspondence between LR and Ref images when they are significantly different; (ii) How to transfer the relevant texture from Ref images to compensate the details for LR images is very challenging. To address these issues of RefSR, this paper proposes a deformable attention Transformer, namely DATSR, with multiple scales, each of which consists of a texture feature encoder (TFE) module, a reference-based deformable attention (RDA) module and a residual feature aggregation (RFA) module. Specifically, TFE first extracts image transformation (e.g., brightness) insensitive features for LR and Ref images, RDA then can exploit multiple relevant textures to compensate more information for LR features, and RFA lastly aggregates LR features and relevant textures to get a more visually pleasant result. Extensive experiments demonstrate that our DATSR achieves state-of-the-art performance on benchmark datasets quantitatively and qualitatively.

READ FULL TEXT

page 2

page 11

page 12

research
06/07/2020

Learning Texture Transformer Network for Image Super-Resolution

We study on image super-resolution (SR), which aims to recover realistic...
research
06/04/2021

MASA-SR: Matching Acceleration and Spatial Adaptation for Reference-Based Image Super-Resolution

Reference-based image super-resolution (RefSR) has shown promising succe...
research
12/19/2022

Reference-based Image and Video Super-Resolution via C2-Matching

Reference-based Super-Resolution (Ref-SR) has recently emerged as a prom...
research
05/10/2023

Reference-based OCT Angiogram Super-resolution with Learnable Texture Generation

Optical coherence tomography angiography (OCTA) is a new imaging modalit...
research
11/08/2022

RRSR:Reciprocal Reference-based Image Super-Resolution with Progressive Feature Alignment and Selection

Reference-based image super-resolution (RefSR) is a promising SR branch ...
research
06/03/2021

Robust Reference-based Super-Resolution via C2-Matching

Reference-based Super-Resolution (Ref-SR) has recently emerged as a prom...
research
04/14/2023

A Unified HDR Imaging Method with Pixel and Patch Level

Mapping Low Dynamic Range (LDR) images with different exposures to High ...

Please sign up or login with your details

Forgot password? Click here to reset