U2-Former: A Nested U-shaped Transformer for Image Restoration

12/04/2021
by   Haobo Ji, et al.
0

While Transformer has achieved remarkable performance in various high-level vision tasks, it is still challenging to exploit the full potential of Transformer in image restoration. The crux lies in the limited depth of applying Transformer in the typical encoder-decoder framework for image restoration, resulting from heavy self-attention computation load and inefficient communications across different depth (scales) of layers. In this paper, we present a deep and effective Transformer-based network for image restoration, termed as U2-Former, which is able to employ Transformer as the core operation to perform image restoration in a deep encoding and decoding space. Specifically, it leverages the nested U-shaped structure to facilitate the interactions across different layers with different scales of feature maps. Furthermore, we optimize the computational efficiency for the basic Transformer block by introducing a feature-filtering mechanism to compress the token representation. Apart from the typical supervision ways for image restoration, our U2-Former also performs contrastive learning in multiple aspects to further decouple the noise component from the background image. Extensive experiments on various image restoration tasks, including reflection removal, rain streak removal and dehazing respectively, demonstrate the effectiveness of the proposed U2-Former.

READ FULL TEXT

page 4

page 6

page 7

research
06/06/2021

Uformer: A General U-Shaped Transformer for Image Restoration

In this paper, we present Uformer, an effective and efficient Transforme...
research
08/20/2023

WMFormer++: Nested Transformer for Visible Watermark Removal via Implict Joint Learning

Watermarking serves as a widely adopted approach to safeguard media copy...
research
10/09/2020

Deep-Masking Generative Network: A Unified Framework for Background Restoration from Superimposed Images

Restoring the clean background from the superimposed images containing a...
research
08/17/2023

Learning A Coarse-to-Fine Diffusion Transformer for Image Restoration

Recent years have witnessed the remarkable performance of diffusion mode...
research
09/06/2023

Prompt-based All-in-One Image Restoration using CNNs and Transformer

Image restoration aims to recover the high-quality images from their deg...
research
04/06/2023

Towards an Effective and Efficient Transformer for Rain-by-snow Weather Removal

Rain-by-snow weather removal is a specialized task in weather-degraded i...
research
02/19/2023

Mixed Hierarchy Network for Image Restoration

Image restoration is a long-standing low-level vision problem, e.g., deb...

Please sign up or login with your details

Forgot password? Click here to reset