Adaptive Cross-Layer Attention for Image Restoration

by   Yancheng Wang, et al.

Non-local attention module has been proven to be crucial for image restoration. Conventional non-local attention processes features of each layer separately, so it risks missing correlation between features among different layers. To address this problem, we propose Cross-Layer Attention (CLA) module in this paper. Instead of finding correlated key pixels within the same layer, each query pixel can attend to key pixels at previous layers of the network. In order to further enhance the learning capability and reduce the inference cost of CLA, we further propose Adaptive CLA, or ACLA, as an improved CLA. Two adaptive designs are proposed for ACLA: 1) adaptively selecting the keys for non-local attention at each layer; 2) automatically searching for the insertion locations for ACLA modules. By these two adaptive designs, ACLA dynamically selects the number of keys to be aggregated for non-local attention at layer. In addition, ACLA searches for the optimal insert positions of ACLA modules by a neural architecture search method to render a compact neural network with compelling performance. Extensive experiments on image restoration tasks, including single image super-resolution, image denoising, image demosaicing, and image compression artifacts reduction, validate the effectiveness and efficiency of ACLA.


page 2

page 3

page 5

page 7

page 8

page 10

page 12

page 13


Residual Non-local Attention Networks for Image Restoration

In this paper, we propose a residual non-local attention network for hig...

Single Image Super-Resolution via a Holistic Attention Network

Informative features play a crucial role in the single image super-resol...

Image Super-Resolution with Cross-Scale Non-Local Attention and Exhaustive Self-Exemplars Mining

Deep convolution-based single image super-resolution (SISR) networks emb...

Concatenated Attention Neural Network for Image Restoration

In this paper, we present a general framework for low-level vision tasks...

Neural Nearest Neighbors Networks

Non-local methods exploiting the self-similarity of natural signals have...

Efficient Non-Local Contrastive Attention for Image Super-Resolution

Non-Local Attention (NLA) brings significant improvement for Single Imag...

Super-resolution Reconstruction of SAR Image based on Non-Local Means Denoising Combined with BP Neural Network

In this article, we propose a super-resolution method to resolve the pro...

Please sign up or login with your details

Forgot password? Click here to reset