PNEN: Pyramid Non-Local Enhanced Networks

08/22/2020
by   Feida Zhu, et al.
10

Existing neural networks proposed for low-level image processing tasks are usually implemented by stacking convolution layers with limited kernel size. Every convolution layer merely involves in context information from a small local neighborhood. More contextual features can be explored as more convolution layers are adopted. However it is difficult and costly to take full advantage of long-range dependencies. We propose a novel non-local module, Pyramid Non-local Block, to build up connection between every pixel and all remain pixels. The proposed module is capable of efficiently exploiting pairwise dependencies between different scales of low-level structures. The target is fulfilled through first learning a query feature map with full resolution and a pyramid of reference feature maps with downscaled resolutions. Then correlations with multi-scale reference features are exploited for enhancing pixel-level feature representation. The calculation procedure is economical considering memory consumption and computational cost. Based on the proposed module, we devise a Pyramid Non-local Enhanced Networks for edge-preserving image smoothing which achieves state-of-the-art performance in imitating three classical image smoothing algorithms. Additionally, the pyramid non-local block can be directly incorporated into convolution neural networks for other image restoration tasks. We integrate it into two existing methods for image denoising and single image super-resolution, achieving consistently improved performance.

READ FULL TEXT

page 1

page 3

page 6

page 7

page 8

page 9

research
06/09/2020

PNL: Efficient Long-Range Dependencies Extraction with Pyramid Non-Local Module for Action Recognition

Long-range spatiotemporal dependencies capturing plays an essential role...
research
06/13/2021

Feedback Pyramid Attention Networks for Single Image Super-Resolution

Recently, convolutional neural network (CNN) based image super-resolutio...
research
07/18/2020

Feature Pyramid Transformer

Feature interactions across space and scales underpin modern visual reco...
research
11/29/2018

Efficient Coarse-to-Fine Non-Local Module for the Detection of Small Objects

An image is not just a collection of objects, but rather a graph where e...
research
07/27/2022

Rethinking Efficacy of Softmax for Lightweight Non-Local Neural Networks

Non-local (NL) block is a popular module that demonstrates the capabilit...
research
10/30/2018

Neural Nearest Neighbors Networks

Non-local methods exploiting the self-similarity of natural signals have...
research
04/07/2017

Convolutional Neural Pyramid for Image Processing

We propose a principled convolutional neural pyramid (CNP) framework for...

Please sign up or login with your details

Forgot password? Click here to reset