Poly-NL: Linear Complexity Non-local Layers with Polynomials

07/06/2021
by   Francesca Babiloni, et al.
8

Spatial self-attention layers, in the form of Non-Local blocks, introduce long-range dependencies in Convolutional Neural Networks by computing pairwise similarities among all possible positions. Such pairwise functions underpin the effectiveness of non-local layers, but also determine a complexity that scales quadratically with respect to the input size both in space and time. This is a severely limiting factor that practically hinders the applicability of non-local blocks to even moderately sized inputs. Previous works focused on reducing the complexity by modifying the underlying matrix operations, however in this work we aim to retain full expressiveness of non-local layers while keeping complexity linear. We overcome the efficiency limitation of non-local blocks by framing them as special cases of 3rd order polynomial functions. This fact enables us to formulate novel fast Non-Local blocks, capable of reducing the complexity from quadratic to linear with no loss in performance, by replacing any direct computation of pairwise similarities with element-wise multiplications. The proposed method, which we dub as "Poly-NL", is competitive with state-of-the-art performance across image recognition, instance segmentation, and face detection tasks, while having considerably less computational overhead.

READ FULL TEXT

page 7

page 8

research
11/21/2017

Non-local Neural Networks

Both convolutional and recurrent operations are building blocks that pro...
research
11/07/2020

Non-local convolutional neural networks (nlcnn) for speaker recognition

Speaker recognition is the process of identifying a speaker based on the...
research
02/24/2023

Spatial Bias for Attention-free Non-local Neural Networks

In this paper, we introduce the spatial bias to learn global knowledge w...
research
08/22/2019

NL-LinkNet: Toward Lighter but More Accurate Road Extraction with Non-Local Operations

Road extraction from very high resolution satellite images is one of the...
research
10/31/2018

Compact Generalized Non-local Network

The non-local module is designed for capturing long-range spatio-tempora...
research
07/08/2020

Non-local modeling with asymptotic expansion homogenization of random materials

The aim of this study is to build a non-local homogenized model for thre...
research
08/12/2020

Representative Graph Neural Network

Non-local operation is widely explored to model the long-range dependenc...

Please sign up or login with your details

Forgot password? Click here to reset