A Spectral Nonlocal Block for Neural Networks

11/04/2019
by   Lei Zhu, et al.
21

The nonlocal network is designed for capturing long-range spatial-temporal dependencies in several computer vision tasks. Although having shown excellent performances, it needs an elaborate preparation for both the number and position of the building blocks. In this paper, we propose a new formulation of the nonlocal block and interpret it from the general graph signal processing perspective, where we view it as a fully-connected graph filter approximated by Chebyshev polynomials. The proposed nonlocal block is more efficient and robust, which is a generalized form of existing nonlocal blocks (e.g. nonlocal block, nonlocal stage). Moreover, we give the stable hypothesis and show that the steady-state of the deeper nonlocal structure should meet with it. Based on the stable hypothesis, a full-order approximation of the nonlocal block is derived for consecutive connections. Experimental results illustrate the clear-cut improvement and practical applicability of the generalized nonlocal block on both image and video classification tasks.

READ FULL TEXT

page 5

page 9

page 15

research
08/05/2021

Unifying Nonlocal Blocks for Neural Networks

The nonlocal-based blocks are designed for capturing long-range spatial-...
research
06/02/2018

Nonlocal Neural Networks, Nonlocal Diffusion and Nonlocal Modeling

Nonlocal neural networks have been proposed and shown to be effective in...
research
11/19/2018

High Order Neural Networks for Video Classification

Capturing spatiotemporal correlations is an essential topic in video cla...
research
08/26/2022

Hardware-aware mobile building block evaluation for computer vision

In this work we propose a methodology to accurately evaluate and compare...
research
04/20/2018

Residual-Guide Feature Fusion Network for Single Image Deraining

Single image rain streaks removal is extremely important since rainy ima...
research
04/19/2018

NAIS-Net: Stable Deep Networks from Non-Autonomous Differential Equations

This paper introduces "Non-Autonomous Input-Output Stable Network" (NAIS...
research
03/11/2022

TFCNet: Temporal Fully Connected Networks for Static Unbiased Temporal Reasoning

Temporal Reasoning is one important functionality for vision intelligenc...

Please sign up or login with your details

Forgot password? Click here to reset