Dynamic Graph Message Passing Networks for Visual Recognition

09/20/2022
by   Li Zhang, et al.
FUDAN University
17

Modelling long-range dependencies is critical for scene understanding tasks in computer vision. Although convolution neural networks (CNNs) have excelled in many vision tasks, they are still limited in capturing long-range structured relationships as they typically consist of layers of local kernels. A fully-connected graph, such as the self-attention operation in Transformers, is beneficial for such modelling, however, its computational overhead is prohibitive. In this paper, we propose a dynamic graph message passing network, that significantly reduces the computational complexity compared to related works modelling a fully-connected graph. This is achieved by adaptively sampling nodes in the graph, conditioned on the input, for message passing. Based on the sampled nodes, we dynamically predict node-dependent filter weights and the affinity matrix for propagating information between them. This formulation allows us to design a self-attention module, and more importantly a new Transformer-based backbone network, that we use for both image classification pretraining, and for addressing various downstream tasks (object detection, instance and semantic segmentation). Using this model, we show significant improvements with respect to strong, state-of-the-art baselines on four different tasks. Our approach also outperforms fully-connected graphs while using substantially fewer floating-point operations and parameters. Code and models will be made publicly available at https://github.com/fudan-zvg/DGMN2

READ FULL TEXT

page 2

page 8

page 10

page 17

page 18

page 19

08/19/2019

Dynamic Graph Message Passing Networks

Modelling long-range dependencies is critical for complex scene understa...
05/13/2023

DRew: Dynamically Rewired Message Passing with Delay

Message passing neural networks (MPNNs) have been shown to suffer from t...
08/05/2021

Unifying Nonlocal Blocks for Neural Networks

The nonlocal-based blocks are designed for capturing long-range spatial-...
12/21/2021

RepMLPNet: Hierarchical Vision MLP with Re-parameterized Locality

Compared to convolutional layers, fully-connected (FC) layers are better...
09/21/2023

SALSA-CLRS: A Sparse and Scalable Benchmark for Algorithmic Reasoning

We introduce an extension to the CLRS algorithmic learning benchmark, pr...
11/23/2022

EurNet: Efficient Multi-Range Relational Modeling of Spatial Multi-Relational Data

Modeling spatial relationship in the data remains critical across many d...
02/25/2019

Star-Transformer

Although the fully-connected attention-based model Transformer has achie...

Code Repositories

DGMN2

[TPAMI 2022 & CVPR 2020 Oral] Dynamic Graph Message Passing Networks


view repo

Please sign up or login with your details

Forgot password? Click here to reset