EurNet: Efficient Multi-Range Relational Modeling of Spatial Multi-Relational Data

11/23/2022
by   Minghao Xu, et al.
4

Modeling spatial relationship in the data remains critical across many different tasks, such as image classification, semantic segmentation and protein structure understanding. Previous works often use a unified solution like relative positional encoding. However, there exists different kinds of spatial relations, including short-range, medium-range and long-range relations, and modeling them separately can better capture the focus of different tasks on the multi-range relations (e.g., short-range relations can be important in instance segmentation, while long-range relations should be upweighted for semantic segmentation). In this work, we introduce the EurNet for Efficient multi-range relational modeling. EurNet constructs the multi-relational graph, where each type of edge corresponds to short-, medium- or long-range spatial interactions. In the constructed graph, EurNet adopts a novel modeling layer, called gated relational message passing (GRMP), to propagate multi-relational information across the data. GRMP captures multiple relations within the data with little extra computational cost. We study EurNets in two important domains for image and protein structure modeling. Extensive experiments on ImageNet classification, COCO object detection and ADE20K semantic segmentation verify the gains of EurNet over the previous SoTA FocalNet. On the EC and GO protein function prediction benchmarks, EurNet consistently surpasses the previous SoTA GearNet. Our results demonstrate the strength of EurNets on modeling spatial multi-relational data from various domains. The implementations of EurNet for image modeling are available at https://github.com/hirl-team/EurNet-Image . The implementations for other applied domains/tasks will be released soon.

READ FULL TEXT

page 5

page 9

research
12/07/2020

Rethinking Learnable Tree Filter for Generic Feature Transform

The Learnable Tree Filter presents a remarkable approach to model struct...
research
11/28/2018

CCNet: Criss-Cross Attention for Semantic Segmentation

Long-range dependencies can capture useful contextual information to ben...
research
01/14/2019

Long range actions, connectedness, and dismantlability in relational structures

In this paper we study alternative characterizations of dismantlability ...
research
09/20/2022

Dynamic Graph Message Passing Networks for Visual Recognition

Modelling long-range dependencies is critical for scene understanding ta...
research
04/11/2022

Panoptic, Instance and Semantic Relations: A Relational Context Encoder to Enhance Panoptic Segmentation

This paper presents a novel framework to integrate both semantic and ins...
research
04/11/2019

A Relation-Augmented Fully Convolutional Network for Semantic Segmentationin Aerial Scenes

Most current semantic segmentation approaches fall back on deep convolut...
research
02/17/2021

LambdaNetworks: Modeling Long-Range Interactions Without Attention

We present lambda layers – an alternative framework to self-attention – ...

Please sign up or login with your details

Forgot password? Click here to reset