DeepAI AI Chat
Log In Sign Up

Simplicial Attention Networks

04/20/2022
by   Christopher Wei Jin Goh, et al.
11

Graph representation learning methods have mostly been limited to the modelling of node-wise interactions. Recently, there has been an increased interest in understanding how higher-order structures can be utilised to further enhance the learning abilities of graph neural networks (GNNs) in combinatorial spaces. Simplicial Neural Networks (SNNs) naturally model these interactions by performing message passing on simplicial complexes, higher-dimensional generalisations of graphs. Nonetheless, the computations performed by most existent SNNs are strictly tied to the combinatorial structure of the complex. Leveraging the success of attention mechanisms in structured domains, we propose Simplicial Attention Networks (SAT), a new type of simplicial network that dynamically weighs the interactions between neighbouring simplicies and can readily adapt to novel structures. Additionally, we propose a signed attention mechanism that makes SAT orientation equivariant, a desirable property for models operating on (co)chain complexes. We demonstrate that SAT outperforms existent convolutional SNNs and GNNs in two image and trajectory classification tasks.

READ FULL TEXT
04/16/2022

Theory of Graph Neural Networks: Representation and Learning

Graph Neural Networks (GNNs), neural network architectures targeted to l...
06/23/2021

Weisfeiler and Lehman Go Cellular: CW Networks

Graph Neural Networks (GNNs) are limited in their expressive power, stru...
05/27/2020

Neural heuristics for SAT solving

We use neural graph networks with a message-passing architecture and an ...
06/01/2022

Higher-Order Attention Networks

This paper introduces higher-order attention networks (HOANs), a novel c...
02/21/2023

Reusable Slotwise Mechanisms

Agents that can understand and reason over the dynamics of objects can h...
03/17/2022

Graph Representation Learning with Individualization and Refinement

Graph Neural Networks (GNNs) have emerged as prominent models for repres...
09/13/2022

Characterizing Graph Datasets for Node Classification: Beyond Homophily-Heterophily Dichotomy

Homophily is a graph property describing the tendency of edges to connec...

Code Repositories

Simplicial-neural-network-benchmark

Simplicial neural network benchmarking software


view repo