Spiking Variational Graph Auto-Encoders for Efficient Graph Representation Learning

10/24/2022
by   Hanxuan Yang, et al.
0

Graph representation learning is a fundamental research issue and benefits a wide range of applications on graph-structured data. Conventional artificial neural network-based methods such as graph neural networks (GNNs) and variational graph auto-encoders (VGAEs) have achieved promising results in learning on graphs, but they suffer from extremely high energy consumption during training and inference stages. Inspired by the bio-fidelity and energy-efficiency of spiking neural networks (SNNs), recent methods attempt to adapt GNNs to the SNN framework by substituting spiking neurons for the activation functions. However, existing SNN-based GNN methods cannot be applied to the more general multi-node representation learning problem represented by link prediction. Moreover, these methods did not fully exploit the bio-fidelity of SNNs, as they still require costly multiply-accumulate (MAC) operations, which severely harm the energy efficiency. To address the above issues and improve energy efficiency, in this paper, we propose an SNN-based deep generative method, namely the Spiking Variational Graph Auto-Encoders (S-VGAE) for efficient graph representation learning. To deal with the multi-node problem, we propose a probabilistic decoder that generates binary latent variables as spiking node representations and reconstructs graphs via the weighted inner product. To avoid the MAC operations for energy efficiency, we further decouple the propagation and transformation layers of conventional GNN aggregators. We conduct link prediction experiments on multiple benchmark graph datasets, and the results demonstrate that our model consumes significantly lower energy with the performances superior or comparable to other ANN- and SNN-based methods for graph representation learning.

READ FULL TEXT
research
04/20/2023

Improving Graph Neural Networks on Multi-node Tasks with Labeling Tricks

In this paper, we provide a theory of using graph neural networks (GNNs)...
research
11/21/2016

Variational Graph Auto-Encoders

We introduce the variational graph auto-encoder (VGAE), a framework for ...
research
09/22/2021

Training Deep Spiking Auto-encoders without Bursting or Dying Neurons through Regularization

Spiking neural networks are a promising approach towards next-generation...
research
06/29/2023

Spiking Denoising Diffusion Probabilistic Models

Spiking neural networks (SNNs) have ultra-low energy consumption and hig...
research
06/08/2020

Graph Representation Learning Network via Adaptive Sampling

Graph Attention Network (GAT) and GraphSAGE are neural network architect...
research
06/22/2021

A Deep Latent Space Model for Graph Representation Learning

Graph representation learning is a fundamental problem for modeling rela...
research
05/14/2019

Correlated Variational Auto-Encoders

Variational Auto-Encoders (VAEs) are capable of learning latent represen...

Please sign up or login with your details

Forgot password? Click here to reset