Log In Sign Up

A Self-Attention Network based Node Embedding Model

by   Dai Quoc Nguyen, et al.

Despite several signs of progress have been made recently, limited research has been conducted for an inductive setting where embeddings are required for newly unseen nodes – a setting encountered commonly in practical applications of deep learning for graph networks. This significantly affects the performances of downstream tasks such as node classification, link prediction or community extraction. To this end, we propose SANNE – a novel unsupervised embedding model – whose central idea is to employ a transformer self-attention network to iteratively aggregate vector representations of nodes in random walks. Our SANNE aims to produce plausible embeddings not only for present nodes, but also for newly unseen nodes. Experimental results show that the proposed SANNE obtains state-of-the-art results for the node classification task on well-known benchmark datasets.


page 1

page 2

page 3

page 4


Unsupervised Universal Self-Attention Network for Graph Classification

Existing graph embedding models often have weaknesses in exploiting grap...

A Capsule Network-based Model for Learning Node Embeddings

In this paper, we focus on learning low-dimensional embeddings of entity...

Pay Attention to Relations: Multi-embeddings for Attributed Multiplex Networks

Graph Convolutional Neural Networks (GCNs) have become effective machine...

Pair-view Unsupervised Graph Representation Learning

Low-dimension graph embeddings have proved extremely useful in various d...

PanRep: Universal node embeddings for heterogeneous graphs

Learning unsupervised node embeddings facilitates several downstream tas...

Inductive Graph Embeddings through Locality Encodings

Learning embeddings from large-scale networks is an open challenge. Desp...

Cascaded Semantic and Positional Self-Attention Network for Document Classification

Transformers have shown great success in learning representations for la...