Revisiting Graph Contrastive Learning from the Perspective of Graph Spectrum

10/05/2022
by   Nian Liu, et al.
0

Graph Contrastive Learning (GCL), learning the node representations by augmenting graphs, has attracted considerable attentions. Despite the proliferation of various graph augmentation strategies, some fundamental questions still remain unclear: what information is essentially encoded into the learned representations by GCL? Are there some general graph augmentation rules behind different augmentations? If so, what are they and what insights can they bring? In this paper, we answer these questions by establishing the connection between GCL and graph spectrum. By an experimental investigation in spectral domain, we firstly find the General grAph augMEntation (GAME) rule for GCL, i.e., the difference of the high-frequency parts between two augmented graphs should be larger than that of low-frequency parts. This rule reveals the fundamental principle to revisit the current graph augmentations and design new effective graph augmentations. Then we theoretically prove that GCL is able to learn the invariance information by contrastive invariance theorem, together with our GAME rule, for the first time, we uncover that the learned representations by GCL essentially encode the low-frequency information, which explains why GCL works. Guided by this rule, we propose a spectral graph contrastive learning module (SpCo), which is a general and GCL-friendly plug-in. We combine it with different existing GCL models, and extensive experiments well demonstrate that it can further improve the performances of a wide variety of different GCL methods.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
04/11/2022

Augmentation-Free Graph Contrastive Learning

Graph contrastive learning (GCL) is the most representative and prevalen...
research
09/14/2022

Graph Contrastive Learning with Personalized Augmentation

Graph contrastive learning (GCL) has emerged as an effective tool for le...
research
10/02/2022

Spectral Augmentation for Self-Supervised Learning on Graphs

Graph contrastive learning (GCL), as an emerging self-supervised learnin...
research
02/06/2023

Spectral Augmentations for Graph Contrastive Learning

Contrastive learning has emerged as a premier method for learning repres...
research
06/16/2023

HomoGCL: Rethinking Homophily in Graph Contrastive Learning

Contrastive learning (CL) has become the de-facto learning paradigm in s...
research
01/12/2023

Signed Directed Graph Contrastive Learning with Laplacian Augmentation

Graph contrastive learning has become a powerful technique for several g...
research
01/04/2021

Beyond Low-frequency Information in Graph Convolutional Networks

Graph neural networks (GNNs) have been proven to be effective in various...

Please sign up or login with your details

Forgot password? Click here to reset