Learning Theory Can (Sometimes) Explain Generalisation in Graph Neural Networks

12/07/2021
by   Pascal Mattia Esser, et al.
0

In recent years, several results in the supervised learning setting suggested that classical statistical learning-theoretic measures, such as VC dimension, do not adequately explain the performance of deep learning models which prompted a slew of work in the infinite-width and iteration regimes. However, there is little theoretical explanation for the success of neural networks beyond the supervised setting. In this paper we argue that, under some distributional assumptions, classical learning-theoretic measures can sufficiently explain generalization for graph neural networks in the transductive setting. In particular, we provide a rigorous analysis of the performance of neural networks in the context of transductive inference, specifically by analysing the generalisation properties of graph convolutional networks for the problem of node classification. While VC Dimension does result in trivial generalisation error bounds in this setting as well, we show that transductive Rademacher complexity can explain the generalisation properties of graph convolutional networks for stochastic block models. We further use the generalisation error bounds based on transductive Rademacher complexity to demonstrate the role of graph convolutions and network architectures in achieving smaller generalisation error and provide insights into when the graph structure can help in learning. The findings of this paper could re-new the interest in studying generalisation in neural networks in terms of learning-theoretic measures, albeit in specific problems.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/22/2023

On Addressing the Limitations of Graph Neural Networks

This report gives a summary of two problems about graph convolutional ne...
research
02/22/2019

Adversarial Attacks on Graph Neural Networks via Meta Learning

Deep learning models for graphs have advanced the state of the art on ma...
research
09/16/2019

Learnability Can Be Independent of ZFC Axioms: Explanations and Implications

In Ben-David et al.'s "Learnability Can Be Undecidable," they prove an i...
research
03/14/2023

Practically Solving LPN in High Noise Regimes Faster Using Neural Networks

We conduct a systematic study of solving the learning parity with noise ...
research
06/04/2019

Sequential Neural Networks as Automata

This work attempts to explain the types of computation that neural netwo...
research
06/20/2020

Generalizing Graph Neural Networks Beyond Homophily

We investigate the representation power of graph neural networks in the ...
research
01/24/2020

Comparison of Syntactic and Semantic Representations of Programs in Neural Embeddings

Neural approaches to program synthesis and understanding have proliferat...

Please sign up or login with your details

Forgot password? Click here to reset