On the Prediction Instability of Graph Neural Networks

05/20/2022
by   Max Klabunde, et al.
0

Instability of trained models, i.e., the dependence of individual node predictions on random factors, can affect reproducibility, reliability, and trust in machine learning systems. In this paper, we systematically assess the prediction instability of node classification with state-of-the-art Graph Neural Networks (GNNs). With our experiments, we establish that multiple instantiations of popular GNN models trained on the same data with the same model hyperparameters result in almost identical aggregated performance but display substantial disagreement in the predictions for individual nodes. We find that up to one third of the incorrectly classified nodes differ across algorithm runs. We identify correlations between hyperparameters, node properties, and the size of the training set with the stability of predictions. In general, maximizing model performance implicitly also reduces model instability.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
01/17/2021

Membership Inference Attack on Graph Neural Networks

Graph Neural Networks (GNNs), which generalize traditional deep neural n...
research
12/21/2019

How Robust Are Graph Neural Networks to Structural Noise?

Graph neural networks (GNNs) are an emerging model for learning graph em...
research
10/12/2022

What Makes Graph Neural Networks Miscalibrated?

Given the importance of getting calibrated predictions and reliable unce...
research
06/09/2020

On the Bottleneck of Graph Neural Networks and its Practical Implications

Graph neural networks (GNNs) were shown to effectively learn from highly...
research
11/22/2020

GNNVis: A Visual Analytics Approach for Prediction Error Diagnosis of Graph Neural Networks

Graph Neural Networks (GNNs) aim to extend deep learning techniques to g...
research
07/24/2023

Addressing the Impact of Localized Training Data in Graph Neural Networks

Graph Neural Networks (GNNs) have achieved notable success in learning f...
research
02/07/2022

Neural Models for Output-Space Invariance in Combinatorial Problems

Recently many neural models have been proposed to solve combinatorial pu...

Please sign up or login with your details

Forgot password? Click here to reset