Understanding Isomorphism Bias in Graph Data Sets

10/26/2019
by   Sergei Ivanov, et al.
0

In recent years there has been a rapid increase in classification methods on graph structured data. Both in graph kernels and graph neural networks, one of the implicit assumptions of successful state-of-the-art models was that incorporating graph isomorphism features into the architecture leads to better empirical performance. However, as we discover in this work, commonly used data sets for graph classification have repeating instances which cause the problem of isomorphism bias, i.e. artificially increasing the accuracy of the models by memorizing target information from the training set. This prevents fair competition of the algorithms and raises a question of the validity of the obtained results. We analyze 54 data sets, previously extensively used for graph-related tasks, on the existence of isomorphism bias, give a set of recommendations to machine learning practitioners to properly set up their models, and open source new data sets for the future experiments.

READ FULL TEXT
research
11/22/2017

No Classification without Representation: Assessing Geodiversity Issues in Open Data Sets for the Developing World

Modern machine learning systems such as image classifiers rely heavily o...
research
06/11/2023

Between-Sample Relationship in Learning Tabular Data Using Graph and Attention Networks

Traditional machine learning assumes samples in tabular data to be indep...
research
05/15/2019

IPC: A Benchmark Data Set for Learning with Graph-Structured Data

Benchmark data sets are an indispensable ingredient of the evaluation of...
research
11/28/2022

Graph Neural Networks for Breast Cancer Data Integration

International initiatives such as METABRIC (Molecular Taxonomy of Breast...
research
11/12/2022

Integrating Transformer and Autoencoder Techniques with Spectral Graph Algorithms for the Prediction of Scarcely Labeled Molecular Data

In molecular and biological sciences, experiments are expensive, time-co...
research
11/27/2017

Classifier Selection with Permutation Tests

This work presents a content-based recommender system for machine learni...
research
02/07/2023

Machine learning benchmarks for the classification of equivalent circuit models from solid-state electrochemical impedance spectra

Analysis of Electrochemical Impedance Spectroscopy (EIS) data for electr...

Please sign up or login with your details

Forgot password? Click here to reset