Towards explaining the generalization gap in neural networks using topological data analysis

03/23/2022
by   Rubén Ballester, et al.
10

Understanding how neural networks generalize on unseen data is crucial for designing more robust and reliable models. In this paper, we study the generalization gap of neural networks using methods from topological data analysis. For this purpose, we compute homological persistence diagrams of weighted graphs constructed from neuron activation correlations after a training phase, aiming to capture patterns that are linked to the generalization capacity of the network. We compare the usefulness of different numerical summaries from persistence diagrams and show that a combination of some of them can accurately predict and partially explain the generalization gap without the need of a test set. Evaluation on two computer vision recognition tasks (CIFAR10 and SVHN) shows competitive generalization gap prediction when compared against state-of-the-art methods.

READ FULL TEXT
research
06/29/2021

Topological Data Analysis through alignment of Persistence Landscapes

Persistence landscapes are functional summaries of persistence diagrams ...
research
04/20/2019

PersLay: A Simple and Versatile Neural Network Layer for Persistence Diagrams

Persistence diagrams, a key descriptor from Topological Data Analysis, e...
research
12/23/2018

Neural Persistence: A Complexity Measure for Deep Neural Networks Using Algebraic Topology

While many approaches to make neural networks more fathomable have been ...
research
01/16/2020

Understanding the Power of Persistence Pairing via Permutation Test

Recently many efforts have been made to incorporate persistence diagrams...
research
04/04/2018

Functional Summaries of Persistence Diagrams

One of the primary areas of interest in applied algebraic topology is pe...
research
10/31/2018

Understanding Deep Neural Networks Using Topological Data Analysis

Deep neural networks (DNN) are black box algorithms. They are trained us...
research
02/13/2018

On Characterizing the Capacity of Neural Networks using Algebraic Topology

The learnability of different neural architectures can be characterized ...

Please sign up or login with your details

Forgot password? Click here to reset