Gi and Pal Scores: Deep Neural Network Generalization Statistics

04/08/2021
by   Yair Schiff, et al.
14

The field of Deep Learning is rich with empirical evidence of human-like performance on a variety of regression, classification, and control tasks. However, despite these successes, the field lacks strong theoretical error bounds and consistent measures of network generalization and learned invariances. In this work, we introduce two new measures, the Gi-score and Pal-score, that capture a deep neural network's generalization capabilities. Inspired by the Gini coefficient and Palma ratio, measures of income inequality, our statistics are robust measures of a network's invariance to perturbations that accurately predict generalization gaps, i.e., the difference between accuracy on training and test sets.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/09/2021

Predicting Deep Neural Network Generalization with Perturbation Response Curves

The field of Deep Learning is rich with empirical evidence of human-like...
research
12/16/2020

Using noise resilience for ranking generalization of deep neural networks

Recent papers have shown that sufficiently overparameterized neural netw...
research
11/29/2019

A Reparameterization-Invariant Flatness Measure for Deep Neural Networks

The performance of deep neural networks is often attributed to their aut...
research
12/04/2020

Representation Based Complexity Measures for Predicting Generalization in Deep Learning

Deep Neural Networks can generalize despite being significantly overpara...
research
10/12/2022

GULP: a prediction-based metric between representations

Comparing the representations learned by different neural networks has r...
research
10/22/2020

In Search of Robust Measures of Generalization

One of the principal scientific challenges in deep learning is explainin...
research
02/25/2022

Deep Neural Network for Automatic Assessment of Dysphonia

The purpose of this work is to contribute to the understanding and impro...

Please sign up or login with your details

Forgot password? Click here to reset