Learning Curves for Analysis of Deep Networks

10/21/2020
by   Derek Hoiem, et al.
0

A learning curve models a classifier's test error as a function of the number of training samples. Prior works show that learning curves can be used to select model parameters and extrapolate performance. We investigate how to use learning curves to analyze the impact of design choices, such as pre-training, architecture, and data augmentation. We propose a method to robustly estimate learning curves, abstract their parameters into error and data-reliance, and evaluate the effectiveness of different parameterizations. We also provide several interesting observations based on learning curves for a variety of image classification models.

READ FULL TEXT

page 1

page 2

page 3

page 4

08/19/2019

Undeniable signatures based on isogenies of supersingular hyperelliptic curves

We present a proposal for an undeniable signature scheme based in supers...
05/22/2022

Investigating classification learning curves for automatically generated and labelled plant images

In the context of supervised machine learning a learning curve describes...
10/10/2019

Probabilistic Rollouts for Learning Curve Extrapolation Across Hyperparameter Settings

We propose probabilistic models that can extrapolate learning curves of ...
06/08/2022

Estimation of Predictive Performance in High-Dimensional Data Settings using Learning Curves

In high-dimensional prediction settings, it remains challenging to relia...
07/11/2019

Minimizers of the Empirical Risk and Risk Monotonicity

Plotting a learner's average performance against the number of training ...
03/19/2021

The Shape of Learning Curves: a Review

Learning curves provide insight into the dependence of a learner's gener...
05/10/2013

Multi-q Pattern Classification of Polarization Curves

Several experimental measurements are expressed in the form of one-dimen...