When Deep Classifiers Agree: Analyzing Correlations between Learning Order and Image Statistics

05/19/2021 ∙ by Iuliia Pliushch, et al. ∙ 16

Although a plethora of architectural variants for deep classification has been introduced over time, recent works have found empirical evidence towards similarities in their training process. It has been hypothesized that neural networks converge not only to similar representations, but also exhibit a notion of empirical agreement on which data instances are learned first. Following in the latter works' footsteps, we define a metric to quantify the relationship between such classification agreement over time, and posit that the agreement phenomenon can be mapped to core statistics of the investigated dataset. We empirically corroborate this hypothesis across the CIFAR10, Pascal, ImageNet and KTH-TIPS2 datasets. Our findings indicate that agreement seems to be independent of specific architectures, training hyper-parameters or labels, albeit follows an ordering according to image statistics.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 6

page 15

page 17

Code Repositories

intrinsic_ordering_nn_training

Agreement computation during nn training, along with selected dataset metrics


view repo
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.