Cold Case: The Lost MNIST Digits

05/25/2019
by   Chhavi Yadav, et al.
0

Although the popular MNIST dataset [LeCun et al., 1994] is derived from the NIST database [Grother and Hanaoka, 1995], the precise processing steps for this derivation have been lost to time. We propose a reconstruction that is accurate enough to serve as a replacement for the MNIST dataset, with insignificant changes in accuracy. We trace each MNIST digit to its NIST source and its rich metadata such as writer identifier, partition identifier, etc. We also reconstruct the complete MNIST test set with 60,000 samples instead of the usual 10,000. Since the balance 50,000 were never distributed, they enable us to investigate the impact of twenty-five years of MNIST experiments on the reported testing performances. Our results unambiguously confirm the trends observed by Recht et al. [2018, 2019]: although the misclassification rates are slightly off, classifier ordering and model selection remain broadly reliable. We attribute this phenomenon to the pairing benefits of comparing classifiers on the same digits.

READ FULL TEXT
research
08/03/2019

Kannada-MNIST: A new handwritten digits dataset for the Kannada language

In this paper, we disseminate a new handwritten digits-dataset, termed K...
research
08/25/2017

Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms

We present Fashion-MNIST, a new dataset comprising of 28x28 grayscale im...
research
07/03/2020

Examining Redundancy in the Context of Safe Machine Learning

This paper describes a set of experiments with neural network classifier...
research
09/28/2020

Afro-MNIST: Synthetic generation of MNIST-style datasets for low-resource languages

We present Afro-MNIST, a set of synthetic MNIST-style datasets for four ...
research
12/11/2017

On Quadratic Penalties in Elastic Weight Consolidation

Elastic weight consolidation (EWC, Kirkpatrick et al, 2017) is a novel a...
research
11/23/2020

Effectiveness of MPC-friendly Softmax Replacement

Softmax is widely used in deep learning to map some representation to a ...

Please sign up or login with your details

Forgot password? Click here to reset