Representative Datasets: The Perceptron Case

03/20/2019
by   Rocio Gonzalez-Diaz, et al.
0

One of the main drawbacks of the practical use of neural networks is the long time needed in the training process. Such training process consists in an iterative change of parameters trying to minimize a loss function. These changes are driven by a dataset, which can be seen as a set of labeled points in an n-dimensional space. In this paper, we explore the concept of it representative dataset which is smaller than the original dataset and satisfies a nearness condition independent of isometric transformations. The representativeness is measured using persistence diagrams due to its computational efficiency. We also prove that the accuracy of the learning process of a neural network on a representative dataset is comparable with the accuracy on the original dataset when the neural network architecture is a perceptron and the loss function is the mean squared error. These theoretical results accompanied with experimentation open a door to the size reduction of the dataset in order to gain time in the training process of any neural network.

READ FULL TEXT
research
01/15/2022

Concise Logarithmic Loss Function for Robust Training of Anomaly Detection Model

Recently, deep learning-based algorithms are widely adopted due to the a...
research
09/03/2019

LCA: Loss Change Allocation for Neural Network Training

Neural networks enjoy widespread use, but many aspects of their training...
research
06/21/2020

Applications of Koopman Mode Analysis to Neural Networks

We consider the training process of a neural network as a dynamical syst...
research
09/30/2018

Nth Absolute Root Mean Error

Neural network training process takes long time when the size of trainin...
research
05/01/2020

Can a powerful neural network be a teacher for a weaker neural network?

The transfer learning technique is widely used to learning in one contex...
research
09/13/2021

Explaining Deep Learning Representations by Tracing the Training Process

We propose a novel explanation method that explains the decisions of a d...
research
11/15/2022

LiePoseNet: Heterogeneous Loss Function Based on Lie Group for Significant Speed-up of PoseNet Training Process

Visual localization is an essential modern technology for robotics and c...

Please sign up or login with your details

Forgot password? Click here to reset