DeepAI AI Chat
Log In Sign Up

On a scalable entropic breaching of the overfitting barrier in machine learning

02/08/2020
by   Illia Horenko, et al.
USI Università della Svizzera italiana
0

Overfitting and treatment of "small data" are among the most challenging problems in the machine learning (ML), when a relatively small data statistics size T is not enough to provide a robust ML fit for a relatively large data feature dimension D. Deploying a massively-parallel ML analysis of generic classification problems for different D and T, existence of statistically-significant linear overfitting barriers for common ML methods is demonstrated. For example, these results reveal that for a robust classification of bioinformatics-motivated generic problems with the Long Short-Term Memory deep learning classifier (LSTM) one needs in a best case a statistics T that is at least 13.8 times larger then the feature dimension D. It is shown that this overfitting barrier can be breached at a 10^-12 fraction of the computational cost by means of the entropy-optimal Scalable Probabilistic Approximations algorithm (eSPA), performing a joint solution of the entropy-optimal Bayesian network inference and feature space segmentation problems. Application of eSPA to experimental single cell RNA sequencing data exhibits a 30-fold classification performance boost when compared to standard bioinformatics tools - and a 7-fold boost when compared to the deep learning LSTM classifier.

READ FULL TEXT

page 4

page 7

page 12

10/08/2021

Kinematically consistent recurrent neural networks for learning inverse problems in wave propagation

Although machine learning (ML) is increasingly employed recently for mec...
01/03/2019

Learning a Generator Model from Terminal Bus Data

In this work we investigate approaches to reconstruct generator models f...
12/09/2019

Exploiting Dual-Gate Ambipolar CNFETs for Scalable Machine Learning Classification

Ambipolar carbon nanotube based field-effect transistors (AP-CNFETs) exh...
05/05/2022

Development of Interpretable Machine Learning Models to Detect Arrhythmia based on ECG Data

The analysis of electrocardiogram (ECG) signals can be time consuming as...
04/28/2021

Risk Bounds for Over-parameterized Maximum Margin Classification on Sub-Gaussian Mixtures

Modern machine learning systems such as deep neural networks are often h...