Performance of regression models as a function of experiment noise

12/17/2019
by   Gang Li, et al.
0

A challenge in developing machine learning regression models is that it is difficult to know whether maximal performance has been reached on a particular dataset, or whether further model improvement is possible. In biology this problem is particularly pronounced as sample labels are typically obtained through experiments and therefore have experiment noise associated with them. Such label noise puts a fundamental limit to the performance attainable by regression models. We address this challenge by deriving a theoretical upper bound for the coefficient of determination (R2) for regression models. This theoretical upper bound depends only on the noise associated with sample labels in a dataset as well as the label variance. The upper bound estimate was validated via Monte Carlo simulations and then used as a tool to bootstrap performance of regression models trained on biological datasets, including protein sequence data, transcriptomic data, and genomic data. Although we study biological datasets in this work, the new upper bound estimates will hold true for regression models from any research field or application area where sample labels are associated with noise.

READ FULL TEXT
research
11/27/2014

Classification with Noisy Labels by Importance Reweighting

In this paper, we study a classification problem in which sample labels ...
research
07/20/2023

Investigating minimizing the training set fill distance in machine learning regression

Many machine learning regression methods leverage large datasets for tra...
research
02/06/2020

Interpolation under latent factor regression models

This work studies finite-sample properties of the risk of the minimum-no...
research
10/20/2020

Localizing Changes in High-Dimensional Regression Models

This paper addresses the problem of localizing change points in high-dim...
research
03/12/2020

A comparison of parameter estimation in function-on-function regression

Recent technological developments have enabled us to collect complex and...
research
11/26/2020

Simultaneous inference for time-varying models

A general class of time-varying regression models is considered in this ...
research
06/25/2020

STORM: Foundations of End-to-End Empirical Risk Minimization on the Edge

Empirical risk minimization is perhaps the most influential idea in stat...

Please sign up or login with your details

Forgot password? Click here to reset