Estimating regression errors without ground truth values

10/09/2019
by   Henri Tiittanen, et al.
0

Regression analysis is a standard supervised machine learning method used to model an outcome variable in terms of a set of predictor variables. In most real-world applications we do not know the true value of the outcome variable being predicted outside the training data, i.e., the ground truth is unknown. It is hence not straightforward to directly observe when the estimate from a model potentially is wrong, due to phenomena such as overfitting and concept drift. In this paper we present an efficient framework for estimating the generalization error of regression functions, applicable to any family of regression functions when the ground truth is unknown. We present a theoretical derivation of the framework and empirically evaluate its strengths and limitations. We find that it performs robustly and is useful for detecting concept drift in datasets in several real-world domains.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/25/2021

The Effect of Ground Truth Accuracy on the Evaluation of Localization Systems

The ability to accurately evaluate the performance of location determina...
research
04/06/2022

AutoCOR: Autonomous Condylar Offset Ratio Calculator on TKA-Postoperative Lateral Knee X-ray

The postoperative range of motion is one of the crucial factors indicati...
research
10/28/2021

Aggregation as Unsupervised Learning and its Evaluation

Regression uses supervised machine learning to find a model that combine...
research
07/18/2017

A Machine Learning Approach for Evaluating Creative Artifacts

Much work has been done in understanding human creativity and defining m...
research
05/26/2023

Detecting Errors in Numerical Data via any Regression Model

Noise plagues many numerical datasets, where the recorded values in the ...
research
08/05/2019

Mass Estimation from Images using Deep Neural Network and Sparse Ground Truth

Supervised learning is the workhorse for regression and classification t...
research
01/27/2020

Predicting Regression Probability Distributions with Imperfect Data Through Optimal Transformations

The goal of regression analysis is to predict the value of a numeric out...

Please sign up or login with your details

Forgot password? Click here to reset