DeepAI AI Chat
Log In Sign Up

Classification of datasets with imputed missing values: does imputation quality matter?

by   Tolou Shadbahr, et al.

Classifying samples in incomplete datasets is a common aim for machine learning practitioners, but is non-trivial. Missing data is found in most real-world datasets and these missing values are typically imputed using established methods, followed by classification of the now complete, imputed, samples. The focus of the machine learning researcher is then to optimise the downstream classification performance. In this study, we highlight that it is imperative to consider the quality of the imputation. We demonstrate how the commonly used measures for assessing quality are flawed and propose a new class of discrepancy scores which focus on how well the method recreates the overall distribution of the data. To conclude, we highlight the compromised interpretability of classifier models trained using poorly imputed data.


page 9

page 31

page 32

page 33

page 34

page 35

page 36

page 37


Generative Imputation and Stochastic Prediction

In many machine learning applications, we are faced with incomplete data...

PROMISSING: Pruning Missing Values in Neural Networks

While data are the primary fuel for machine learning models, they often ...

Goodness (of fit) of Imputation Accuracy: The GoodImpact Analysis

In statistical survey analysis, (partial) non-responders are integral el...

Quality control, data cleaning, imputation

This chapter addresses important steps during the quality assurance and ...

Learning from Incomplete Data by Simultaneous Training of Neural Networks and Sparse Coding

Handling correctly incomplete datasets in machine learning is a fundamen...

Preparing Weather Data for Real-Time Building Energy Simulation

This study introduces a framework for quality control of measured weathe...