Predicting feature imputability in the absence of ground truth

07/14/2020
by   Niamh McCombe, et al.
0

Data imputation is the most popular method of dealing with missing values, but in most real life applications, large missing data can occur and it is difficult or impossible to evaluate whether data has been imputed accurately (lack of ground truth). This paper addresses these issues by proposing an effective and simple principal component based method for determining whether individual data features can be accurately imputed - feature imputability. In particular, we establish a strong linear relationship between principal component loadings and feature imputability, even in the presence of extreme missingness and lack of ground truth. This work will have important implications in practical data imputation strategies.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
05/10/2023

Blockwise Principal Component Analysis for monotone missing data imputation and dimensionality reduction

Monotone missing data is a common problem in data analysis. However, imp...
research
06/30/2022

Solving the "many variables" problem in MICE with principal component regression

Multiple Imputation (MI) is one of the most popular approaches to addres...
research
06/04/2017

Evolving imputation strategies for missing data in classification problems with TPOT

Missing data has a ubiquitous presence in real-life applications of mach...
research
01/11/2023

The Berkelmans-Pries Feature Importance Method: A Generic Measure of Informativeness of Features

Over the past few years, the use of machine learning models has emerged ...
research
01/23/2021

Unlabeled Principal Component Analysis

We consider the problem of principal component analysis from a data matr...
research
09/06/2022

Understanding and Reducing Crater Counting Errors in Citizen Science Data and the Need for Standardisation

Citizen science has become a popular tool for preliminary data processin...
research
05/26/2023

Contouring by Unit Vector Field Regression

This work introduces a simple deep-learning based method to delineate co...

Please sign up or login with your details

Forgot password? Click here to reset