On the consistency of supervised learning with missing values

by   Julie Josse, et al.
Ecole Polytechnique

In many application settings, the data are plagued with missing features. These hinder data analysis. An abundant literature addresses missing values in an inferential framework, where the aim is to estimate parameters and their variance from incomplete tables. Here, we consider supervised-learning settings where the objective is to best predict a target when missing values appear in both training and test sets. We analyze which missing-values strategies lead to good prediction. We show the consistency of two approaches to estimating the prediction function. The most striking one shows that the widely-used mean imputation prior to learning method is consistent when missing values are not informative. This is in contrast with inferential settings as mean imputation is known to have serious drawbacks in terms of deformation of the joint and marginal distribution of the data. That such a simple approach can be consistent has important consequences in practice. This result holds asymptotically when the learning algorithm is consistent in itself. We contribute additional analysis on decision trees as they can naturally tackle empirical risk minimization with missing values. This is due to their ability to handle the half-discrete nature of variables with missing values. After comparing theoretically and empirically different missing-values strategies in trees, we recommend using the missing incorporated in attributes method as it can handle both non-informative and informative missing values.


The Missing Indicator Method: From Low to High Dimensions

Missing data is common in applied data science, particularly for tabular...

Benchmarking missing-values approaches for predictive models on health databases

BACKGROUND: As databases grow larger, it becomes harder to fully control...

What's a good imputation to predict with missing values?

How to learn a good predictor on data with missing values? Most efforts ...

Fairness without Imputation: A Decision Tree Approach for Fair Prediction with Missing Values

We investigate the fairness concerns of training a machine learning mode...

Joints in Random Forests

Decision Trees (DTs) and Random Forests (RFs) are powerful discriminativ...

Conservative Policy Construction Using Variational Autoencoders for Logged Data with Missing Values

In high-stakes applications of data-driven decision making like healthca...

Imputing Missing Values in the Occupational Requirements Survey

The U.S. Bureau of Labor Statistics allows public access to much of the ...

Please sign up or login with your details

Forgot password? Click here to reset