Efficient Estimation and Evaluation of Prediction Rules in Semi-Supervised Settings under Stratified Sampling

10/19/2020
by   Jessica Gronsbell, et al.
0

In many contemporary applications, large amounts of unlabeled data are readily available while labeled examples are limited. There has been substantial interest in semi-supervised learning (SSL) which aims to leverage unlabeled data to improve estimation or prediction. However, current SSL literature focuses primarily on settings where labeled data is selected randomly from the population of interest. Non-random sampling, while posing additional analytical challenges, is highly applicable to many real world problems. Moreover, no SSL methods currently exist for estimating the prediction performance of a fitted model under non-random sampling. In this paper, we propose a two-step SSL procedure for evaluating a prediction rule derived from a working binary regression model based on the Brier score and overall misclassification rate under stratified sampling. In step I, we impute the missing labels via weighted regression with nonlinear basis functions to account for nonrandom sampling and to improve efficiency. In step II, we augment the initial imputations to ensure the consistency of the resulting estimators regardless of the specification of the prediction model or the imputation model. The final estimator is then obtained with the augmented imputations. We provide asymptotic theory and numerical studies illustrating that our proposals outperform their supervised counterparts in terms of efficiency gain. Our methods are motivated by electronic health records (EHR) research and validated with a real data analysis of an EHR-based study of diabetic neuropathy.

READ FULL TEXT
research
11/15/2017

Semi-Supervised Approaches to Efficient Evaluation of Model Prediction Performance

In many modern machine learning applications, the outcome is expensive o...
research
01/17/2017

Efficient and Adaptive Linear Regression in Semi-Supervised Settings

We consider the linear regression problem under semi-supervised settings...
research
03/07/2021

Risk Prediction with Imperfect Survival Outcome Information from Electronic Health Records

Readily available proxies for time of disease onset such as time of the ...
research
08/16/2022

Semi-supervised Transfer Learning for Evaluation of Model Classification Performance

In modern machine learning applications, frequent encounters of covariat...
research
11/16/2020

On systems of quotas based on bankruptcy with a priori unions: estimating random arrival-style rules

This paper addresses a sampling procedure for estimating extensions of t...
research
03/26/2020

Prior Adaptive Semi-supervised Learning with Application to EHR Phenotyping

Electronic Health Records (EHR) data, a rich source for biomedical resea...
research
07/16/2021

Semi-supervised Learning for Marked Temporal Point Processes

Temporal Point Processes (TPPs) are often used to represent the sequence...

Please sign up or login with your details

Forgot password? Click here to reset