Improving uplift model evaluation on RCT data

10/05/2022
by   Björn Bokelmann, et al.
0

Estimating treatment effects is one of the most challenging and important tasks of data analysts. Traditional statistical methods aim to estimate average treatment effects over a population. While being highly useful, such average treatment effects do not help to decide which individuals profit most by the treatment. This is where uplift modeling becomes important. Uplift models help to select the right individuals for treatment, to maximize the overall treatment effect (uplift). A challenging problem in uplift modeling is to evaluate the models. Previous literature suggests methods like the Qini curve and the transformed outcome mean squared error. However, these metrics suffer from variance: Their evaluations are strongly affected by random noise in the data, which makes these evaluations to a certain degree arbitrary. In this paper, we analyze the variance of the uplift evaluation metrics, on randomized controlled trial data, in a sound statistical manner. We propose certain outcome adjustment methods, for which we prove theoretically and empirically, that they reduce the variance of the uplift evaluation metrics. Our statistical analysis and the proposed outcome adjustment methods are a step towards a better evaluation practice in uplift modeling.

READ FULL TEXT
research
03/17/2018

Consistent estimation of treatment effects under endogeneous heteroskedasticity

The empirical literature on program evaluation limits its scope almost e...
research
03/17/2018

Consistent estimation of treatment effects under heterogeneous heteroskedasticity

The empirical literature on program evaluation limits its scope almost e...
research
11/16/2021

Practical Guidance on Modeling Choices for the Virtual Twins Method

Individuals can vary drastically in their response to the same treatment...
research
12/28/2021

Analysis of N-of-1 trials using Bayesian distributed lag model with autocorrelated errors

An N-of-1 trial is a multi-period crossover trial performed in a single ...
research
01/27/2022

A Knowledge-Based Decision Support System for In Vitro Fertilization Treatment

In Vitro Fertilization (IVF) is the most widely used Assisted Reproducti...

Please sign up or login with your details

Forgot password? Click here to reset