Cross-Fitting and Averaging for Machine Learning Estimation of Heterogeneous Treatment Effects
share
We investigate the finite sample performance of sample splitting, cross-fitting and averaging for the estimation of the conditional average treatment effect. Recently proposed methods, so-called meta-learners, make use of machine learning to estimate different nuisance functions and hence allow for fewer restrictions on the underlying structure of the data. To limit a potential overfitting bias, that may result when using machine learning methods, cross-fitting estimators have been proposed. This includes the splitting of the data in different folds. To the best of our knowledge, it is not yet clear how exactly the data should be split and averaged. We employ a simulation study with different data generation processes and consider different estimators that vary in sample-splitting, cross-fitting and averaging procedures. We investigate the performance of each estimator independently on four different meta-learners: The doubly-robust-learner, the R-learner, the T-learner and the X-learner. We find that the performance of all meta-learners heavily depends on the procedure of splitting and averaging. The best performance in terms of mean squared error (MSE) could be achieved when using a 5-fold cross-fitting estimator which is averaged by the median over multiple different sample-splittings.
READ FULL TEXT