Varying impacts of letters of recommendation on college admissions: Approximate balancing weights for subgroup effects in observational studies

by   Eli Ben-Michael, et al.

In a pilot study during the 2016-17 admissions cycle, the University of California, Berkeley invited many applicants for freshman admission to submit letters of recommendation. We are interested in estimating how impacts vary for under-represented applicants and applicants with differing a priori probability of admission. Assessing treatment effect variation in observational studies is challenging, however, because differences in estimated impacts across subgroups reflect both differences in impacts and differences in covariate balance. To address this, we develop balancing weights that directly optimize for “local balance” within subgroups while maintaining global covariate balance between treated and control populations. We then show that this approach has a dual representation as a form of inverse propensity score weighting with a hierarchical propensity score model. In the UC Berkeley pilot study, our proposed approach yields excellent local and global balance, unlike more traditional weighting methods, which fail to balance covariates within subgroups. We find that the impact of letters of recommendation increases with the predicted probability of admission, with mixed evidence of differences for under-represented minority applicants.



There are no comments yet.


page 1

page 2

page 3

page 4


The Balancing Act in Causal Inference

The idea of covariate balance is at the core of causal inference. Invers...

Large Sample Properties of Entropy Balancing Estimators of Average Causal Effects

Weighting methods are used in observational studies to adjust for covari...

One-step weighting to generalize and transport treatment effect estimates to a target population

Weighting methods are often used to generalize and transport estimates o...

A Framework for Covariate Balance using Bregman Distances

A common goal in observational research is to estimate marginal causal e...

Covariate Balancing by Uniform Transformer

In observational studies, it is important to balance covariates in diffe...

Is it who you are or where you are? Accounting for compositional differences in cross-site treatment variation

Multisite trials, in which treatment is randomized separately in multipl...

A tutorial for using propensity score weighting for moderation analysis: an application to smoking disparities among LGB adults

Objective. To provide step-by-step guidance and STATA and R code for usi...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.