-
Confounding-Robust Policy Evaluation in Infinite-Horizon Reinforcement Learning
Off-policy evaluation of sequential decision policies from observational...
read it
-
Confounding-Robust Policy Improvement
We study the problem of learning personalized decision policies from obs...
read it
-
Controlling for Unobserved Confounds in Classification Using Correlational Constraints
As statistical classifiers become integrated into real-world application...
read it
-
Generalization and Invariances in the Presence of Unobserved Confounding
The ability to extrapolate, or generalize, from observed to new related ...
read it
-
Off-policy Evaluation in Infinite-Horizon Reinforcement Learning with Latent Confounders
Off-policy evaluation (OPE) in reinforcement learning is an important pr...
read it
-
Instrumental variables, spatial confounding and interference
Unobserved spatial confounding variables are prevalent in environmental ...
read it
-
How Algorithmic Confounding in Recommendation Systems Increases Homogeneity and Decreases Utility
Recommendation systems occupy an expanding role in everyday decision mak...
read it
Off-policy Policy Evaluation For Sequential Decisions Under Unobserved Confounding
When observed decisions depend only on observed features, off-policy policy evaluation (OPE) methods for sequential decision making problems can estimate the performance of evaluation policies before deploying them. This assumption is frequently violated due to unobserved confounders, unrecorded variables that impact both the decisions and their outcomes. We assess robustness of OPE methods under unobserved confounding by developing worst-case bounds on the performance of an evaluation policy. When unobserved confounders can affect every decision in an episode, we demonstrate that even small amounts of per-decision confounding can heavily bias OPE methods. Fortunately, in a number of important settings found in healthcare, policy-making, operations, and technology, unobserved confounders may primarily affect only one of the many decisions made. Under this less pessimistic model of one-decision confounding, we propose an efficient loss-minimization-based procedure for computing worst-case bounds, and prove its statistical consistency. On two simulated healthcare examples—management of sepsis patients and developmental interventions for autistic children—where this is a reasonable model of confounding, we demonstrate that our method invalidates non-robust results and provides meaningful certificates of robustness, allowing reliable selection of policies even under unobserved confounding.
READ FULL TEXT
Comments
There are no comments yet.