When the Oracle Misleads: Modeling the Consequences of Using Observable Rather than Potential Outcomes in Risk Assessment Instruments

03/01/2020
by   Niccolò Dalmasso, et al.
0

Machine learning-based Risk Assessment Instruments are increasingly widely used to support decision making in domains as diverse as healthcare, criminal justice, and consumer finance. When the decision maker’s goal is to reduce the risk of the predicted outcome, they are naturally concerned with potential outcomes. However, RAIs are typically designed to predict outcomes under the historical decision process that generated the data, making them unsuitable for helping decision makers choose among different courses of action. While many of the limitations of such RAIs have been recognized, there does not appear to be a general mathematical model that provides insight into how and why such RAIs can lead users astray. In this work, we aim to fill this gap, showing how RAIs based on observable outcomes can lead to worse outcomes, i.e., more severe departures from an optimal treatment regime, than before the RAI was introduced. This has nothing to do with the quality of prediction; it can occur even when (1) the oracle predictor is available and (2) there is no unmeasured confounding. We describe several dangerous properties of these RAIs and illustrate their suboptimality with a simple example.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
09/07/2020

Fairness in Risk Assessment Instruments: Post-Processing to Achieve Counterfactual Equalized Odds

Algorithmic fairness is a topic of increasing concern both within resear...
research
10/10/2019

Estimation of Utility-Maximizing Bounds on Potential Outcomes

Estimation of individual treatment effects is often used as the basis fo...
research
02/22/2023

Counterfactual Prediction Under Outcome Measurement Error

Across domains such as medicine, employment, and criminal justice, predi...
research
09/18/2019

Fair-by-design explainable models for prediction of recidivism

Recidivism prediction provides decision makers with an assessment of the...
research
10/04/2022

Making Decisions under Outcome Performativity

Decision-makers often act in response to data-driven predictions, with t...
research
11/26/2020

Outcome Indistinguishability

Prediction algorithms assign numbers to individuals that are popularly u...
research
09/18/2023

Beyond expected values: Making environmental decisions using value of information analysis when measurement outcome matters

In ecological and environmental contexts, management actions must someti...

Please sign up or login with your details

Forgot password? Click here to reset