Safe Policy Learning through Extrapolation: Application to Pre-trial Risk Assessment

09/22/2021
by   Eli Ben-Michael, et al.
0

Algorithmic recommendations and decisions have become ubiquitous in today's society. Many of these and other data-driven policies are based on known, deterministic rules to ensure their transparency and interpretability. This is especially true when such policies are used for public policy decision-making. For example, algorithmic pre-trial risk assessments, which serve as our motivating application, provide relatively simple, deterministic classification scores and recommendations to help judges make release decisions. Unfortunately, existing methods for policy learning are not applicable because they require existing policies to be stochastic rather than deterministic. We develop a robust optimization approach that partially identifies the expected utility of a policy, and then finds an optimal policy by minimizing the worst-case regret. The resulting policy is conservative but has a statistical safety guarantee, allowing the policy-maker to limit the probability of producing a worse outcome than the existing policy. We extend this approach to common and important settings where humans make decisions with the aid of algorithmic recommendations. Lastly, we apply the proposed methodology to a unique field experiment on pre-trial risk assessments. We derive new classification and recommendation rules that retain the transparency and interpretability of the existing risk assessment instrument while potentially leading to better overall outcomes at a lower cost.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 35

page 37

12/04/2020

Experimental Evaluation of Algorithm-Assisted Human Decision-Making: Application to Pretrial Public Safety Assessment

Despite an increasing reliance on fully-automated algorithmic decision m...
01/23/2020

The impact of overbooking on a pre-trial risk assessment tool

Pre-trial risk assessment tools are used to make recommendations to judg...
02/08/2019

Improving Consequential Decision Making under Imperfect Predictions

Consequential decisions are increasingly informed by sophisticated data-...
07/09/2019

A framework for the pre-specification of statistical analysis strategies in clinical trials (Pre-SPEC)

Bias can be introduced into clinical trials if statistical methods are c...
05/04/2018

Algorithmic Decision Making in the Presence of Unmeasured Confounding

On a variety of complex decision-making tasks, from doctors prescribing ...
07/01/2021

Impact Remediation: Optimal Interventions to Reduce Inequality

A significant body of research in the data sciences considers unfair dis...
11/02/2020

Optimal Policies for the Homogeneous Selective Labels Problem

Selective labels are a common feature of consequential decision-making a...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.