DeepAI AI Chat
Log In Sign Up

The Risks of Recourse in Binary Classification

by   Hidde Fokkema, et al.

Algorithmic recourse provides explanations that help users overturn an unfavorable decision by a machine learning system. But so far very little attention has been paid to whether providing recourse is beneficial or not. We introduce an abstract learning-theoretic framework that compares the risks (i.e. expected losses) for classification with and without algorithmic recourse. This allows us to answer the question of when providing recourse is beneficial or harmful at the population level. Surprisingly, we find that there are many plausible scenarios in which providing recourse turns out to be harmful, because it pushes users to regions of higher class uncertainty and therefore leads to more mistakes. We further study whether the party deploying the classifier has an incentive to strategize in anticipation of having to provide recourse, and we find that sometimes they do, to the detriment of their users. Providing algorithmic recourse may therefore also be harmful at the systemic level. We confirm our theoretical findings in experiments on simulated and real-world data. All in all, we conclude that the current concept of algorithmic recourse is not reliably beneficial, and therefore requires rethinking.


page 2

page 22


Multi-Agent Algorithmic Recourse

The recent adoption of machine learning as a tool in real world decision...

Outsider Oversight: Designing a Third Party Audit Ecosystem for AI Governance

Much attention has focused on algorithmic audits and impact assessments ...

Hypothesis Transfer Learning with Surrogate Classification Losses

Hypothesis transfer learning (HTL) contrasts domain adaptation by allowi...

A Learning-Theoretic Framework for Certified Auditing of Machine Learning Models

Responsible use of machine learning requires that models be audited for ...

Metric Elicitation; Moving from Theory to Practice

Metric Elicitation (ME) is a framework for eliciting classification metr...

Abstract Attribute Exploration with Partial Object Descriptions

Attribute exploration has been investigated in several studies, with par...