CARLA: A Python Library to Benchmark Algorithmic Recourse and Counterfactual Explanation Algorithms

08/02/2021
by   Martin Pawelczyk, et al.
203

Counterfactual explanations provide means for prescriptive model explanations by suggesting actionable feature changes (e.g., increase income) that allow individuals to achieve favorable outcomes in the future (e.g., insurance approval). Choosing an appropriate method is a crucial aspect for meaningful counterfactual explanations. As documented in recent reviews, there exists a quickly growing literature with available methods. Yet, in the absence of widely available opensource implementations, the decision in favor of certain models is primarily based on what is readily available. Going forward - to guarantee meaningful comparisons across explanation methods - we present CARLA (Counterfactual And Recourse LibrAry), a python library for benchmarking counterfactual explanation methods across both different data sets and different machine learning models. In summary, our work provides the following contributions: (i) an extensive benchmark of 11 popular counterfactual explanation methods, (ii) a benchmarking framework for research on future counterfactual explanation methods, and (iii) a standardized set of integrated evaluation measures and data sets for transparent and extensive comparisons of these methods. We have open-sourced CARLA and our experimental results on Github, making them available as competitive baselines. We welcome contributions from other research groups and practitioners.

READ FULL TEXT

page 17

page 18

page 19

research
07/09/2021

A Framework and Benchmarking Study for Counterfactual Generating Methods on Tabular Data

Counterfactual explanations are viewed as an effective way to explain ma...
research
04/14/2022

Global Counterfactual Explanations: Investigations, Implementations and Improvements

Counterfactual explanations have been widely studied in explainability, ...
research
04/13/2023

counterfactuals: An R Package for Counterfactual Explanation Methods

Counterfactual explanation methods provide information on how feature va...
research
08/02/2022

ferret: a Framework for Benchmarking Explainers on Transformers

Many interpretability tools allow practitioners and researchers to expla...
research
12/04/2019

Counterfactual Explanation Algorithms for Behavioral and Textual Data

We study the interpretability of predictive systems that use high-dimens...
research
10/21/2022

A Survey on Graph Counterfactual Explanations: Definitions, Methods, Evaluation

In recent years, Graph Neural Networks have reported outstanding perform...
research
01/26/2023

Finding Regions of Counterfactual Explanations via Robust Optimization

Counterfactual explanations play an important role in detecting bias and...

Please sign up or login with your details

Forgot password? Click here to reset