Assessing the Fairness of Classifiers with Collider Bias

10/08/2020
by   Zhenlong Xu, et al.
3

The increasing maturity of machine learning technologies and their applications to decisions relate to everyday decision making have brought concerns about the fairness of the decisions. However, current fairness assessment systems often suffer from collider bias, which leads to a spurious association between the protected attribute and the outcomes. To achieve fairness evaluation on prediction models at the individual level, in this paper, we develop the causality-based theorems to support the use of direct causal effect estimation for fairness assessment on a given a classifier without access to original training data. Based on the theorems, an unbiased situation test method is presented to assess individual fairness of predictions by a classifier, through the elimination of the impact of collider bias of the classifier on the fairness assessment. Extensive experiments have been performed on synthetic and real-world data to evaluate the performance of the proposed method. The experimental results show that the proposed method reduces bias significantly.

READ FULL TEXT

page 1

page 2

page 3

page 4

10/20/2019

PC-Fairness: A Unified Framework for Measuring Causality-based Fairness

A recent trend of fair machine learning is to define fairness as causali...
03/30/2020

Fairness Evaluation in Presence of Biased Noisy Labels

Risk assessment tools are widely used around the country to inform decis...
06/01/2022

How Biased is Your Feature?: Computing Fairness Influence Functions with Global Sensitivity Analysis

Fairness in machine learning has attained significant focus due to the w...
07/09/2020

Transparency Tools for Fairness in AI (Luskin)

We propose new tools for policy-makers to use when assessing and correct...
11/16/2018

Machine Decisions and Human Consequences

As we increasingly delegate decision-making to algorithms, whether direc...
09/03/2019

Quantifying Infra-Marginality and Its Trade-off with Group Fairness

In critical decision-making scenarios, optimizing accuracy can lead to a...
01/25/2020

On the Fairness of Randomized Trials for Recommendation With Heterogeneous Demographics and Beyond

Observed events in recommendation are consequence of the decisions made ...