Assessing the Fairness of Classifiers with Collider Bias

by   Zhenlong Xu, et al.

The increasing maturity of machine learning technologies and their applications to decisions relate to everyday decision making have brought concerns about the fairness of the decisions. However, current fairness assessment systems often suffer from collider bias, which leads to a spurious association between the protected attribute and the outcomes. To achieve fairness evaluation on prediction models at the individual level, in this paper, we develop the causality-based theorems to support the use of direct causal effect estimation for fairness assessment on a given a classifier without access to original training data. Based on the theorems, an unbiased situation test method is presented to assess individual fairness of predictions by a classifier, through the elimination of the impact of collider bias of the classifier on the fairness assessment. Extensive experiments have been performed on synthetic and real-world data to evaluate the performance of the proposed method. The experimental results show that the proposed method reduces bias significantly.


page 1

page 2

page 3

page 4


PC-Fairness: A Unified Framework for Measuring Causality-based Fairness

A recent trend of fair machine learning is to define fairness as causali...

Fairness Evaluation in Presence of Biased Noisy Labels

Risk assessment tools are widely used around the country to inform decis...

How Biased is Your Feature?: Computing Fairness Influence Functions with Global Sensitivity Analysis

Fairness in machine learning has attained significant focus due to the w...

Transparency Tools for Fairness in AI (Luskin)

We propose new tools for policy-makers to use when assessing and correct...

Machine Decisions and Human Consequences

As we increasingly delegate decision-making to algorithms, whether direc...

Quantifying Infra-Marginality and Its Trade-off with Group Fairness

In critical decision-making scenarios, optimizing accuracy can lead to a...

On the Fairness of Randomized Trials for Recommendation With Heterogeneous Demographics and Beyond

Observed events in recommendation are consequence of the decisions made ...