An Empirical Study on Learning Fairness Metrics for COMPAS Data with Human Supervision

10/22/2019
by   Hanchen Wang, et al.
16

The notion of individual fairness requires that similar people receive similar treatment. However, this is hard to achieve in practice since it is difficult to specify the appropriate similarity metric. In this work, we attempt to learn such similarity metric from human annotated data. We gather a new dataset of human judgments on a criminal recidivism prediction (COMPAS) task. By assuming the human supervision obeys the principle of individual fairness, we leverage prior work on metric learning, evaluate the performance of several metric learning methods on our dataset, and show that the learned metrics outperform the Euclidean and Precision metric under various criteria. We do not provide a way to directly learn a similarity metric satisfying the individual fairness, but to provide an empirical study on how to derive the similarity metric from human supervisors, then future work can use this as a tool to understand human supervision.

READ FULL TEXT
research
06/01/2019

Metric Learning for Individual Fairness

There has been much discussion recently about how fairness should be mea...
research
07/02/2019

Operationalizing Individual Fairness with Pairwise Fair Representations

We revisit the notion of individual fairness proposed by Dwork et al. A ...
research
09/04/2019

Metric Learning from Imbalanced Data

A key element of any machine learning algorithm is the use of a function...
research
03/23/2022

Is Fairness Only Metric Deep? Evaluating and Addressing Subgroup Gaps in Deep Metric Learning

Deep metric learning (DML) enables learning with less supervision throug...
research
02/18/2020

Individual Fairness Revisited: Transferring Techniques from Adversarial Robustness

We turn the definition of individual fairness on its head—rather than as...
research
08/09/2020

Disentangled Multidimensional Metric Learning for Music Similarity

Music similarity search is useful for a variety of creative tasks such a...
research
05/12/2022

Accounting for the Sequential Nature of States to Learn Features for Reinforcement Learning

In this work, we investigate the properties of data that cause popular r...

Please sign up or login with your details

Forgot password? Click here to reset