Fairness With Minimal Harm: A Pareto-Optimal Approach For Healthcare

1 Introduction

Machine learning algorithms play an important role in decision making in society. When these algorithms are used to make high-impact healthcare decisions such as deciding whether or not to classify a skin lesion as benign or not, or predicting mortality for intensive care unit patients, it is paramount to guarantee that these decisions are accurate and unbiased with respect to sensitive attributes such as gender or ethnicity. A model that is trained naively may not have these properties by default (barocas2016big). It is desirable in these critical applications to impose some fairness criteria. There are several lines of work in fairness in machine learning, such as Demographic Parity (louizos2015variational; zemel2013learning; feldman2015certifying

), Equality of Odds, Equality of Opportunity (

hardt2016equality; woodworth2017learning), or Calibration (pleiss2017fairness). These notions of fairness are appropriate in many scenarios, but in domains where quality of service is paramount, such as healthcare, we argue that it is necessary to strive for models that are as close to fair as possible without introducing unnecessary harm to any subgroup (ustun2019fairness).

In this work, we measure discrimination (unfairness) in terms of difference in predictive risks across sub-populations defined by our sensitive attributes. This concept has been explored in other works such as calders2010three; dwork2012fairness; feldman2015certifying; chen2018my; ustun2019fairness. We examine the subset of models from our hypothesis class that have the best trade-offs between sub-population risks, and select from this set the one with the smallest risk disparity gap. This is in contrast to common post-hoc correction methods like the ones proposed in hardt2016equality; woodworth2017learning

where randomness is potentially added to the decisions of all sub-populations. While this type of approach diminishes the accuracy-disparity gap, it does so by potentially introducing randomness into the final decision (i.e., with some probability, disregard classifier output and produce arbitrary decision based solely on sensitive label), which leads to performance degradation in most common risk metrics. Since our proposed methodology does not require test-time access to sensitive attributes, and can be applied to any standard classification or regression task, it can also be used to reduce risk disparity between outcomes, acting as an adaptive risk equalization loss compatible with unbalanced classification scenarios.

Main Contributions

We formalize the notion of no-unnecessary-harm fairness using Pareto optimality, a state of resource allocations from which it is impossible to reallocate without making one subgroup worse. We show that finding a Pareto-fair classifier is equivalent to finding a model in our hypothesis class that is both Pareto optimal with respect to the sub-population risks (no unnecessary harm) and minimizes risk disparity. This notion is already amenable to non-binary sensitive attributes. We analyze Pareto fairness on an illustrative example, and compare it to alternative approaches. We provide an algorithm that promotes fair solutions belonging to the Pareto front; this algorithm can be applied to any standard classifier or regression task. Finally, we show how our methodology performs on real tasks such as predicting ICU mortality rates in the MIMIC-III dataset from hospital notes johnson2016mimic, and classifying skin lesions in the HAM10000 dataset tschandl2018ham10000.

2 Problem Statement

Consider we have access to a dataset $D = {(x_{i}, y_{i}, a_{i})}_{i = 1}^{n}$ containing $n$

independent triplet samples drawn from a joint distribution

(x_{i}, y_{i}, a_{i}) \sim P (X, Y, A)

where

x_{i} \in X

are our input features (e.g., images, tabular data, etc.),

y_{i} \in Y

is our target variable, and

a_{i} \in A

indicates group membership or sensitive status (e.g., ethnicity, gender); our input features

X

may or may not explicitly contain

A

Let $h \in H$ be a classifier from our hypothesis class $H$ trained to infer $y$ from $x$ , $h : X \to Y$

; and a loss function

ℓ : Y \times Y \to R

. We define the class-specific risk of classifier

h

on subgroup

a

R_{a} (h) = E_{X, Y | A = a} [ℓ (Y, h (X))]

. The risk discrimination gap between two subgroups

a, a^{'} \in A

is measured as

Γ_{a, a^{'}} (h) = | R_{a} (h) - R_{a^{'}} (h) |

, and we define the pairwise discrimination gap as

{\to Γ}_{A} (h) = {Γ_{a, a^{'}} (h)}_{a, a^{'} \in A}

. Our goal is to obtain a classifier

h \in H

that minimizes this gap without causing unnecessary harm to any particular group. To formalize this notion, we define:

Definition 2.1.

Pareto front: The set of Pareto front classifiers is defined as

Definition 2.2.

Pareto-fair classifier and Pareto-fair vector: A classifier

h^{*}

is an optimal no-harm classifier if it minimizes the discrimination gap among all Pareto front classifiers,

h^{*} = a r g m i n h \in P (H, A) | | {\to Γ}_{A} (h) | |_{\infty}

. The Pareto-fair vector is defined as

r^{*} \in R^{| A |} :

r^{*} = {R_{a} (h^{*})}_{a \in A}

The Pareto front defines the best achievable trade-offs between population risks $R_{a} (h)$ , while the Pareto-fair classifier gives the trade-off with least disparity. Building on analysis done by chen2018my; domingos2000unified

, it is possible to decompose the risk in bias, variance and noise for some given loss functions. The noise represents the smallest achievable risk for infinitely large datasets. If it differs between sensitive groups, zero-discrimination (perfect fairness) can be only achieved by introducing bias or variance, hence doing harm. Figure

1 shows a scenario where the Pareto-front does not intersect the equality of risk line for the case of two sensitive groups

a \in {0, 1}

and a binary output variable

y \in {0, 1}

. Here the level of noise between subgroups differs, and the Pareto-fair vector

r^{*}

would not be achieved by either a naive classifier (minimizes expected global risk), or a classifier where low-occurrence subgroups are over-sampled (re-balanced naive classifier).

Figure 1: Example of binary target ( $y$ ) and sensitive ( $a$ ) variable. Bottom left figure shows conditional distributions of observation variable ( $x$ ) $p (x | a)$ , while top left shows distribution of $y$ as a function of $x, a$ , $p (y | x, a)$ , these distributions are simple piece-wise constants with two levels $ρ^{l o w}$ and $ρ^{h i g h}$ , $Δ$ represents the gap between level transitions for each group. Middle image shows the Pareto front between group risks, since noise levels are not the same across subgroups, perfect fairness is not attainable. Equality of Risk requires pure degradation of service for group $a = 1$ . Both Naive and Re-balanced Naive classifiers do not attain the best possible no-harm classifier. Pareto-fair point is shown in green. Rightmost figure shows the trade-offs attainable between discrimination and mean risks, an equivalent problem to the risk trade-off figure.

3 Methods

Any loss function that is monotonically decreasing with $R_{a} (h), \forall a$ is minimized by a classifier in the Pareto front. Since we want to minimize the discrimination gap, we will build an adaptive loss function that is shares this property with the following form:

\begin{matrix} ϕ (h; μ, c) & = \sum a \in A R_{a} (h) + μ_{a} (R_{a} (h) - c)^{+ 2}, \end{matrix}

(1)

with $c < {min}_{a} R_{a} (h)$ and $μ = {μ_{a}}_{a \in A}$ . It can be shown that for convex Pareto sets (with respect to risk vectors $r = {R_{a} (h)}_{a \in A}$ ), there exist a set of $μ^{*}$ such that the Pareto-fair vector $r^{*}$ is a unique solution. In Algorithm 1 we jointly search for $μ^{*}$ , $r^{*}$ , and $h^{*}$ .

Given:

h_{θ}, D, γ > 0, ξ \in (0, 1), ζ \in (0, 1), lr,

μ \leftarrow 0, μ^{*} \leftarrow 0,, c \leftarrow 0, h^{*} \leftarrow h_{θ}

while Improving do

h_{θ} \leftarrow SGDwithEarlyStopping (h_{θ}, ϕ, μ, c, D)

// Optimize adaptive loss with fixed

μ

on dataset

D

r^{val} \leftarrow EvaluateRisk (h_{θ}, D^{val})

if $| | {\to Γ}_{A} (h) | |_{\infty} < Γ^{*}$ and $r^{Val}$ is not dominated by previous validation risks then

h^{*} \leftarrow h_{θ}, Γ^{*} \leftarrow | | {\to Γ}_{A} (h) | |_{\infty}, c_{old} \leftarrow c, c \leftarrow {min}_{a} \frac{r_{a}^{Val}}{k}

μ^{*} \leftarrow μ \cdot \frac{(r^{Val} - c_{old})^{+}}{(r^{Val} - c)^{+}}, a^{'} \leftarrow {a r g m a x}_{a} r_{a}^{Val}

else

\leftarrow

ζ

lr,

μ \leftarrow μ^{*}, γ \leftarrow γ ξ, h_{θ} \leftarrow h^{*}

end if

μ_{a^{'}} \leftarrow (1 + γ) μ_{a^{'}}

end while

Return:

h^{*}

// Exit loop due to excessive iterations or no improvement in fairness

Algorithm 1 ParetoFairOptimization

4 Experiments and Results

We evaluate our method on mortality prediction and skin lesion classification and show empirically how we reduce accuracy and risk disparity without unnecessary harm. We compare it against a naive classifier, class-rebalancing (equally sampled sub-groups), the post-processing framework presented in hardt2016equality, and the Disparate Mistreatment framework of zafar2017parity.

4.1 Predicting Mortality in Intensive Care Patients

We analyze clinical notes from adult ICU patients at the Beth Israel Deaconess Medical Center (MIMIC-III dataset) johnson2016mimic to predict patient mortality. We follow the pre-processing methodology outlined in chen2018my and use tf-idf statistics on the $10, 000$ most frequent words in clinical notes as input features. Fairness is measured with respect to age (under/over 55 years old), ethnicity, and outcome. We used a fully connected neural network with two 2048-unit hidden layers trained with Brier score (BS) loss. Table 1 shows accuracy and BS of all tested methodologies. We observe from columns (PF BS) and (PF Acc) that our model has the best accuracy and BS discrepancies. We can reduce accuracy disparities further by applying Hardt post processing (HPF Acc), at the cost of reducing overall performance. Note that, as expected, it is better to apply this post processing on our method than on the rebalanced naive classifier (HPF Acc vs HRen Acc). This illustrates the goal of the proposed Pareto-fair paradigm: develop the fairest algorithm with no unnecessary harm, and if (e.g., due to policy) the resulting fairness level needs to be improved, use other post-processing techniques on the Pareto-fair classifier such as hardt2016equality.

Out/Age/Race	Ratio	Na Acc	ReN Acc	Zafar Acc	PF Acc	HReN Acc	HPF Acc	ReN BS	PF BS
A/A/NW	5.7%	99.1 $\pm$ 0.4%	86.3 $\pm$ 1.5%	93.0 $\pm$ 1.4%	83.4 $\pm$ 2.6%	76.3 $\pm$ 1.9%	71.6 $\pm$ 3.2%	0.2 $\pm$ 0.02	0.25 $\pm$ 0.03
A/A/W	13.3%	98.8 $\pm$ 0.5%	86.3 $\pm$ 1.1%	90.0 $\pm$ 1.3%	83.2 $\pm$ 1.5%	76.7 $\pm$ 1.6%	71.8 $\pm$ 1.9%	0.2 $\pm$ 0.01	0.25 $\pm$ 0.02
A/S/NW	12.9%	97.5 $\pm$ 0.6%	76.5 $\pm$ 1.7%	81.8 $\pm$ 1.7%	71.4 $\pm$ 3.0%	76.4 $\pm$ 2.2%	71.3 $\pm$ 3.1%	0.31 $\pm$ 0.02	0.36 $\pm$ 0.03
A/S/W	56.7%	97.9 $\pm$ 0.3%	79.0 $\pm$ 0.6%	77.4 $\pm$ 0.7%	74.6 $\pm$ 1.6%	76.2 $\pm$ 2.2%	72.1 $\pm$ 1.2%	0.28 $\pm$ 0.01	0.34 $\pm$ 0.02
D/A/NW	0.4%	23.4 $\pm$ 9.4%	76.1 $\pm$ 8.5%	47.7 $\pm$ 9.6%	78.6 $\pm$ 6.1%	66.6 $\pm$ 9.9%	74.1 $\pm$ 9.2%	0.36 $\pm$ 0.06	0.34 $\pm$ 0.04
D/A/W	0.9%	32.6 $\pm$ 3.7%	80.1 $\pm$ 3.3%	60.5 $\pm$ 6.3%	83.3 $\pm$ 4.2%	66.4 $\pm$ 2.4%	73.6 $\pm$ 4.3%	0.29 $\pm$ 0.04	0.28 $\pm$ 0.03
D/S/NW	1.8%	21.4 $\pm$ 2.2%	66.9 $\pm$ 2.4%	48.2 $\pm$ 2.0%	73.3 $\pm$ 2.9%	64.8 $\pm$ 2.1%	71.2 $\pm$ 2.9%	0.42 $\pm$ 0.02	0.37 $\pm$ 0.03
D/S/W	8.3%	23.4 $\pm$ 2.2%	67.4 $\pm$ 1.9%	57.1 $\pm$ 2.2%	72.5 $\pm$ 3.6%	66.2 $\pm$ 2.9%	72.4 $\pm$ 3.5%	0.42 $\pm$ 0.02	0.37 $\pm$ 0.04
Sample Mean	-	89.5 $\pm$ 0.2%	78.9 $\pm$ 0.7%	78.0 $\pm$ 0.7%	75.7 $\pm$ 1.1%	75.1 $\pm$ 1.8%	71.9 $\pm$ 1.2%	0.29 $\pm$ 0.01	0.33 $\pm$ 0.01
Group Mean	12.5%	61.8 $\pm$ 1.5%	77.3 $\pm$ 1.3%	69.4 $\pm$ 1.3%	77.5 $\pm$ 0.7%	71.2 $\pm$ 1.1%	72.3 $\pm$ 1.0%	0.31 $\pm$ 0.01	0.32 $\pm$ 0.01
Discrepancy	56.3%	81.1 $\pm$ 2.5%	22.5 $\pm$ 2.5%	49.3 $\pm$ 4.4%	17.1 $\pm$ 2.6%	18.6 $\pm$ 3.2%	13.1 $\pm$ 5.6%	0.24 $\pm$ 0.03	0.16 $\pm$ 0.03

Table 1: Result summary table on MIMIC dataset. Sensitive attributes are the combination of Outcome (Alive or Deceased), Age (Adult or Senior) and Race (White or Non-White). Target attribute is Outcome. We report Brier scores (BS) and Accuracies (Acc) for Naive (Na), Rebalanced Naive (ReN), Zafar (Zafar), and Pareto-Fair (PF) classifiers; the prefix (H) indicates that Hardt post-processing was applied to control for Age and Race. Zafar was applied on top of the embeddings of the Naive classifier, as that produced the best results.

4.1.1 Skin Lesion Classification

The HAM10000 dataset tschandl2018ham10000 collects over $10, 000$ dermatoscopic images of skin lesions over a diverse population, lesions are classified in 7 categories. We used a pretrained DenseNet121 huang2017densely as our base classifier, and measured fairness with respect to diagnosis class, casting balanced risk minimization as a particular use-case for Pareto fairness. Table 2 shows our empirical results.

Groups	Ratio	PF Acc	ReN Acc	Na Acc	PF BS	ReN BS	Na BS
akiec	2.5%	51.9%	55.6%	3.7%	0.741	0.671	1.289
bcc	2.7%	56.7%	76.7%	56.7%	0.549	0.341	0.613
bkl	7.2%	59.5%	58.2%	36.7%	0.62	0.606	0.931
df	0.5%	66.7%	33.3%	0.0%	0.536	0.898	1.721
nv	81.0%	83.5%	90.9%	96.0%	0.241	0.128	0.054
vasc	1.3%	71.4%	85.7%	0.0%	0.36	0.246	1.604
mel	4.8%	53.8%	48.1%	32.7%	0.586	0.657	0.941
Sample Mean	-	78.6%	84.8%	83.6%	0.308	0.213	0.234
Group Mean	14.3%	63.4%	64.1%	32.3%	0.519	0.507	1.022
Discrepancy	80.4%	31.7%	57.5%	96.0%	0.501	0.769	1.667

Table 2: Group-specific risks and accuracies for Naive (Na), Rebalanced Naive (ReN), and Pareto-Fair (PF) classifiers. Lesion Type was chosen as both target and sensitive variable, they are classified as Actinic keratoses and intraepithelial carcinoma (akiec), basal cell carcinoma (bcc), benign keratosis-like lesions (bkl), dermatofibroma (df), melanoma (mel), melanocytic nevi (nv) and vascular lesions (vasc). The Pareto-fair classifier exhibits the lowest discrepancy on Acc and BS while still being Pareto-optimal.

5 Discussion

Here we explore the problem of reducing the risk disparity gaps in the most ethical way possible (i.e., minimizing unnecessary harm). We provide an algorithm that can be used with any standard classification or regression loss to bridge risk disparity gaps without introducing unnecessary harm. We show its performance on two real-world case studies; we take advantage of the fact that our method does not require test-time access to sensitive attributes to frame balanced classification as a fairness problem. In future work, we wish to analyze if we can automatically identify high-risk sub-populations as part of the learning process and attack risk disparities as they arise, rather than relying on preexisting notions of disadvantaged groups or populations. We believe that no-unnecessary-harm notions of fairness are of great interest for several applications, especially so on domains such as healthcare.

Fairness With Minimal Harm: A Pareto-Optimal Approach For Healthcare

Authors

Minimax Pareto Fairness: A Multi Objective Perspective

Achieving Fairness at No Utility Cost via Data Reweighing

Why Is My Classifier Discriminatory?

Assessing Fairness in Classification Parity of Machine Learning Models in Healthcare

What is Fair? Exploring Pareto-Efficiency for Fairness Constrained Classifiers

Fairness Without Demographics in Repeated Loss Minimization

On the Fairness of Swarm Learning in Skin Lesion Classification

1 Introduction

Main Contributions

2 Problem Statement

Definition 2.1.

Definition 2.2.

3 Methods

4 Experiments and Results

4.1 Predicting Mortality in Intensive Care Patients

4.1.1 Skin Lesion Classification

5 Discussion

References

Fairness With Minimal Harm: A Pareto-Optimal Approach For Healthcare

Authors

Related Research

Minimax Pareto Fairness: A Multi Objective Perspective

Achieving Fairness at No Utility Cost via Data Reweighing

Why Is My Classifier Discriminatory?

Assessing Fairness in Classification Parity of Machine Learning Models in Healthcare

What is Fair? Exploring Pareto-Efficiency for Fairness Constrained Classifiers

Fairness Without Demographics in Repeated Loss Minimization

On the Fairness of Swarm Learning in Skin Lesion Classification

This week in AI

1 Introduction

Main Contributions

2 Problem Statement

Definition 2.1.

Definition 2.2.

3 Methods

4 Experiments and Results

4.1 Predicting Mortality in Intensive Care Patients

4.1.1 Skin Lesion Classification

5 Discussion

References