Models of fairness in federated learning

In many real-world situations, data is distributed across multiple locations and can't be combined for training. Federated learning is a novel distributed learning approach that allows multiple federating agents to jointly learn a model. While this approach might reduce the error each agent experiences, it also raises questions of fairness: to what extent can the error experienced by one agent be significantly lower than the error experienced by another agent? In this work, we consider two notions of fairness that each may be appropriate in different circumstances: egalitarian fairness (which aims to bound how dissimilar error rates can be) and proportional fairness (which aims to reward players for contributing more data). For egalitarian fairness, we obtain a tight multiplicative bound on how widely error rates can diverge between agents federating together. For proportional fairness, we show that sub-proportional error (relative to the number of data points contributed) is guaranteed for any individually rational federating coalition.

• 6 publications
• 68 publications
04/22/2020

Hierarchically Fair Federated Learning

Federated learning facilitates collaboration among self-interested agent...
10/10/2020

Fairness-aware Agnostic Federated Learning

Federated learning is an emerging framework that builds centralized mach...
09/19/2021

Improving Fairness for Data Valuation in Federated Learning

Federated learning is an emerging decentralized machine learning scheme ...
06/17/2021

Optimality and Stability in Federated Learning: A Game-theoretic Approach

Federated learning is a distributed learning paradigm where multiple age...
07/10/2022

Mechanisms that Incentivize Data Sharing in Federated Learning

Federated learning is typically considered a beneficial technology which...
02/20/2020

Dynamic Federated Learning

Federated learning has emerged as an umbrella term for centralized coord...
12/20/2021

REFORM: Reputation Based Fair and Temporal Reward Framework for Crowdsourcing

Crowdsourcing is an effective method to collect data by employing distri...

1 Introduction

In traditional machine learning, an agent (such as a hospital, school, or company) trains a model based on data it has locally. In many cases, the agent may have access to a limited amount of local data, which may mean that any model created with it will have relatively high error. However, in many real-world scenarios, multiple agents all are trying to solve similar problems using related local data sets. For example, consider a hospital aiming to train a model predicting patient outcomes. Local learning would rely on the hospital’s own data from observations of its patience. However, it may be the case that multiple hospitals in the same area all similarly have local patient data related to this problem. This raises the question - could it be possible to leverage this related data to help build a better model? Federated learning is a new and rapidly growing field (

[10, 8]) that combined learned parameters from multiple federating agents in order to train models with lower expected error. It has also been the subject of game theoretical analysis, such as in [2, 7, 6, 3, 4], and in the summary paper [19].

In general, federated learning methods can provide benefits to federating agents: allowing them to produce models with lower error. However, federated learning may itself produce additional problems, especially around fairness. Often different agents federating with the same model will see different levels of error, potentially as a result of differences in how much data each agent contributed to the overall model. While this disparity might be undesirable, in federated learning, a “fair” solution can have multiple definitions, reflecting different values and goals of the particular situation.

For example, consider a case with two hospitals and , where with fewer relevant data points than . It might be the case that has less data because it is a smaller hospital with fewer resources. In this case, a reasonable fairness goal might be for the federating solution to rectify this inequity, producing errors for and that are approximately equal. The goal of having error rates that are the same or similar is reflected in egalitarian fairness. However, a different scenario might call for a different fairness goal. Consider the case where hospitals and have similar access to resources, but hospital has less data because it has devoted less effort to collecting data. In this case, a reasonable fairness objective might be to reward player for its higher contributions. One specific approach might be to have each agent’s error rate be inversely proportional to the amount of data it contributes: this motivates proportional fairness. While egalitarian and proportional fairness can’t be simultaneously achieved, each definition reflects goals that could be reasonable for different situations. In the federated learning literature, multiple papers separately discuss notions related to egalitarian and proportional fairness, such as [14, 11, 12, 23].

The present work:
In this work, we provide the first theoretical analysis and comparison of egalitarian and proportional fairness. In order for this analysis to be tractable theoretically, we build on the simplified model of federated learning first proposed in Donahue and Kleinberg [3], whose framework we summarize later in this work for convenience of the reader.

• For egalitarian fairness, we give a tight multiplicative bound on how dissimilar the error rates of federating agents can be. This bound varies based on the number of samples the largest federating agent contributes, as well as a property of the federating problem relating to the ratio of noise and bias in the learning task. We show that this bound holds for a wide range of federating methods satisfying a few straightforward properties.

• For proportional fairness, we show that in general there exist federating coalitions that could display error rates that are sub-proportional, proportional, or super-proportional. That is, the error a federating agent experiences, compared to the error another agent experiences, could be either more or less than what the ratio of the size of their datasets suggests. However, we show that any federating coalition with super-proportional error fails to be individually rational: at least one agent could achieve lower error by leaving the federating coalition for local learning.

2 Model and assumptions

First, we will present the model we will use to analyze federated learning. We say that there are total federating agents (sometimes referred to as players). Each agent has drawn data points, labeled by its true labeling distribution , where are their true local parameters and is some labeling function. A player’s goal is to learn a model

with low expected error on its own distribution. If a player opts for local learning, then it uses its local estimate of these parameters

to predict future data points. If a set of players are federating together, we say that they are in a coalition or cluster together. A federating method describes how players combine their learned parameters into federating models. For example, the most common federating method is the weighted average method in Equation 1, though other methods are also in use ([13]):

 ^θC=1∑i∈Cni⋅∑i∈Cni⋅^θi (1)

A federating player obtains error , where denotes the federating method being used. Note that in general : players federating in the same coalition may experience different error rates. For example, if player has more samples than player , then as calculated in Equation 1 will be weighted more towards player , meaning that player will have lower expected error than .

Because it is the most straightforward method, it is sometimes called “vanilla” federated learning. Alternative ways of federation might involve customizing the model for individuals, as in domain adaptation. For example, [3] models three methods of federation: vanilla (called “uniform”), as well as two models of domain adaptation, fine-grained and coarse-grained.

2.1 Fairness definitions and ethical considerations

Fairness in federated learning is an especially rich topic: naturally, no one set of definitions is going to resolve the complex questions it raises. (For a survey of fairness issues in federated learning and methods to address them, refer to [17]). In this work, our goal is to theoretically evaluate two of the most common notions of fairness in federated learning.

We adopt the terminology from Xu and Lyu [20] calling the first “egalitarian”. Egalitarian notions of fairness roughly revolve around ensuring agents have levels of error that are all roughly comparable. For example, Mohri et al. [14] works to minimize the error of the worst-off player, while Du et al. [5] considers a similar notion with an explicit fairness penalty. By contrast, Li et al. [11] has a more flexible goal of minimizing the spread of error rates. Abay et al. [1] analyzes multiple causes of unfairness in federated learning and potential solutions. Other papers that consider notions related to egalitarian fairness include [21, 22, 16, 9].

The motivation for this notion of fairness is that federating agents may differ in how much data they have for reasons beyond their control: persistent inequities in resources, for example. The goal of federation, in this view, is to try and ameliorate these injustices by bringing error rates more close to each other. We formalize this idea in Definition 1.

Definition 1.

A coalition satisfies -egalitarian fairness if for some constant ,

 erri(C)errj(C)≤λ ∀i,j∈[M]

By contrast, proportional notions of fairness generally focus on ensuring that different agents are rewarded for contributing more to the overall model. For example, Lyu et al. [12], Xu and Lyu [20] both use gradient-based methods to detect players contributing useful model updates (as opposed to noise) and reward them with more powerful variants of the model. Song et al. [18] takes a similar approach with the goal of allocating profit. Zhang et al. [23] proposes having a trusted party track the amount of effort each player is contributing, which will then be used to reward or penalize agents with access to different models.

Under the view of proportional fairness, the number of samples a player contributes to the federated model is under their control, so we wish to reward players who contribute more. In Definition 2, we formalize the idea of proportional fairness by describing how error rates relate to the number of samples that a player has.

Definition 2.

A coalition with satisfies sub-proportional error if, for all pairs of players and , with with ,

 erri(C)≤njni⋅errj(C) ∀i,j∈[M] ni≤nj

The case where this inequality is strict is “strict sub-proportional error”. The case where this inequality is reversed is “super-proportional error”:

 erri(C)≥njni⋅errj(C) ∀i,j∈[M] ni≤nj

Note that sub-proportional error means that smaller players get lower error than proportionality would suggest, while super-proportional error means that smaller players get higher error than proportional. Either case may be desirable under different situations: we will restrict our attention to describing when either is possible to achieve.

2.2 Theoretical model of federation from Donahue and Kleinberg [3]

Federated learning has been the subject of both applied and theoretical analysis; our focus here is on the theoretical side. In order for our theoretical fairness analysis to be feasible, we require a model that gives exact errors for each player, rather than bounds: these are needed in order to be able to argue that the ratio of the errors two players experience is bounded, for example.

We opt to use the model developed in our prior work Donahue and Kleinberg [3], which produces the closed-form error value seen in Lemma 1 below. While we work within this model, we emphasize that [3] asked different questions from this paper’s focus: our prior work focused on developing the federated learning model and analyzing the stability of federating coalitions, while our current work analyzes optimality and Price of Anarchy. (Separately, in [4] we similarly build on [3], but with the orthogonal goal of analyzing the optimality of federated learning: which arrangements minimize total error.)

Lemma 1 (Lemma 4.2, from [3]).

Consider a mean estimation task as follows: player is trying to learn its true mean . It has access to samples drawn i.i.d. , a distribution with mean

and variance

. Given a population of players, each has drawn parameters from some common distribution . A coalition federating together produces a single model based on the weighted average of local means (Eq. 1). Then, the expected mean squared error player experiences in coalition is:

 errj(C)=μe∑i∈Cni+σ2⋅∑i∈C,i≠jn2i+(∑i∈C,i≠jni)2(∑i∈Cni)2 (2)

where (the average noise in data sampling) and (the average distance between the true means of players).

Note that [3]

also analyzes a linear regression game with a similar cost function, though in this work we will restrict our attention to the mean estimation game.

We use some of the same notion and modeling assumptions as [3]. For example, we use to refer to a coalition of federating agents and to refer to a collection of coalitions that partitions the agents. We will use to refer to the total number of samples present in coalition : . In a few lemmas we will re-use minor results proven in [3], citing them for completeness.

Finally, it is worth emphasizing key differences between this current work and [3]. The focus of [3] is defining a theoretical model of federated learning and taking a game-theoretic analysis of the stability of such an arrangement. While we use the theoretical model developed in [3], in our work we analyze completely distinct questions: what are reasonable definitions of “fairness” in federated learning? When are certain fairness notions guaranteed to hold? Additionally, this paper is in some ways more general: while some key results in [3] only allow players to have two different numbers of samples (“small” or “large”), every result in our work holds for arbitrarily many players with arbitrarily many different numbers of samples.

2.3 Research ethics and social impact

This work is primarily theoretical and involves no real data. However, it touches on questions with multiple possible societal implications. As we discussed previously, there are many reasonable definitions of fairness in federated learning. For example, fairness could mean something besides error rates, such as equality of access to opportunities or access to recourse. In this work, we narrow our focus to two of the most commonly used and natural definitions, but it is important to note that this by no means is the last word on fairness in federated learning. Besides fairness, federated learning also has privacy concerns. While federated learning involves sharing model parameters, rather than raw data, under some circumstances it still may be possible to learn information about the data distribution of specific players [15]. Beyond this, users may have data control concerns: a patient may be willing to have their data used to build a model for the hospital they went to, but not be willing to have their data (or parameters derived from training on their data) shared with other hospitals. Such objections, while reflecting reasonable concerns, would make federated learning more difficult or potentially infeasible. Finally, one core assumption of this paper is that the problem at hand will be improved by a machine learning model with lower error. In many situations, a different approach might be more suitable, such as an algorithm that is more explainable or a non-machine learning solution.

3 Motivating example

As a motivating example, we will expand on the hospital application area mentioned in the introduction. Consider the case where there are two hospitals that are considering federating together in order to jointly estimate a parameter or model of interest (for example, patient recovery time after a procedure). Each hospital ideally wants to end up with a model with low error on its own patient population. However, society as a whole also has an interest that these models be fair. As we discussed in previous sections, “fairness” could have multiple different definitions. In this work, we will focus primarily on the ratio of error rates between agents that are federating together - in this case, the ratio of errors between the two hospitals.

3.1 Hypothetical scenarios

In Table 2, we present a series of hypothetical scenarios for how error rates between the two hospitals could differ. We will assume one hospital contributes less data (is “small”) and one contributes more data (is “large”). Note that “small” and “large” refers solely to how many data points each hospital contributes to the federating model. The “large” hospital may actually serve fewer patients (and vice versa), depending on their data collection practices. We use to refer to the error the small hospital experiences when federating with the large hospital (and to refer to the error the large hospital experiences).

Each row in Table 2 contains a hypothetical scenario, which allows us to illustrate potential fairness concerns. In the first row, we consider the case where both the small and large hospitals, when federating together, have identical error, leading to an error ratio of 1. This scenario could be desirable under egalitarian fairness, but would be less desirable under proportional fairness, since it wouldn’t reward the larger hospital for having contributed more data. The second row gives a scenario where the small hospital has even lower error than the large hospital, when they are federating together. This might satisfy some strict notions of egalitarian fairness (especially if the goal is to attempt to compensate for historical and/or continuing injustices). However, this situation will certainly not satisfy any notions of proportional fairness, given that the large hospital appears to be penalized for having contributed more data.

In row 3, we see a scenario where the large hospital has slightly lower error than the small hospital. Specifically, the error ratio is 3 (the small hospital has error that is 3 times as high). In row 4, we see a case where the large hospital has error that is much lower, leading to an error ratio of 15. Clearly, under egalitarian fairness, scenario 4 would be much worse than scenario 3. For proportional fairness, which is preferred depends on the relative sizes of the small and large hospitals: exactly how many data points they contribute. For simplicity, we will assume the small hospital contributes 8 data points and the large hospital contributes 20, for a ratio of 3.33. In this case, scenario 3 gives an error ratio that is closer to the ratio of data contributed, and would likely be preferred under proportional fairness as well.

3.2 Actual scenarios

Throughout the work of this paper, we will show that, in a variety of common federation methods, the typical outcome looks like Scenario 3: the larger agent gets lower error than the smaller player, but the error ratio is upper bounded.

Table 2 gives different scenarios for error rates, but this time with values calculated from an actual model of federated learning (as compared with Table 2, where the values were purely hypothetical). Each of the error rates are calculated according to vanilla federation from Lemma 1 (as derived from [3]). In this example, we fix the noise-bias ratio (as defined in Lemma 1) to be . The small agent again contributes 6 data points, but here we allow the number of data points contributed by the large agent to vary from 20 to 30 to 40.

In Section 4, we explore egalitarian fairness and show that the error ratio can be upper and lower bounded. Specifically, we show that it is upper bounded by , where (the noise-bias ratio). We also show that this bound is tight: we can create scenarios that get arbitrarily close to this upper bound of . Additionally, we show that the error ratio must be at least 1 (the small player must get error that is at least as high as the large player). In Table 2, these results are illustrated. The third column gives the ratio of errors. Note that in all three rows in Table 2, the large agent experiences strictly lower error than the small agent, so this ratio is strictly greater than 1. The fourth column gives the upper bound, which for every entry is larger than the error ratio, which we prove in Section 4 must hold.

Finally, in Section 5, we consider proportional fairness. The fifth column of Table 2 gives the ratio of the sizes of each of the two agents. In exactly proportional fairness, the ratio of errors (3rd column) would exactly match the ratio of sizes (5th column). In this table, the middle row (when the large agent has 30 samples) exactly satisfies proportional fairness. The first row (where the large agent has 20 samples) has sub-proportional error: the small agent has lower error than proportionality would suggest (because the error ratio is lower than the ratio of sizes). In the third row (where the large agent has 40 samples), we see super-proportional error: the small player experiences higher error than what proportionality would suggest (because the error ratio is higher than the ratio of sizes). This table illustrates that federating agents could experience proportional, sub-proportional, or super-proportional error rates. However, in Section 5, we will prove that any federating coalition with super-proportional error must fail to be individually rational: at least one agent would reduce its error by switching to local learning (building a model with only its own data). In this case, Lemma 1 gives that the large player would experience error of in local learning, which is strictly lower than the error of 0.251 it experiences in federated learning.

4 Egalitarian fairness

In this section, we will analyze egalitarian fairness, where a typical goal is to ensure that all players have similar levels of error. This may be motivated by the fact that players may differ in their circumstances (differing amounts of data) for reasons out of their control. For example, imagine a low-resource hospital that is federating with a high-resource hospital. The low-resource hospital may have access to less local data for historical and/or continuing reasons of discrimination and unequal access to technology and other resources. This motivation would suggest that federated learning should seek to close the gap the low resource and high resource hospital, making their error as close to equal as possible. In some cases, it may even be desirable for the low-resource (small) hospital to have lower error than the high-resource hospital, potentially as a way to begin to compensate for its lack of resources. In this section, we give theoretical bounds for what types of egalitarian fairness levels can be obtained.

First, we begin with the simplest and most commonly-studied federation method: vanilla federation, where parameters are calculated as the weighted average, as in Equation 1. Theorem 1 shows that, for this federation method, smaller players always get higher error, but that there exists a tight multiplicative bound for egalitarian fairness (the error ratio between small and large players is upper bounded).

Theorem 1.

Vanilla federation satisfies the following three properties:

• For any two players federating in the same coalition such that , the smaller player always experiences error that is at least as high as the larger player (strictly higher if ).

• Any federating coalition such that the largest player has no more than samples satisfies -egalitarian fairness.

• This bound is tight (up to an additive factor of ).

We won’t directly give a proof of Theorem 1, but instead, we will show that it is a consequence of a more general result presented in Theorem 2. This theorem shows that the same egalitarian fairness bounds hold for any modular function, where Definition 3, below, gives a formal definition of what this means.

Theorem 2.

Any modular federating method (as defined in Definition 3) has the following three egalitarian fairness properties:

• For any two players federating in the same coalition such that , the smaller player always experiences error that is at least as high as the larger player (strictly higher if ).

• Any federating coalition such that the largest player has no more than samples satisfies -egalitarian fairness.

• This bound is tight (up to an additive factor of ).

Definition 3.

Consider any coalition with players such that . Then, a federating method is called modular if it satisfies the following five properties:

Property 1: The large player always has lower error than the small player, with the error being strictly lower if (the large player is strictly larger).

 errs(C)errl(C)≥1

Property 2: The worst-case situation for the error ratio (the ratio of the small player’s error to the large player’s error) is always in the two-player case. That is,

 errs(C)errl(C)≤errs({ns,nℓ})errl({ns,nℓ)}

Property 3: The error ratio increases as the large player gets more samples: the small player’s error either increases or else decreases more slowly than the large player’s error decreases. That is,

 ∂∂nℓerrs({ns,nℓ})errl({ns,nℓ)}≥0

Property 4: The error ratio decreases as the small player gets more samples: the large player’s error either increases or else decreases more slowly than the small player’s error decreases. That is,

 ∂∂nserrs({ns,nℓ})errl({ns,nℓ)}≤0

Property 5: As (the large player has many more samples than the small player), the error ratio converges to the following fraction:

 limnsnℓ→0errs({ns,nℓ)}errl({ns,nℓ)}=μenℓ+2σ2μenℓ

First, it may be useful to discuss the properties in Definition 3. Property 1 is simply stating that, for any pair of players both federating in the same coalition, the smaller of the two always has higher error. The remaining four properties contribute to the upper bound (and showing that this bound is tight). First, Property 2 states that the error ratio between any two players is highest when they are federating in a two-player coalition (without any other players). This property is especially helpful because the two-player coalition is much simpler to analyze. Properties 3 and 4 consider this two-player coalition, showing that the error ratio is highest when the large player has many samples and the small player has very few. Finally, Property 5 describes the limit of the error ratio in the case where the large player has many more samples than the small player, stating that it must go to a fixed ratio depending on the noise-bias ratio, as well as the number of samples the large player has.

Given these properties, it becomes straightforward to prove Theorem 2:

Proof of Theorem 2.

Property 1 automatically satisfies the first property in Theorem 2.

Proving the bound is given by applying the four properties:

 errs(C)errl(C)≤errs({ns,nℓ})errl({ns,nℓ)}≤errs({ns,c⋅μeσ2})errl({ns,c⋅μeσ2})
 ≤μenℓ+2σ2μenℓ≤μenℓ+2σ2μenℓ∣∣∣nℓ=cμeσ2=σ2c+2σ2σ2c=2c+1

Next, we will show that the bound is tight. Our goal is to show that, for all , there exists some set of parameters such that

 errs({ns,nℓ})errl({ns,nℓ})≥2c+1−ϵ

The version of Property 4 can be written as: for all , there exists a such that

 nsnℓ<δ⇒μenℓ+2σ2μenℓ−errs({ns,nℓ)}errl({ns,nℓ)}<ϵ

Setting gives that Property 4 becomes:

 σ2c⋅μe<δ⇒2c+1−errs({ns,nℓ)}errl({ns,nℓ)}<ϵ

Because are free parameters, we can set in order to satisfy the precondition. ∎

Finally, we will show that there exist multiple federating methods that are modular. Specifically, Lemma 2 shows that uniform federation is modular and Lemma 3 shows that fine-grained federation is modular. Proofs for both are given in Appendix A.

Lemma 2.

Uniform federation is modular.

Lemma 3.

Fine-grained federation is modular.

These results, taken together, show that, while small players must get higher error than larger players in the same federation, the error that small players experience is upper bounded. Specifically, this upper bound is a multiple of the error that the larger player experiences, and depends on the size of the larger player, as a function of the noise-bias ratio. Because this bound is tight, we know that no lower bound is possible.

5 Proportional fairness

Next, we consider proportional fairness, which revolves around the idea that players who contribute more data should be rewarded with lower error rates. This notion relies on the view that players do have control over how much data they obtain. For example, imagine again two hospitals, this time with comparable levels of resources, but one of them has devoted a substantially larger share of its annual budget towards collecting high-quality local data. As a result, it has substantially more local data it can use for training. A federating coalition that gives both hospitals roughly similar error rates might be seen as failing to recognize and reward that hospital’s extraordinary efforts towards data collection.

First, Lemma 4 shows empirically that there exist situations where a federating coalition could have proportional, sub-, or super-proportional error rates.

Lemma 4.

There exist cases where a coalition using uniform federation satisfies proportional, strict sub-proportional, and strict sup-proportional error.

Proof.

This proof is constructive: we give an explicit example of such situations. Recall that in Section 3, the “Actual scenarios” section already gave such an example. This is reproduced for convenience here as Table 3. Throughout, we use the error form given in Lemma 1 with . We will consider the case where there are exactly two players: , and with a varying number of samples.

The first row gives the error that the small and large players experience with . Note that the ratio of their errors (given in the third column) is strictly less than the (inverse) ratio of their sizes (given in the fifth column). This is an example of sub-proportional error: the small player gets lower error than proportional scaling with respect to their sizes would suggest. The second row gives the error that the players experience with . Here, the ratio of their errors exactly matches the ratio of their sizes: this is a case of exact proportionality. Finally, the third row gives the error the players experience when . Here the ratio of their errors is strictly greater than the ratio of their sizes: this represents super-proportionality.

In later analyses, we will show that all arrangements with super-proportional error fail to be individually rational: at least one federating agent would prefer local learning. For the example with , the error form from Lemma 1 gives that the large player would obtain error of with local learning. This is strictly less than its federating error of , so it is incentivized to leave the federating coalition.

Table 3 also illustrates the bound derived in Theorem 2. Given that the error/bias ratio here is , the bound scales with the size of the largest player, in multiples of 10. The bound in the fifth column is always strictly greater than the ratio in the third column, illustrating the result from Theorem 2.

The result of Lemma 4 is to show that, in general, a federating coalition might experience error that is sub or super-proportional. Next, Theorem 3 gives conditions for when sub-proportionality is guaranteed: specifically, whenever the coalition is individually rational. A coalition is individually rational if each player in the coalition prefers being in to doing local learning.

Theorem 3.

Any individually rational coalition using vanilla federation satisfies sub-proportionality of errors.

As mentioned previously, sub-proportionality of errors means that smaller players (contributing fewer samples to the federating model) experience error that is lower than proportionality would suggest. This may or may not be desirable, depending on the federating situation. The full proof of Theorem 3 is given in Appendix A, but we include a proof sketch below.

Proof Sketch.

We will show this result by proving the contrapositive: if sub-proportionality of errors is violated, then at least one player wishes to leave that coalition for local learning (so individual rationality is violated).

In our analysis, we will consider two players , though the players could be arbitrarily situated with respect to . We will assume that all of the players are federating together in coalition . For notational convenience, we will write the federating coalition of interest as , so refers to all players except .

The case where sub-proportionality fails to hold is:

 errS(A)⋅ns≥errL(A)⋅nℓ

Note that the sign of the inequality is flipped, as compared to sub-proportionality.

The full proof, in Appendix A, shows that this inequality can be rewritten as:

 −2σ2⋅NC+μens⋅NC+σ2ns⋅(∑i∈Cn2i+N2C)2σ2−μens≤nℓ (3)

Equation 3 gives a lower bound on how many samples the large player has to have before sub-proportionality is violated. Next, the proof obtains a similar lower bound for how many samples the large player would need before it would prefer to defect: we will show this lower bound is lower than Equation 3 above.

The case when the large player wishes to defect from to local learning (or is ambivalent about defecting) is when:

 errL(πl)≤errL(A)

The full version of the proof shows that this can be simplified to:

 −2σ2⋅NC+μens⋅NC+σ2ns⋅(∑i∈Cn2i+N2C)2σ2−μens≤nℓ (4)

The full proof concludes by showing that the bound in Equation 4 is lower than the bound in Equation 3: any situation where the term is large enough so that sub-proportionality is violated, we know that the large player already wishes to leave. ∎

Overall, the bound in Theorem 3 is different from the results on egalitarian fairness in Section 4. For egalitarian fairness, we obtained results that hold for any pair of players federating together. In this section, for proportional fairness, we obtain a guarantee that holds whenever a federating coalition is individually rational: whenever all federating players obtain lower error than they would in local learning. The result in Theorem 3 could be useful if federating is truly voluntary: then, any federating coalition where smaller players experience super-proportional could never exist, because it would unravel into local learning. However, this result could still be useful even if federating isn’t something individual players could opt out of. Theorem 3 shows that any federating coalition with super-proportional error is one that also has the problem that some players receive higher error than they would in local learning - which indicates a broader failure of federation, which may prompt any federating organizer to change their federating groups or coalitions.

6 Discussion

In this work, we have used a theoretical model to analyze the fairness properties of federated learning. We formalized two notions of fairness already under analysis in federated learning research and gave conditions when those fairness properties are or are not satisfied. For egalitarian fairness, we gave a tight multiplicative bound on the gap in error rates between any two federating players. For proportional fairness, we showed that any individually rational federating coalition must satisfy sub-proportional fairness.

Future research could consider questions in two directions. First: which other notions of fairness might be relevant? It might be the case that an additive version of egalitarian fairness would be more appropriate for certain situations, for example, or a version of proportional fairness based on some metric related to the quality of data points, rather than how their quantity. Secondly, which fairness properties might other methods federations display? While we have analyzed two federation methods (vanilla and fine-grained), it could be fruitful to analyze other methods to determine how well they achieve certain fairness results.

Acknowledgments

This work was supported in part by a Simons Investigator Award, a Vannevar Bush Faculty Fellowship, MURI grant W911NF-19-0217, AFOSR grant FA9550-19-1-0183, ARO grant W911NF19-1-0057, a Simons Collaboration grant, a grant from the MacArthur Foundation, and NSF grant DGE-1650441. We are grateful to the AI, Policy, and Practice working group at Cornell for invaluable discussions.

References

• [1] A. Abay, Y. Zhou, N. Baracaldo, S. Rajamoni, E. Chuba, and H. Ludwig (2020) Mitigating bias in federated learning. External Links: 2012.02447 Cited by: §2.1.
• [2] A. Blum, N. Haghtalab, R. L. Phillips, and H. Shao (2021) One for one, or all for all: equilibria and optimality of collaboration in federated learning. arXiv preprint arXiv:2103.03228. Cited by: §1.
• [3] K. Donahue and J. Kleinberg (2021) Model-sharing games: analyzing federated learning under voluntary participation. AAAI 2021. External Links: Link Cited by: Appendix A, §1, §1, §2.2, §2.2, §2.2, §2.2, §2.2, §2, §3.2, Lemma 1.
• [4] K. Donahue and J. Kleinberg (2021) Optimality and stability in federated learning: a game-theoretic approach. In Advances in Neural Information Processing Systems, External Links: Link Cited by: §1, §2.2.
• [5] W. Du, D. Xu, X. Wu, and H. Tong (2020) Fairness-aware agnostic federated learning. ArXiv abs/2010.05057. Cited by: §2.1.
• [6] F. Grimberg, M. Hartley, S. P. Karimireddy, and M. Jaggi (2021) Optimal model averaging: towards personalized collaborative learning. arXiv preprint arXiv:2110.12946. Cited by: §1.
• [7] C. Hasan (2021) Incentive mechanism design for federated learning: hedonic game approach. External Links: 2101.09673 Cited by: §1.
• [8] P. Kairouz, H. B. McMahan, B. Avent, A. Bellet, M. Bennis, A. N. Bhagoji, K. Bonawitz, Z. Charles, G. Cormode, R. Cummings, R. G. L. D’Oliveira, S. E. Rouayheb, D. Evans, J. Gardner, Z. Garrett, A. Gascón, B. Ghazi, P. B. Gibbons, M. Gruteser, Z. Harchaoui, C. He, L. He, Z. Huo, B. Hutchinson, J. Hsu, M. Jaggi, T. Javidi, G. Joshi, M. Khodak, J. Konečný, A. Korolova, F. Koushanfar, S. Koyejo, T. Lepoint, Y. Liu, P. Mittal, M. Mohri, R. Nock, A. Özgür, R. Pagh, M. Raykova, H. Qi, D. Ramage, R. Raskar, D. Song, W. Song, S. U. Stich, Z. Sun, A. T. Suresh, F. Tramèr, P. Vepakomma, J. Wang, L. Xiong, Z. Xu, Q. Yang, F. X. Yu, H. Yu, and S. Zhao (2019) Advances and open problems in federated learning. External Links: 1912.04977 Cited by: §1.
• [9] S. Kanaparthy, M. Padala, S. Damle, and S. Gujar (2021) Fair federated learning for heterogeneous face data. arXiv preprint arXiv:2109.02351. Cited by: §2.1.
• [10] T. Li, A. K. Sahu, A. Talwalkar, and V. Smith (2020-05) Federated learning: challenges, methods, and future directions. IEEE Signal Processing Magazine 37 (3), pp. 50–60. External Links: ISSN 1558-0792, Link, Document Cited by: §1.
• [11] T. Li, M. Sanjabi, A. Beirami, and V. Smith (2019) Fair resource allocation in federated learning. External Links: 1905.10497 Cited by: §1, §2.1.
• [12] L. Lyu, X. Xu, and Q. Wang (2020) Collaborative fairness in federated learning. ArXiv abs/2008.12161. Cited by: §1, §2.1.
• [13] H. B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas (2016) Communication-efficient learning of deep networks from decentralized data. External Links: 1602.05629 Cited by: §2.
• [14] M. Mohri, G. Sivek, and A. T. Suresh (2019) Agnostic federated learning. External Links: 1902.00146 Cited by: §1, §2.1.
• [15] V. Mothukuri, R. M. Parizi, S. Pouriyeh, Y. Huang, A. Dehghantanha, and G. Srivastava (2021) A survey on security and privacy of federated learning. Future Generation Computer Systems 115, pp. 619–640. Cited by: §2.3.
• [16] A. Papadaki, N. Martinez, M. Bertran, G. Sapiro, and M. Rodrigues (2021) Federating for learning group fair models. arXiv preprint arXiv:2110.01999. Cited by: §2.1.
• [17] Y. Shi, H. Yu, and C. Leung (2021) A survey of fairness-aware federated learning. arXiv preprint arXiv:2111.01872. Cited by: §2.1.
• [18] T. Song, Y. Tong, and S. Wei (2019) Profit allocation for federated learning. 2019 IEEE International Conference on Big Data (Big Data), pp. 2577–2586. Cited by: §2.1.
• [19] X. Tu, K. Zhu, N. C. Luong, D. Niyato, Y. Zhang, and J. Li (2021) Incentive mechanisms for federated learning: from economic and game theoretic perspective. arXiv preprint arXiv:2111.11850. Cited by: §1.
• [20] X. Xu and L. Lyu (2020) Towards building a robust and fair federated learning system. ArXiv abs/2011.10464. Cited by: §2.1, §2.1.
• [21] Y. Zeng, H. Chen, and K. Lee (2021) Improving fairness via federated learning. arXiv preprint arXiv:2110.15545. Cited by: §2.1.
• [22] F. Zhang, K. Kuang, Y. Liu, C. Wu, F. Wu, J. Lu, Y. Shao, and J. Xiao (2021) Unified group fairness on federated learning. arXiv preprint arXiv:2111.04986. Cited by: §2.1.
• [23] J. Zhang, C. Li, A. Robles-Kelly, and M. Kankanhalli (2020) Hierarchically fair federated learning. ArXiv abs/2004.10386. Cited by: §1, §2.1.

Appendix A Proofs

See 2

Proof.

In this proof, we will denote the uniform federation error by . Because Definition 3 has five components, this proof will have five sections.
Property 1:
For the first property, we wish to show that (for any federating coalition ), the large player always has strictly lower error than the small player, or:

 errs(C)errl(C)≤1⇔errs(C)≥errl(C)

Using the form of error found in Lemma 1 (Equation 2), we can rewrite this as:

 μe∑i∈Cni+σ2⋅∑i∈Cn2i−n2s+(∑i∈Cni−ns)2(∑i∈Cni)2≥μe∑i∈Cni+σ2⋅∑i∈Cn2i−n2ℓ+(∑i∈Cni−nℓ)2(∑i∈Cni)2

Cancelling common terms:

 ∑i∈Cn2i−n2s+(∑i∈Cni−ns)2 ≥∑i∈Cn2i−n2ℓ+(∑i∈Cni−nℓ)2 −n2s+(∑i∈Cni)2−2⋅ns⋅(∑i∈Cni)+n2s ≥−n2ℓ+(∑i∈Cni)2−2⋅nℓ⋅(∑i∈Cni)+n2ℓ −2⋅ns⋅(∑i∈Cni) ≥−2⋅nℓ⋅(∑i∈Cni) ns ≤nℓ

This means that the small player has higher error whenever it has fewer samples than the large player- and strictly higher error whenever it has strictly fewer samples.
Property 2:
For the seconnd property, we wish to show that the worst case ratio of errors occurs in the two-player case. That is,

 errus(C)errul(C)≤errus({ns,nℓ})errul({ns,nℓ)}

In order to prove this, we’ll show something stronger. Take any player (with ). Then, we will show that that the derivative of the ratio with respect to the size of player () is always negative:

 ∂∂nkerrus(C)errul(C)<0

For uniform federation, we know that

 limnk→0errus(C)=errus(C∖{nk})

and similarly for the large player. Then, we can convert into by sending the size of every other player to 0: by the result we are trying to prove, this will only ever increase the ratio of their errors.

Next, we will start the proof.

 ∂∂nkerrus(C)errul(C)=errus(C)′⋅errul(C)−errus(C)⋅errul(C)′(errul(C))2

This is negative whenever:

 errus(C)′⋅errul(C)
 errus(C)′errus(C)

To show this result, we will calculate the lefthand side (ratio relating to the small player’s error) and then show that it is less than the equivalent ratio for the large player. The error of the small player can be written:

 μeN+σ2∑i∈Cn2i+N2−2N⋅nsN2

The derivative with respect to is:

 −μeN2+2⋅σ2(nk+N−ns)⋅N−(∑i∈Cn2i+N2−2N⋅ns)N3
 =−μe⋅N+2⋅σ2⋅((nk+N−ns)⋅N−(∑i∈Cn2i+N2−2N⋅ns))N3

The term we’re interested in is the ratio of the derivative of the small player’s error (which we just calculated) to the error of the small player. Note that the error itself can be written as:

 μe⋅N+σ2(∑i∈Cn2i+N2−2N⋅ns)N2

So the ratio of the derivative to the overall error is:

 1N⋅−μe⋅N+2⋅σ2⋅((nk+N−ns)⋅N−(∑i∈Cn2i+N2−2N⋅ns))μe⋅N+σ2(∑i∈Cn2i+N2−2N⋅ns)

What we would like to show is that the above term is less than the analogous term for the variant, which can be symmetrically written as:

 1N⋅−μe⋅N+2⋅σ2⋅((nk+N−nℓ)⋅N−(∑i∈Cn2i+N2−2N⋅nℓ))μe⋅N+σ2(∑i∈Cn2i+N2−2N⋅nℓ)

This is equivalent to proving:

 B+2σ2⋅N⋅nsA−2⋅σ2⋅N⋅ns

for

 A=μe⋅N+σ2⋅(∑i∈Cn2i+N2)B=−μe⋅N+2σ2⋅((nk+N)⋅N−(∑i∈Cn2i+N2))

We can cross multiply the inequality to get:

 (B+2σ2⋅N⋅ns)⋅(A−2⋅σ2⋅N⋅nℓ)<(B+2σ2⋅N⋅nℓ)⋅(A−2⋅σ2⋅N⋅ns)
 0<(B+2σ2⋅N⋅nℓ)⋅(A−2⋅σ2⋅N⋅ns)−(B+2σ2⋅N⋅ns)⋅(A−2⋅σ2⋅N⋅nℓ)

Which simplifies to:

 0<2⋅(A+B)⋅(nℓ−ns)⋅N⋅σ2

Because we have assumed that , this is true if . We can evaluate as being:

 = μe⋅N+σ2⋅(∑i∈Cn2i+N2)−μe⋅N+2σ2⋅((nk+N)⋅N−(∑i∈Cn2i+N2)) = 2σ2⋅(nk+N)⋅N)−σ2⋅(∑i∈Cn2i−N2) = 2σ2⋅nk⋅N+2σ2⋅N2−σ2⋅(∑i∈Cn2i−N2) = 2σ2⋅nk⋅N+σ2⋅(N2−∑i∈Cn2i) > 0

as desired.

The remaining properties all relate to the ratio of errors for the two-player group . For uniform federation, this ratio can be written as:

 errus({ns,nℓ})errul({ns,nℓ)}=μe⋅(ns+nℓ)+2σ2⋅n2ℓμe⋅(ns+nℓ)+2σ2⋅n2s

Property 3:

This property relates to the derivative of the ratio with respect to the size of the large player ():

 ∂∂nℓμe⋅(ns+nℓ)+2σ2⋅n2ℓμe⋅(ns+nℓ)+2σ2⋅n2s

This derivative is given by:

 (μe+4⋅σ2⋅nℓ)⋅(μe⋅(ns+nℓ)+2σ2⋅n2s)−(μe⋅(ns+nℓ)+2σ2⋅n2ℓ)⋅μe(μe⋅(ns+nℓ)+2σ2⋅n2s)2

The derivative is positive whenever:

 (μe+4⋅σ2⋅nℓ)⋅(μe⋅(ns+nℓ)+2σ2⋅n2s)−(μe⋅(ns+nℓ)+2σ2⋅n2ℓ)⋅μe>0

Pulling over terms and expanding gives:

 μ2e⋅(ns+nℓ)+2⋅μe⋅σ2⋅n2s+4⋅σ2⋅nℓ⋅μe⋅(ns+nℓ)+8⋅σ4⋅nℓ⋅n2s>
 μ2e⋅(ns+nℓ)+2σ2⋅μe⋅n2ℓ

Which simplifies to:

 2⋅μe⋅σ2⋅n2s+2⋅σ2⋅nℓ⋅μe⋅(2ns+nℓ)+8⋅σ4⋅nℓ⋅n2s>0

Property 4:
This property relates to the derivative of the ratio with respect to the size of the small player ():

 ∂∂nsμe⋅(ns+nℓ)+2σ2⋅n2ℓμe⋅(ns+nℓ)+2σ2⋅n2s

This derivative is given by:

 μe⋅(μe⋅(ns+nℓ)+2σ2⋅n2s)−(μe⋅(ns+nℓ)+2σ2⋅n2ℓ)⋅(μe+4⋅σ2⋅ns)(μe⋅(ns+nℓ)+2σ2⋅n2s)2

The derivative is negative whenever:

 μe⋅(μe⋅(ns+nℓ)+2σ2⋅n2s)<(μe⋅(ns+nℓ)+2σ2⋅n2ℓ)⋅(μe+4⋅σ2⋅ns)

Expanding:

 μe⋅(μe⋅(ns+nℓ)+2σ2⋅n2s)<μ2e⋅(ns+nℓ)+2σ2⋅n2ℓ⋅μe+4⋅σ2⋅ns⋅μe⋅(ns+nℓ)+2σ2⋅μe⋅n2ℓ

Simplifying:

 0<2σ2⋅n2ℓ⋅μe+2⋅σ2⋅ns⋅μe⋅(ns+2nℓ)+2σ2⋅μe⋅n2ℓ

which is satisfied.
Property 5:
This can be found by rewriting and applying the limit.

 limnsnℓ→0errs({ns,nℓ)}errl({ns,nℓ)}= limnsnℓ→0μe⋅(ns+nℓ)+2σ2⋅n2ℓμe⋅(ns+nℓ)+2σ2⋅n2s = limnsnℓ→0μe⋅nsnℓ+μe+2σ2⋅nℓμe⋅nsnℓ+μe+2σ2⋅ns⋅nsnℓ = μe+2σ2⋅nℓμe = μenℓ+2σ2μenℓ

Lemma 5.

Consider a set of federating players, using optimal fine-grained federation. The error player (with samples) experiences can be given by:

 μenjVj⋅T⋅(1+σ2(T−1Vj))

with and .

Proof.

Lemma 4.2 in [3] gives the expected error as:

 μeM∑i=1v2ji⋅1ni+⎛⎜⎝∑i≠jv2ji+⎛⎝∑i≠jvji⎞⎠2⎞⎟⎠⋅σ2

Fine-grained federation requires that , so , or . We can then write the error out as:

 μeM∑i=1v2jini+⎛⎝