# Incentive Mechanism Design for Federated Learning: Hedonic Game Approach

Incentive mechanism design is crucial for enabling federated learning. We deal with clustering problem of agents contributing to federated learning setting. Assuming agents behave selfishly, we model their interaction as a stable coalition partition problem using hedonic games where agents and clusters are the players and coalitions, respectively. We address the following question: is there any utility allocation method ensuring a Nash-stable coalition partition? We propose the Nash-stable set and analyze the conditions of non-emptiness. Besides, we deal with the decentralized coalition partition. We formulate the problem as a non-cooperative game and prove the existence of a potential.

11/06/2019

### Federated Learning for Edge Networks: Resource Optimization and Incentive Mechanism

Recent years have witnessed a rapid proliferation of smart Internet of T...
06/17/2021

### Optimality and Stability in Federated Learning: A Game-theoretic Approach

Federated learning is a distributed learning paradigm where multiple age...
10/02/2020

### Model-sharing Games: Analyzing Federated Learning Under Voluntary Participation

Federated learning is a setting where agents, each with access to their ...
03/04/2021

### One for One, or All for All: Equilibria and Optimality of Collaboration in Federated Learning

In recent years, federated learning has been embraced as an approach for...
11/03/2020

### Hedonic Expertise Games

We consider a team formation setting where agents have varying levels of...
12/19/2020

### FedServing: A Federated Prediction Serving Framework Based on Incentive Mechanism

Data holders, such as mobile apps, hospitals and banks, are capable of t...
02/22/2020

### FMore: An Incentive Scheme of Multi-dimensional Auction for Federated Learning in MEC

Promising federated learning coupled with Mobile Edge Computing (MEC) is...

## I Introduction

Data protection is a major concern. If we do not trust someone withholding our data, we may opt for federated learning by privately developing intelligent systems to create privacy-preserving AI. Federated learning

enables privacy-preserving machine learning in a decentralized way

[bib:Li]. It is used in situations where data is distributed among different agents and training is impossible due to the difficulty to collect data centrally. All data is kept on device while a shared (global) learning model is trained in each device and aggregated (combined) centrally. Formally, we consider the following setting: i) data owner agents which locally trains the shared learning model, and ii) model aggregating entity (MAE) which combines learning model of its own with the agents. MAE and agents contribute to the same shared learning model. Federated learning has been identified as a distributed machine learning framework which sees rapid advances and broad adoption in next generation networking and edge systems [bib:MehdiBennis, bib:Li, bib:OpenProblems, bib:CongShen, bib:WalidSaad1, bib:WalidSaad2, bib:MungChiang, bib:YuanmingShi]

. Obviously, the motivation to implement federated learning is to reduce the variance in a learned model by accessing more data.

A very crucial question is how would MAE motivate the agents to participate in federated learning. Designing the mechanism of agents’ incentives can be performed by utilizing various frameworks such game theory, auction theory, etc

[bib:WalidSaad1]. Any clustering among agents (players) being able to make strategic decisions becomes a coalition formation game when the players –for various individual reasons– may wish to belong to a relative small coalition rather than the grand coalition–the set of all players. Players’ moves from one to another coalition are governed by a set of rules. Basically, an agent (player) will move to a new coalition when it may obtain a better utility from this coalition. We shall not consider any permission requirements, which means that a player is always accepted by a coalition to which the player is willing to join. Based on those rules, the crucial question in the game context is how a stable partition exists. This is essential to enable federated learning.

In the following, we study the hedonic coalition formation games and the stability notion that we analyze is the Nash stability [bib:hajdukova]. A coalition formation game is called hedonic if each player’s preferences over partitions of players depend only on the members of his/her coalition. Finding a stable coalition partition is the main question in a coalition formation game. We refer to [bib:Existenceofstabilityinhedoniccoalitionformationgames] discussing the stability concepts associated to hedonic conditions. In the sequel, we concentrate on the Nash stability. The definition of the Nash stability is quite simple: a partition of players is Nash stable whenever no player deviates from its coalition to another coalition in the partition.

In this work, the following problem is studied: having coalitions associated with their utilities, we seek the answer of how must the coalition utilities be allocated to the players in order to obtain a stable coalition partition. Clearly, the fundamental question is to determine which utility allocation methods may ensure a Nash-stable partition. We first propose the definition of the Nash-stable set which is the set of all possible utility allocation methods resulting in Nash-stable partitions. We show that additively separable and symmetric utility allocation always ensures Nash-stable partitions. Moreover, our work aims also at finding the partitions in a decentralized setting. We model the problem of finding a Nash-stable partition by formulating it as a non-cooperative game and show that such a game is a potential game.

## Ii Motivation and Problem Description

We consider a set of agents denoted by that can participate in the federated learning setting, and a model aggregating entity (MAE) which aggregates (combines) learning model of its own with the agents. MAE and agents contribute in the same global learning model. The parameters of learning model of MAE and agent are represented by and , respectively where every agent has data samples. We also consider that the communication link between MAE and agent

can be characterized by a probability of reliable transmission (e.g. low bit error rate) denoted by

. If a cluster, say , of agents are federated then the aggregated (combined) learning model is found by using an aggregation method which in generic case is denoted as where shows the aggregation function. On the other hand, in this paper, we consider the following aggregation method:

 θS=∑i∈Sximiθi∑i∈Sximi,xi={1,reception successful,0,otherwise. (1)

in which if MAE receives successfully information of . This is nothing more than choosing agent with probability where we have and . Furthermore, we denote by the loss of learning model given by parameters . If two disjoint clusters and , i.e. , are federated, then we denote the new parameters as . It is reasonable to assume that the loss of is lower than average loss of and :

 L(θS∪T|x)≤L(θS|x)+L(θT|x)2. (2)

From the perspective of agents, we assume that the model aggregating entity can assign a utility to all possible clusters. Note that from the MAE point-view, this is the cost that must be paid to the cluster; however, we shall formulate the problem using the utility term which is the monetary cost charged to MAE.

Then, we come up with the question how to design the incentives in order that the agents are willing to participate in federated learning setting taking into account their preferences. We represent by the utility which assigns a real value for every subset of , i.e. where is the collection of all possible non-empty subsets of and empty set , and we set . Any agent can join a cluster if guaranteed to be paid at least price which is given by

 πi=~f(piL(θi)) (3)

where is a monotonically increasing function and inversely proportional to meaning that the less loss the more gain. Besides, is the second parameter which has an impact on the price asked by agent. It corresponds to the fact that as has a poor value, the agent asks lower price to participate in the federation.

We define the utility received by agent being in cluster as following:

 utility of agent i=πi+ϕSi (4)

where is the gain of agent by joining to cluster , and we set for all . On the other hand, MAE determines the utility of any cluster :

 u(S)=f(|S|)raw utility+~f(1E[L(θS)])learning % utility (5)

where is a subadditive function, i.e. for all and

. The rational behind this choice is that the total cost of two disjoint clusters is more than the cost when they are merged. The expected value of loss function is given by

 E[L(θS)]=∑x∈XL(θS|x)P[x] (6) P[x]=∏i∈Npxii(1−pi)1−xi (7)

where with is the set of all possible combinations of vectors.

The utility function is superadditive if for all possible and in such that .

 ~f(1E[L(θS∪T)])−~f(1E[L(θS)])−~f(1E[L(θT)])≥f(|S|)+f(|T|)−f(|S∪T|),∀S,T∈2N,S∩T=∅ (8)
###### Remark II.1

Note that if superadditivity would hold, we would have all the agents in the same cluster. On the other hand, the theorem implies that superadditivity can rarely exist since it may be very difficult to find a loss function which satisfies the conditions.

Considering that the agents strategically decide to which cluster to join, we can define the problem of clustering as a coalition formation game. We then change the language of problem formulation using game theoretic terms, i.e.

agents players, cluster coalition

Players may not be always fully cooperative and behave selfishly. We then face the problem of finding coalition structures which are stable under selfishness. In the sequel, we deal with such a setting.

## Iii Hedonic Game

A hedonic game is given by a pair , where denotes the preference profile, specifying for each player his preference relation , i.e. a reflexive, complete and transitive binary relation.

###### Definition III.1

A coalition structure or a coalition partition is a set which partitions the players’ set such that . All coalitions in are disjoint coalitions, i.e., for all .

Given and , denotes the set such that . Moreover, is the set of all possible coalition partitions over . In its partition form, a coalition formation game is defined on the set by associating a utility value to each subset of any partition of . In its characteristic form, the utility value of a set is independent of the other coalitions, and therefore, . The games of this form are more restrictive but present interesting properties to reach an equilibrium. Practically speaking, this assumption means that the gain of a group is independent of the other players outside the group. Hedonic games fall into this category with an additional assumption:

###### Definition III.2

A coalition formation game is hedonic if

• the gain of any player depends solely on the members of the coalition to which the player belongs, and

• the coalitions arise as a result of the preferences of the players over their possible coalitions’ set.

### Iii-a Preference Relation

The preference relation of a player can be defined over a preference function. We consider the case where the preference relation is chosen to be the utility allocated to the player in a coalition. Thus, player prefers the coalition to iff,

 ϕSi≥ϕTi⇔S⪰iT. (9)

### Iii-B The Nash Stability

The stability concepts for a hedonic game are various. In the literature, a hedonic game is said individually stable, Nash stable, core stable, strict core stable, Pareto optimal, strong Nash stable, or, strict strong Nash stable. We refer to [bib:Existenceofstabilityinhedoniccoalitionformationgames] for a thorough definition of these different stability concepts. In this paper, we are only interested in the Nash stability because the players do not cooperate to take their decisions jointly.

###### Definition III.3 (Nash Stability)

A partition of players is Nash-stable whenever no player has incentive to unilaterally change his or her coalition to another coalition in the partition which can be mathematically formulated as: partition is said to be Nash-stable if no player can benefit from moving from his coalition to another existing coalition , i.e.:

 SΠNS(i)⪰iT∪i,∀T∈ΠNS∪∅;∀i∈N. (10)

which can be similarly defined over preference function as following:

 ϕSΠNS(i)i≥ϕT∪ii,∀T∈ΠNS∪∅;∀i∈N. (11)

Nash-stable partitions are immune to individual movements even when a player who wants to change does not need permission to join or leave an existing coalition [bogomonlaia].

###### Remark III.1

Stability concepts being immune to individual deviation are Nash stability, individual stability, contractual individual stability. Nash stability is the strongest within above. The notion of core stability has been used already in some models where immunity to coalition deviation is required [bib:hajdukova].

###### Remark III.2 (Impossibility of a stability concept)

In [bib:Barber], the authors propose some set of axioms which are non-emptiness, symmetry pareto optimality, self-consistency; and they analyze the existence of any stability concept that can satisfy these axioms. It is proven that for any game , there does not exist any solution which satisfies these axioms. In this work, we show how non-emptiness can be guaranteed using Nash stability as a solution concept.

### Iii-C Aggregated Learning Model Parameters

When a stable partition exists, then this means that all the players (agents) are agreed to participate to federation. As a result of this, MAE utilizes the following aggregation of learning model parameters:

 θf=wθMAE+(1−w)θN (12) θN=∑i∈Nximiθi∑i∈Nximi (13)

where is a weighting parameter showing how much MAE favors the aggregated learning model parameters of agents (players), shows the learning parameters of MAE’s local model. In summary, we have the following procedure:

while Nash-stable partition exists

1. Player (agent) sends information of , for all

2. MAE calculate aggregated learning model parameters

end

Given , the expected value of loss function in federation can be calculated as following:

 E[L(θf)] =∑x∈XL(θf|x)P[x] ≥L(E[θf])(Jensen's% inequality) (14)

where

 E[θf] =wθMAE+(1−w)E[θN] =wθMAE+(1−w)∑x∈X∑i∈Nximiθi∑i∈NximiP[x] (15)

## Iv The Nash-stable Set

As the utility associated with all possible coalitions are known, we are interested in finding a utility distribution to ensure Nash stability. We thus define a utility allocation method where as following:

 ϕ={ϕSi:∀i∈S,∀S∈2N} (16)

which directly sets up a preference profile. The set of all possible utility allocation methods is denoted by . Let us define the mapping . Clearly, for any utility allocation method , mapping finds corresponding partition, i.e. . We define the Nash-stable set which includes all those efficient allocation methods that build the following set:

 Nstable={ϕ∈Rκ:SM(ϕ)(i)⪰iT∪i,∀T∈M(ϕ)∪∅;∀i∈N}. (17)

Let us define the set of constraints stemming from the preference function in order to check if the Nash-stable set is non-empty. Due to the utility bound, for any utility allocation method , we have for all called as budged balanced utility allocation which further can be given by

 ∑i∈S~f(1L(θi))+∑i∈SϕSi≤f(|S|L(θS)),∀S∈2N. (18)

For simplicity, let us define marginal utility as following:

 Δθ(S)=⎧⎨⎩f(|S|L(θS))−∑i∈S~f(1L(θi)),∀S∈2N∖i,∀i∈N0,∀i∈N. (19)

which results in the following constraints:

 C1θ(ϕ):={∑i∈SϕSi≤Δθ(S),∀S∈2N}. (20)

On the other hand, for any , the constraints that ensure the Nash stability are given by

 C2θ(ϕ):={ϕSM(ϕ)(i)i≥ϕT∪ii,∀T∈M(ϕ)∪∅;∀i∈N}, (21)

Then, the Nash-stable set is non-empty, iif:

 Nstable(θ)={ϕ∈Rκ:C1θ(ϕ) and C2θ(ϕ)}, (22)

which allows us to conclude:

###### Theorem IV.1

The Nash-stable set can be non-empty.

###### Proof IV.1

We can check if the Nash-stable is non-empty by solving the following optimization problem:

 maxϕ∑S∈2N∑i∈SϕSi % subject to C1θ(ϕ) and C2θ(ϕ).

If there exists any feasible solution of this problem, then we conclude that there is at least one utility allocation method which provides a Nash-stable partition.

However, searching in an exhaustive manner over the whole partitions is NP-hard as the number of partitions grows according to the Bell number. Typically, with only players, the number of partitions is as large as .

If the utility function is superadditive, then it is trivial to check that the marginal utility is also superadditive: , for all possible and such that . Due to eq. (18), we have

 ∑i∈S∪TϕS∪Ti=∑i∈SϕS∪Ti+∑i∈TϕS∪Ti≤Δθ(S∪T) ∑i∈SϕSi+∑i∈TϕTi≤Δθ(S)+Δθ(T) ⇒∑i∈SϕS∪Ti+∑i∈TϕS∪Ti≥∑i∈SϕSi+∑i∈TϕTi.

This result means that any player is better off in a larger coalition which ultimately all players have the most gain in the grand coalition. This is obvious from eq. (21) where for every player , for all .

### Iv-B Additively Separable and Symmetric Utility

Preferences of a player are additively separable whenever the preference can be stated with a function characterizing how a player prefers another player in each coalition. This means that the player’s preference for a coalition is based on individual preferences. This can be formalized as follows:

###### Definition IV.1

The preferences of a player are said to be additively separable if there exists a function such that

 ∑j∈Svi(j)≥∑j∈Tvi(j)⇔S⪰iT,∀S,T∈2N. (23)

is normalized and set to . A profile of additively separable preferences satisfies symmetry if , for all . The meaning of is the mutual gain of player and when they are in the same coalition. Let be the all possible bipartite coalitions which can occur in coalition such that:

 V(S):={(i,j)∈S:j>i},|S|≥2.

We then define which shall serve as a utility allocation method to generate additively separable and symmetric preferences:

 v={v(i,j):∀(i,j)∈V(N)}

and mapping shall find corresponding partition, i.e. . The constraints that define the Nash-stable set are then defined over :

 C1θ(ϕ)→C1θ(v) and C2θ(ϕ)→C2θ(v)

Further, note that the utility that player gains in coalition is given by

 πi+ϕSi=~f(1L(θi))+∑j∈Sv(i,j) ⇒ϕSi=∑j∈Sv(i,j) (24)

On the other hand, due to the symmetry property of mutual gain, we have the following:

 ∑i,j∈Sv(i,j)=2∑(i,j)∈V(S)v(i,j).

For example, if , then .

###### Theorem IV.2

Additively separable and symmetric preferences always admit a Nash-stable partition. Therefore, constraints in are always satisfied [bib:hajdukova].

Based on this theorem, we only need to satisfy the constraints given by . Thus, we define the Nash-stable set which generates additively separable and symmetric preferences as following:

 (25)

Finding the values of in eq. (25) satisfying conditions can be done straightforward. However, we propose to formulate as an optimization problem for finding the values of

. A feasible solution of the following linear program guarantees the non-emptiness of

:

 maxv∑(i,j)∈V(N)v(i,j) subject to ∑(i,j)∈V(S)v(i,j)≤Δθ(S)2,∀S∈2N, (26)

where note that any feasible solution is upper bounded by . Furthermore, the coalition partition that stems from is given by which is Nash-stable.

## V Decentralized Coalition Partition

In this section, we study finding a Nash-stable partition in a decentralized setting. We, in fact, model the problem of finding a Nash-stable partition as a non-cooperative game.

A hedonic coalition formation game is equivalent to a non-cooperative game. Denote as the set of strategies. We assume that the number of strategies is equal to the number of players, i.e. . This is sufficient to represent all possible choices. Indeed, the players that select the same strategy are interpreted as a coalition. For example, if every player chooses different strategies, then this corresponds to the coalition partition comprised of singletons.

Consider the best-reply dynamics where in a particular step , only one player chooses its best strategy. A strategy tuple in step is denoted as , where is the strategy of player in step . In every step, only one dimension is changed in . We further denote as the partition in step . Define as the set of players that share the same strategy with player . Thus, note that for each step. The utility of player is and verifies the following equivalence:

 ϕi(σl)≥ϕi(σl−1)⇔S(l)σ(i)⪰iS(l−1)σ(i), (27)

where player is the one that takes its turn in step . Any sequence of strategy-tuple in which each strategy-tuple differs from the preceding one in only one coordinate is called a path, and a unique deviator in each step strictly increases the utility he receives is an improvement path. Obviously, any maximal improvement path which is an improvement path that can not be extended is terminated by stability.

### V-a Stability Analysis

The Nash stability is defined as following:

 σNSi∈argmaxσi∈Σiϕi(σi,σ−i), ∀i∈N. (28)

Moreover, Nash-stable coalition partition is given by collection:

 SσNS(j)={k∈N:σNSk=j},∀j∈Σ ΠNS={SσNS(j):∀j∈Σ}. (29)

In the sequel, we prove that the additively separable and symmetric utilities result in a potential game.

###### Theorem V.1

Any additively separable and symmetric utility results in a potential game with potential:

 Pv(σ)=∑S∈Π(σ)∑(i,j)∈V(S)v(i,j). (30)
###### Proof V.1

A non-cooperative game is a potential game whenever there exists a function such that:

 Pv(σi,σ−i)−Pv(σ′i,σ−i)=ϕ(σi,σ−i)−ϕ(σ′i,σ−i)

where . This means that when player switches from strategy to the difference of its utility can be given by the difference of a function . We choose the following potential function:

 Pv(σ)=∑S∈Π(σ)∑(i,j)∈V(S)v(i,j) (31)

Let us denote as and the coalitions when player switches from strategy to , respectively. Potential function is given as following before and after switching

 Pv(σi,σ−i)=∑(i,j)∈V(S)v(i,j)+∑(k,j)∈V(S′)v(k,j) +∑T∈Π(σ)∖{S,S′}∑(k,j)∈V(T)v(k,j)
 Pv(σ′i,σ−i)=∑(k,j)∈V(S∖i)v(k,j)+∑(k,j)∈V(S′∪i)v(k,j) +∑T∈Π(σ)∖{S,S′}∑(k,j)∈V(T)v(k,j)

where note that we have and after switching. Thus, we have

 Pv(σi,σ−i)−Pv(σ′i,σ−i)= ∑(i,j)∈V(S)v(i,j)+∑(k,j)∈V(S′)v(k,j) −∑(k,j)∈V(S∖i)v(k,j)−∑(k,j)∈V(S′∪i)v(k,j)= ∑j∈Sv(i,j)−∑j∈S′∪iv(i,j)

On the other hand, the utility shift before and after strategy switch is given by

 ϕ(σi,σ−i)−ϕ(σ′i,σ−i)=∑j∈Sv(i,j)−∑j∈S′∪iv(i,j)

which concludes the proof that .

In a potential game, a Nash equilibrium shall result in an optimum in potential . Therefore, corresponds to a coalition partition which is Nash-stable.

## Vi Conclusions

We analyzed stable clustering problem in federated learning setting. Clusters are made up of the agents contributing to federated learning. We considered that every agent gains a utility when switching from one cluster to another one. We modeled the decisions of agents in the framework hedonic games which is a widely used cooperative game model for this type of problems. A fundamental question in hedonic games is to analyze the conditions how stable coalition partitions can occur. We studied the existence of stable coalition partitions by introducing the Nash-stable set, and analyzed the existence of decentralized coalition partitions.