MMZDA: Enabling Social Welfare Maximization in Cross-Silo Federated Learning

02/16/2022
by   Jianan Chen, et al.
Indiana University
0

As one of the typical settings of Federated Learning (FL), cross-silo FL allows organizations to jointly train an optimal Machine Learning (ML) model. In this case, some organizations may try to obtain the global model without contributing their local training, lowering the social welfare. In this paper, we model the interactions among organizations in cross-silo FL as a public goods game for the first time and theoretically prove that there exists a social dilemma where the maximum social welfare is not achieved in Nash equilibrium. To overcome this social dilemma, we employ the Multi-player Multi-action Zero-Determinant (MMZD) strategy to maximize the social welfare. With the help of the MMZD, an individual organization can unilaterally control the social welfare without extra cost. Since the MMZD strategy can be adopted by all organizations, we further study the scenario where multiple organizations jointly adopt the MMZD strategy and form an MMZD Alliance (MMZDA). We prove theoretically that the MMZDA strategy strengthens the control of the maximum social welfare. Experimental results validate that the MMZD strategy is effective in maximizing the social welfare and the MMZDA can achieve a larger maximum value.

READ FULL TEXT VIEW PDF

Authors

page 1

page 9

page 10

02/18/2022

Social Welfare Maximization in Cross-Silo Federated Learning

As one of the typical settings of Federated Learning (FL), cross-silo FL...
06/22/2021

Enabling Long-Term Cooperation in Cross-Silo Federated Learning: A Repeated Game Perspective

Cross-silo federated learning (FL) is a distributed learning approach wh...
12/13/2019

Toward an Automated Auction Framework for Wireless Federated Learning Services Market

In traditional machine learning, the central server first collects the d...
03/23/2020

Challenges of Bridging the Gap between Mass People and Welfare Organizations in Bangladesh

Computing for the development of marginalized communities is a big deal ...
05/27/2022

Workload-Aware Game-Theoretic Framework for Wireless Federated Learning

Federated learning (FL) has been proposed as a popular learning framewor...
02/26/2021

Discord and Harmony in Networks

Consider a coordination game played on a network, where agents prefer ta...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

I Introduction

In Federated Learning (FL), clients cooperatively train a Machine Learning (ML) model with their decentralized datasets under the coordination of a central server[mcmahan2017communication]. One of the typical settings of FL is cross-silo FL[kairouz2019advances] where a neutral third-party agent acts as the central server and clients are a group of organizations, aiming to jointly train an optimal ML model for their respective use. In this case, these organizations are also the owners of the global model and can utilize the well-trained global model to further process tasks for their own interests.

An optimal global model with high performance requires the organizations in cross-silo FL to collaborate efficiently so as to bring considerable benefits to all participants, which can be regarded as the maximization of the social welfare. In fact, there are many studies on optimizing the social welfare in cross-silo FL by directly improving the model performance[huang2021personalized, wang2021efficient, nandury2021cross, majeed2020cross], increasing the convergence speed[marfoq2020throughput], reducing the communication cost[zhang2020batchcrypt], protecting privacy[heikkila2020differentially, li2021practical, long2021federated] and security[jiang2021flashe], etc.

However, since every organization in cross-silo FL can obtain the final global model regardless of its contribution, the well-trained model becomes a public good, which is non-excludable and non-rivalrous for all organizations[tang2021incentive]. This leads to selfish behaviors that some organizations may only consider their own interests via inactively participating in local training to obtain the final global model for free or at a lower cost. The spread of this behavior can result in huge loss of the social welfare, and then none of the organizations can get the optimal model, which compromises the long-term stability and sustainability of cross-silo FL.

Most of the existing studies motivate organizations to fully contribute to cross-silo FL by designing an incentive mechanism[tang2021incentive, tang2018multi, zhao2018dynamic, feng2019joint, shao2019multimedia]. However, this requires extra negotiation costs since organizations need to reach a consensus on the mechanism in advance, and demands additional running costs where a distributed algorithm runs over all organizations, clearly adding more burden to organizations. Recently, game-theoretical approach is applied to investigate participation behaviors in cross-silo FL[zhang2022enabling], which needs additional information transmission between the server and organizations, and thus may cause potential privacy issues.

In this paper, we take a brand-new approach using the Multi-player Multi-action Zero-Determinant (MMZD) strategy[7381622] to maximize the social welfare in cross-silo FL without causing additional costs and transmission for organizations. Moreover, the MMZD Alliance (MMZDA) formed by multiple MMZD players is able to control the maximum social welfare more effectively. Another outstanding advantage of our methods is that they can control the social welfare and be applied to any cross-silo FL scenario no matter what strategies or actions other organizations perform. In summary, our contributions include (a preliminary version of this paper is presented in ICASSP 2022[chen2022social]):

  • We model the interactions among organizations in cross-silo FL as a public goods game for the first time, focusing on the organization’s strategy rather than designing an extra mechanism to solve the social welfare maximization problem.

  • We reveal the existence of the social dilemma in cross-silo FL by mathematical proof for the first time, which demonstrates the adverse effect of selfish behaviors in cross-silo FL in the view of game theory. This can be used as a theoretical basis for exploring organizations’ behaviors in cross-silo FL.

  • We overcome the social dilemma by employing the MMZD strategy from the perspectives of individual organization and alliance. Specifically, any organization can unilaterally maximize the social welfare, which ensures the social welfare in cross-silo FL at a certain level and maintains the stability of the system.

  • We further study the scenario in which multiple organizations adopt the same MMZD strategy, forming the MMZDA. We have mathematically proved that the maximum social welfare controlled by the MMZDA can get improved. This approach can also extend the application domains of the MMZD.

  • Experiments prove the effectiveness of the MMZD strategy in maximizing the social welfare. The maximum value of social welfare is able to be enlarged by the MMZDA strategy.

The rest of the paper is organized as follows. The related work about cross-silo FL is summarized in Section II. In Section III, we formulate the cross-silo FL game to model the interactions among organizations in cross-silo FL, and further discover the social dilemma in the game. We propose a method based on MMZD strategy for individual organization to control the social welfare in Section IV. Section V studies the MMZD strategy employed by multiple organizations, namely MMZDA, and validates that the MMZD alliance can enlarge the maximum value of social welfare compared to the individual MMZD. Simulation results are reported in Section VI, followed by the conclusion in Section VII.

Ii Related Work

Existing works related to cross-silo FL can be classified into three categories: the optimization of aggregation algorithm, security and privacy protection, and the incentive mechanism design.

Aggregation algorithms are developed to enhance the performance of cross-silo FL. Based on the original FedAvg algorithm[mcmahan2017communication], various algorithms to improve the convergence speed, accuracy, and security were proposed in cross-silo FL setting. Marfoq et al. introduced practical algorithms to design an averaging policy under a decentralized model for achieving the fastest convergence[marfoq2020throughput]. And Huang et al. proposed FedAMP to overcome non-iid challenges[huang2021personalized]. Zhang et al. reduced the encryption and communication overhead caused by additively homomorphic encryption, which lowered the cost of aggregation as well[zhang2020batchcrypt].

Security and privacy protection issues also received attention in cross-silo FL. Heikkil et al. combined additively homomorphic secure summation protocols with differential privacy to guarantee strict privacy for individual data subjects in the cross-silo FL setting[heikkila2020differentially]. Li et al. proposed a brand-new one-shot algorithm that can flexibly achieve differential privacy guarantees[li2020practical]. Jiang et al. designed FLASHE, an optimized homomorphic encryption scheme to meet requirements of semantic security and additive homomorphism in cross-silo FL[jiang2021flashe]

. Chu et al. proposed a federated estimation method to accurately estimate the fairness of a model, namely avoiding the bias of data, without infringing the data privacy of any party

[chu2021fedfair].

Many studies used incentive mechanisms to encourage organizations to participate in training since the performance of cross-silo FL is affected by organizations’ behaviors. Tang et al. formulated a social welfare maximization problem and proposed an incentive mechanism for cross-silo FL to address the organization heterogeneity and the public goods characteristics[tang2021incentive]. Many incentive mechanisms based on auction[tang2018multi], contract[zhao2018dynamic, feng2019joint], and pricing[shao2019multimedia] were proposed for FL. Unfortunately, they cannot be adapted to the cross-silo FL. First of all, most of the existing incentive mechanisms focus on encouraging more clients to participate in FL, instead of improving the performance of the global model from the perspective of social welfare. Secondly, the global model in cross-silo FL has the non-exclusive nature of public goods, leading to potential free-riding behaviors, which is not considered in the existing incentive mechanisms. Last but not least, current incentive mechanisms usually require participants or servers to spend additional computing resources. As a result, complex designs and additional computing costs can bring a burden on the organizations.

Recently, Zhang et al. explored the long-term participation of organizations in cross-silo FL by deploying a game theory approach[zhang2022enabling], where a strategy and an algorithm were displayed for organizations to reduce free riders and increase the amount of local data for global training. However, the proposed algorithm requires extra information interactions between the organizations and server, which leads to the risk of information leakage.

In light of the above analysis, the following aspects distinguish our work from the existing approaches. First, our study reveals the social dilemma in the cross-silo FL. Second, we are committed to improving the final performance of the FL model, namely the social welfare. Moreover, we adopt the MMZD strategy without additional cost to control the maximum value of social welfare, and we use the MMZD alliance to further expand the ability to control the maximum value of social welfare.

Iii System Model

We consider a cross-silo FL scenario with a set of organizations, denoted as . All organizations rely on a central server to collaboratively conduct global model training for a specific task, where each of them has their own data for local training. The goal of organizations is to obtain an optimal global model, minimizing the loss based on all datasets. The central server collects the results of local model updates from all organizations, aggregates to obtain the global model, and then distributes it to everyone for the next round of training.

In each round of local training, every organization performs iterations of model training. We denote the number of global communication rounds for aggregation as . For the current task, the action of organization , denoted as , represents the number of communication rounds it participates in the task. Then,

denotes the action vector of all organizations. Here we assume that all organizations in this cross-silo FL may participate in fewer global aggregations due to laziness or selfishness, but they do not carry out malicious attacks, such as model poisoning attack.

According to the cross-silo FL model, all organizations get the same model in return. Inspired by [tang2021incentive], we define the revenue of organization as:

(1)

where (in dollars per unit of precision function) denotes the unit revenue of organization by using the returned final model, denotes the precision of the untrained model, and denotes the precision of the trained global model after the actions of organizations in the action vector . Specifically, can be modeled as

(2)

with positive coefficients and [li2019convergence]

being derived based on the loss function, neural network, and local datasets. In particular, we have

. The revenue of each organization is proportional to the difference between the expected loss after rounds aggregation (i.e., ) and the minimum expected loss (i.e., )[tang2021incentive]. As the number of total participation rounds increases, the marginal decrease of the difference reduces.

We define the cost of organization as:

(3)

The cost is composed of the organization’s computation cost and its communication cost . On the one hand, the computation cost , where is a positive parameter, denoting the computation cost of each iteration in organization ’s local training111As [tran2019federated] shows, , where is the effective capacitance coefficient of organization ’s computing chipset, denotes the calculation processing capacity, denotes the number of data units, and denotes the number of CPU cycles required by organization to process one data unit.. On the other hand, the communication cost is defined as a parameter since we assume that every organization uploads its updates in each global aggregation round. If it chooses not to participate in global aggregation, it will submit a zero vector as updates.

Then the utility of organization is defined as the difference between its revenue and cost:

(4)

According to previous statements, we model the interactions among organizations as a cross-silo FL game.

Definition III.1.

(Cross-silo FL game). In the cross-silo FL game, organizations participating in cross-silo FL act as players, where their actions and utilities are and , respectively.

The cross-silo FL game can be iterative since these organizations in cross-silo FL usually cooperate for a long time to finish multiple FL tasks. Each game round in the cross-silo FL game corresponds to a certain FL task. Moreover, the social welfare in cross-silo FL game can be denoted as the total utility of all organizations , namely . In the cross-silo FL game, we find that the social dilemma occurs if , which can be summarized as below.

Theorem 1.

(Social dilemma). If , there exists a social dilemma in the cross-silo FL game.

Proof.

The social dilemma forms when the Nash equilibrium is not the point of maximum social welfare. First, we study the Nash equilibrium of the cross-silo FL game. Referring to (4), we can derive the derivative of as . Given , we have

which leads to

Thus, the derivative of is negative, and the utility function decreases monotonically with . The Nash equilibrium strategy of each organization is , so the Nash equilibrium point is . Noted that, there is a natural and necessary premise of FL that the utility of the well-trained model must be positive. Thus, we prove that the point with the social welfare

higher than that in the Nash equilibrium point

So the social dilemma exists if . ∎

The condition in the above theorem indicates that if any organization only trains the local model using its local dataset, the utility is negative. In fact, this condition strengthens organizations’ motivation to participate in global training in cross-silo FL game.

Thus, there is a social dilemma in the cross-silo FL game and problem follows. If the organization only pursues its own interests and does not participate in the communication round, it will lead to a decrease in the accuracy of the global model, which in turn leads to low social welfare. So it is of great importance to optimize the public goods in the cross-silo FL game, i.e., the accuracy of the global model, which is actually equivalent to the social welfare.

For reference, we summarize key notations used in the system model in Table I.

Notation Meaning
The number of organizations
The number of local training iterations for every organization
The number of global communication rounds for aggregation
The number of communication rounds organization
participates in the current task
The action vector of all organizations
The revenue of organization
The unit revenue of organization by using the final model
The precision of the trained global model with the
corresponding action vector
The cost of organization
The computation cost of organization
The communication cost of organization
The computation cost of each iteration in organization ’s
local training
The utility of organization with the corresponding
action vector
TABLE I: Key Notations.

Iv Social welfare maximization by MMZD

According to the analysis above, we can see that the underlying cause of the social dilemma is selfishness, leading to the loss of all organizations, namely the low social welfare. Aiming to solve this problem, we resort to the Multi-player Multi-action Zero-Determinant (MMZD) strategy for the social welfare maximization in this section. In each game round, any organization can choose the action , so there are possible outcomes for each game round. We assume that the organizations have one-round memory since a long-memory player has no priority against others with short memory[7381622]. Fig. 1 describes an example of the cross-silo FL game with two organizations and three actions, i.e., and , in which all possible outcomes can be denoted as . For arbitrary organization , its mixed strategy is defined as:

(5)

where

represents the probability of organization

choosing action in the current game round and other organizations choosing the same actions as -th outcome of the previous game round. As presented in Fig. 1, for the previous outcome , the conditional probability of organization 1 to select action is and the conditional probability of organization 2 to select action is . In addition, the corresponding utility vector is denoted as:

(6)

where each utility of organization choosing action in the -th outcome can be calculated by , with denoting the action vector corresponding to the -th outcome but . Fig. 2(a) presents the strategy vectors and utility vectors based on the example shown in Fig. 1.

Fig. 1: Illustration of the cross-silo FL game with two organizations and three actions ( and ).

In the cross-silo FL model, an organization’s current move depends only on its last action and the action vector

in the last game round. We can construct a Markov matrix

, with each element denoting the one-step transition probability from state to . Fig. 2(b) shows the Markov matrix for the case of and . For example, the element is at the 3rd row and 4th column in Fig. 2, which represents the possibility of transitioning from the 3rd-outcome (0,2) in the previous round to the 4th-outcome (1,0) in the current round. Then, we define where I

is an identity matrix. And we assume the stationary vector of

M is . Given , we can draw that . According to Cramer’s rule, we have

(7)

where denotes the adjugate matrix of . Thus, it can be noted that every row of is proportional to . Hence for any vector , we can draw according to [press2012iterated].

Fig. 2: The strategy vectors, utility vectors, Markov matrix, and determinant of after elementary transformations in the cross-silo FL game example with two organizations and three actions shown in Fig. 1. (a) The strategy vectors and the corresponding utility vectors of the cross-silo FL game. (b) The Markov matrix of the cross-silo FL game. (c) After several elementary column operations on , the dot product of the stationary vector and an arbitrary vector is equal to , where the first columns is only controlled by organization 1.

In particular, when , then organization ’s expected utility in the stationary state is:

(8)

which makes a linear combination of all organizations’ expected utilities yielding the following equation:

(9)

In the above equation, and are constants for the linear combination. Moreover, after some elementary column operations on , in which there can be a certain column controlled by a certain organization. Fig. 2(c) displays the determinant of based on the example in Fig. 1, in which the first column is solely determined by organization 1. Thus, when organization chooses a strategy that satisfies

(10)

where is a non-zero constant and is under the control of organization , the corresponding column of and the last column of will be proportional. And (9) can be converted to:

(11)

We further study the social welfare maximization problem with the help of the MMZD in this circumstance. Take organization performing the MMZD strategy as an example. According to (11), by setting , the social welfare can be calculated as . Thus, the issue of maximizing the social welfare is equivalent to the following optimization problem:

We denote as the th element in , then we can solve the above optimization problem by considering the following two cases:

Iv-a

To meet the constraint , we can get the lower bound of as follows:

To meet the constraint , we can get the upper bound of as follows:

Only if , can have a feasible solution, which is equivalent to . If there exists satisfying the above constraint, we can obtain the minimum value of as follow:

(12)

Iv-B

Similarly, when , we have ; considering , we have . In addition, is feasible only when , i.e., . Finally, we can get the following result:

(13)

In summary, by (12) and (13), organization can unilaterally set the expected social welfare with the MMZD strategy meeting , with each element of calculated by:

where denotes the -th element in .

V Social welfare maximization by MMZD alliance

In the previous section, we proved that a single organization is able to maximize the social welfare within a range. According to (10), every organization can deploy the MMZD strategy to control the social welfare. Therefore, it’s possible that multiple organizations utilize the MMZD strategy at the same time. So in this section, we explore multiple organizations that play the MMZD strategy to form an alliance in the cross-silo FL game. We call them MMZD Alliance (MMZDA) organizations (denoted as ), assuming that all MMZDA organizations use the same MMZD strategy to prevent the low social welfare. Besides, we define the other organizations as outsider organizations (denoted as ). Outsider organizations may inactively participate in communication rounds and just want to get model trained by other organizations for free.

The current challenge is how MMZDA controls the maximum value of social welfare. Specifically, will the MMZDA have stronger control and enlarge the maximum social welfare compared with the individual MMZD strategy?

For the sake of convenience, we assume that organization is an alliance, which performs the same MMZD strategy as the leader organization . As for an outsider organization , it may not participate in any communication round.

In this new scenario based on the MMZDA, we pay more attention to strategies and behaviors rather than the organizations themselves. Since the MMZDA members take the same actions, we treat them as an entity represented by the leader organization . In each game round, any outsider organization or alliance leader (organization ) can choose the action , so there are possible outcomes for each game round. We assume that the organizations have one-round-memory. And we define as the players of the cross-silo FL game based on MMZDA. For arbitrary organization , its mixed strategy is defined as:

(14)

where represents the probability of organization choosing action in the current game round conditioning on the -th outcome of the previous game round. In addition, the corresponding utility vector is denoted as:

(15)

where each utility of organization choosing action in the -th outcome can be calculated by , with denoting the action vector corresponding to the -th outcome but . Moreover, if , then . In Section IV, we perform that the linear combination of all organizations’ expected utilities can be represented as (8) and (9). Similarly, we can draw that

(16)

where and are constants. Thus, when organization chooses a strategy that satisfies , where is a non-zero constant and is under the control of organization , the column related to and the last column of will be proportional. Then (9) can be converted to:

(17)

In order to investigate the social welfare maximization problem by MMZDA, we rewrite the optimization problem as:

Similar to Section IV, we denote as the th element in , then we can solve this optimization problem by discussing these two situations:
(1)
When , we have ; given , we have . In addition, is feasible only when , i.e., . Finally, we can get the following result:

(18)

(2)
Given , we have ; while , we can get . In addition, is feasible only when , i.e., . Finally, we can get the following result:

(19)

In summary, by (18) and (19), the alliance can unilaterally set the expected social welfare with the MMZD strategy meeting , with each element of calculated by:

where denotes the -th element in .

Theorem 2.

The social welfare can achieve larger maximum value by MMZDA than that by single MMZD organization.

Proof.

In the same cross-silo FL game, we can achieve a maximum social welfare using MMZD strategy by an individual organization. Meanwhile, we are able to draw another maximum social welfare by MMZDA.

V-a

In this case, we have:

The value of is the maximum value in the certain values, and we denote this candidate values as set . While the value of generating from the values, we denote this candidate values as set . Note that the values cover all possible outcomes, but the values do not cover all outcomes, since the MMZDA organizations employ the same strategy. Given a certain element like , as , we have:

(20)

where . Otherwise, given an element like , we can draw

As deduced above, the elements in are all in , so is a subset of . Thus, holds true.

V-B

When , the situation is quite similar.