Optimality and Stability in Federated Learning: A Game-theoretic Approach

06/17/2021
by   Kate Donahue, et al.
7

Federated learning is a distributed learning paradigm where multiple agents, each only with access to local data, jointly learn a global model. There has recently been an explosion of research aiming not only to improve the accuracy rates of federated learning, but also provide certain guarantees around social good properties such as total error. One branch of this research has taken a game-theoretic approach, and in particular, prior work has viewed federated learning as a hedonic game, where error-minimizing players arrange themselves into federating coalitions. This past work proves the existence of stable coalition partitions, but leaves open a wide range of questions, including how far from optimal these stable solutions are. In this work, we motivate and define a notion of optimality given by the average error rates among federating agents (players). First, we provide and prove the correctness of an efficient algorithm to calculate an optimal (error minimizing) arrangement of players. Next, we analyze the relationship between the stability and optimality of an arrangement. First, we show that for some regions of parameter space, all stable arrangements are optimal (Price of Anarchy equal to 1). However, we show this is not true for all settings: there exist examples of stable arrangements with higher cost than optimal (Price of Anarchy greater than 1). Finally, we give the first constant-factor bound on the performance gap between stability and optimality, proving that the total error of the worst stable solution can be no higher than 9 times the total error of an optimal solution (Price of Anarchy bound of 9).

READ FULL TEXT

page 1

page 2

page 3

page 4

01/24/2021

Incentive Mechanism Design for Federated Learning: Hedonic Game Approach

Incentive mechanism design is crucial for enabling federated learning. W...
10/02/2020

Model-sharing Games: Analyzing Federated Learning Under Voluntary Participation

Federated learning is a setting where agents, each with access to their ...
12/01/2021

Models of fairness in federated learning

In many real-world situations, data is distributed across multiple locat...
03/04/2021

One for One, or All for All: Equilibria and Optimality of Collaboration in Federated Learning

In recent years, federated learning has been embraced as an approach for...
08/27/2020

Forming better stable solutions in Group Formation Games inspired by Internet Exchange Points (IXPs)

We study a coordination game motivated by the formation of Internet Exch...
02/25/2020

Three Approaches for Personalization with Applications to Federated Learning

The standard objective in machine learning is to train a single model fo...
06/29/2021

Achieving Statistical Optimality of Federated Learning: Beyond Stationary Points

Federated Learning (FL) is a promising framework that has great potentia...