Agnostic Federated Learning

02/01/2019
by   Mehryar Mohri, et al.
16

A key learning scenario in large-scale applications is that of federated learning, where a centralized model is trained based on data originating from a large number of clients. We argue that, with the existing training and inference, federated models can be biased towards different clients. Instead, we propose a new framework of agnostic federated learning, where the centralized model is optimized for any target distribution formed by a mixture of the client distributions. We further show that this framework naturally yields a notion of fairness. We present data-dependent Rademacher complexity guarantees for learning with this objective, which guide the definition of an algorithm for agnostic federated learning. We also give a fast stochastic optimization algorithm for solving the corresponding optimization problem, for which we prove convergence bounds, assuming a convex loss function and hypothesis set. We further empirically demonstrate the benefits of our approach in several datasets. Beyond federated learning, our framework and algorithm can be of interest to other learning scenarios such as cloud computing, domain adaptation, drifting, and other contexts where the training and test distributions do not coincide.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
04/06/2021

Communication-Efficient Agnostic Federated Averaging

In distributed learning settings such as federated learning, the trainin...
research
10/10/2020

Fairness-aware Agnostic Federated Learning

Federated learning is an emerging framework that builds centralized mach...
research
02/25/2020

Three Approaches for Personalization with Applications to Federated Learning

The standard objective in machine learning is to train a single model fo...
research
10/14/2019

SCAFFOLD: Stochastic Controlled Averaging for On-Device Federated Learning

Federated learning is a key scenario in modern large-scale machine learn...
research
10/14/2022

A Primal-Dual Algorithm for Hybrid Federated Learning

Very few methods for hybrid federated learning, where clients only hold ...
research
12/17/2021

Federated Learning with Heterogeneous Data: A Superquantile Optimization Approach

We present a federated learning framework that is designed to robustly d...
research
02/05/2021

Federated Reconstruction: Partially Local Federated Learning

Personalization methods in federated learning aim to balance the benefit...

Please sign up or login with your details

Forgot password? Click here to reset