On the Generalization of Wasserstein Robust Federated Learning

06/03/2022
by   Tung-Anh Nguyen, et al.
0

In federated learning, participating clients typically possess non-i.i.d. data, posing a significant challenge to generalization to unseen distributions. To address this, we propose a Wasserstein distributionally robust optimization scheme called WAFL. Leveraging its duality, we frame WAFL as an empirical surrogate risk minimization problem, and solve it using a local SGD-based algorithm with convergence guarantees. We show that the robustness of WAFL is more general than related approaches, and the generalization bound is robust to all adversarial distributions inside the Wasserstein ball (ambiguity set). Since the center location and radius of the Wasserstein ball can be suitably modified, WAFL shows its applicability not only in robustness but also in domain adaptation. Through empirical evaluation, we demonstrate that WAFL generalizes better than the vanilla FedAvg in non-i.i.d. settings, and is more robust than other related methods in distribution shift settings. Further, using benchmark datasets we show that WAFL is capable of generalizing to unseen target domains.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/27/2021

What Do We Mean by Generalization in Federated Learning?

Federated learning data is drawn from a distribution of distributions: c...
research
06/16/2020

Robust Federated Learning: The Case of Affine Distribution Shifts

Federated learning is a distributed paradigm that aims at training model...
research
12/12/2022

On Generalization and Regularization via Wasserstein Distributionally Robust Optimization

Wasserstein distributionally robust optimization (DRO) has found success...
research
11/03/2022

FedGen: Generalizable Federated Learning

Existing federated learning models that follow the standard risk minimiz...
research
04/16/2022

DRFLM: Distributionally Robust Federated Learning with Inter-client Noise via Local Mixup

Recently, federated learning has emerged as a promising approach for tra...
research
06/02/2023

Federated Domain Generalization: A Survey

Machine learning typically relies on the assumption that training and te...
research
01/14/2020

Wasserstein Distributionally Robust Motion Control for Collision Avoidance Using Conditional Value-at-Risk

In this paper, a risk-aware motion control scheme is considered for mobi...

Please sign up or login with your details

Forgot password? Click here to reset