Federated Few-Shot Learning with Adversarial Learning

04/01/2021
by   Chenyou Fan, et al.
0

We are interested in developing a unified machine learning model over many mobile devices for practical learning tasks, where each device only has very few training data. This is a commonly encountered situation in mobile computing scenarios, where data is scarce and distributed while the tasks are distinct. In this paper, we propose a federated few-shot learning (FedFSL) framework to learn a few-shot classification model that can classify unseen data classes with only a few labeled samples. With the federated learning strategy, FedFSL can utilize many data sources while keeping data privacy and communication efficiency. There are two technical challenges: 1) directly using the existing federated learning approach may lead to misaligned decision boundaries produced by client models, and 2) constraining the decision boundaries to be similar over clients would overfit to training tasks but not adapt well to unseen tasks. To address these issues, we propose to regularize local updates by minimizing the divergence of client models. We also formulate the training in an adversarial fashion and optimize the client models to produce a discriminative feature space that can better represent unseen data samples. We demonstrate the intuitions and conduct experiments to show our approaches outperform baselines by more than 10 language tasks.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

11/01/2020

One-Shot Federated Learning with Neuromorphic Processors

Being very low power, the use of neuromorphic processors in mobile devic...
11/08/2020

Adaptive Federated Dropout: Improving Communication Efficiency and Generalization for Federated Learning

With more regulations tackling users' privacy-sensitive data protection ...
11/14/2020

CatFedAvg: Optimising Communication-efficiency and Classification Accuracy in Federated Learning

Federated learning has allowed the training of statistical models over r...
01/14/2021

Auto-weighted Robust Federated Learning with Corrupted Data Sources

Federated learning provides a communication-efficient and privacy-preser...
08/13/2020

WAFFLe: Weight Anonymized Factorization for Federated Learning

In domains where data are sensitive or private, there is great value in ...
11/20/2021

Federated Learning with Domain Generalization

Federated Learning (FL) enables a group of clients to jointly train a ma...
08/24/2018

Functional Federated Learning in Erlang (ffl-erl)

The functional programming language Erlang is well-suited for concurrent...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.