DeepAI AI Chat
Log In Sign Up

Federated Few-Shot Learning with Adversarial Learning

by   Chenyou Fan, et al.

We are interested in developing a unified machine learning model over many mobile devices for practical learning tasks, where each device only has very few training data. This is a commonly encountered situation in mobile computing scenarios, where data is scarce and distributed while the tasks are distinct. In this paper, we propose a federated few-shot learning (FedFSL) framework to learn a few-shot classification model that can classify unseen data classes with only a few labeled samples. With the federated learning strategy, FedFSL can utilize many data sources while keeping data privacy and communication efficiency. There are two technical challenges: 1) directly using the existing federated learning approach may lead to misaligned decision boundaries produced by client models, and 2) constraining the decision boundaries to be similar over clients would overfit to training tasks but not adapt well to unseen tasks. To address these issues, we propose to regularize local updates by minimizing the divergence of client models. We also formulate the training in an adversarial fashion and optimize the client models to produce a discriminative feature space that can better represent unseen data samples. We demonstrate the intuitions and conduct experiments to show our approaches outperform baselines by more than 10 language tasks.


page 1

page 2

page 3

page 4


One-Shot Federated Learning with Neuromorphic Processors

Being very low power, the use of neuromorphic processors in mobile devic...

Federated Few-shot Learning for Cough Classification with Edge Devices

Automatically classifying cough sounds is one of the most critical tasks...

Federated Zero-Shot Learning for Visual Recognition

Zero-shot learning is a learning regime that recognizes unseen classes b...

FewFedWeight: Few-shot Federated Learning Framework across Multiple NLP Tasks

Massively multi-task learning with large language models has recently ma...

CatFedAvg: Optimising Communication-efficiency and Classification Accuracy in Federated Learning

Federated learning has allowed the training of statistical models over r...

FedYolo: Augmenting Federated Learning with Pretrained Transformers

The growth and diversity of machine learning applications motivate a ret...

Functional Federated Learning in Erlang (ffl-erl)

The functional programming language Erlang is well-suited for concurrent...