FiT: Parameter Efficient Few-shot Transfer Learning for Personalized and Federated Image Classification

by   Aliaksandra Shysheya, et al.
University of Cambridge

Modern deep learning systems are increasingly deployed in situations such as personalization and federated learning where it is necessary to support i) learning on small amounts of data, and ii) communication efficient distributed training protocols. In this work we develop FiLM Transfer (FiT) which fulfills these requirements in the image classification setting. FiT uses an automatically configured Naive Bayes classifier on top of a fixed backbone that has been pretrained on large image datasets. Parameter efficient FiLM layers are used to modulate the backbone, shaping the representation for the downstream task. The network is trained via an episodic fine-tuning protocol. The approach is parameter efficient which is key for enabling few-shot learning, inexpensive model updates for personalization, and communication efficient federated learning. We experiment with FiT on a wide range of downstream datasets and show that it achieves better classification accuracy than the state-of-the-art Big Transfer (BiT) algorithm at low-shot and on the challenging VTAB-1k benchmark, with fewer than 1 Finally, we demonstrate the parameter efficiency of FiT in distributed low-shot applications including model personalization and federated learning where model update size is an important performance metric.


page 1

page 2

page 3

page 4


On the Efficacy of Differentially Private Few-shot Image Classification

There has been significant recent progress in training differentially pr...

Communication-Efficient and Privacy-Preserving Feature-based Federated Transfer Learning

Federated learning has attracted growing interest as it preserves the cl...

Contextual Squeeze-and-Excitation for Efficient Few-Shot Image Classification

Recent years have seen a growth in user-centric applications that requir...

Pretraining Federated Text Models for Next Word Prediction

Federated learning is a decentralized approach for training models on di...

Federated Learning Over Images: Vertical Decompositions and Pre-Trained Backbones Are Difficult to Beat

We carefully evaluate a number of algorithms for learning in a federated...

Federated Few-shot Learning for Cough Classification with Edge Devices

Automatically classifying cough sounds is one of the most critical tasks...

Code Repositories

Please sign up or login with your details

Forgot password? Click here to reset