Learning a Universal Template for Few-shot Dataset Generalization

05/14/2021
by   Eleni Triantafillou, et al.
15

Few-shot dataset generalization is a challenging variant of the well-studied few-shot classification problem where a diverse training set of several datasets is given, for the purpose of training an adaptable model that can then learn classes from new datasets using only a few examples. To this end, we propose to utilize the diverse training set to construct a universal template: a partial model that can define a wide array of dataset-specialized models, by plugging in appropriate components. For each new few-shot classification problem, our approach therefore only requires inferring a small number of parameters to insert into the universal template. We design a separate network that produces an initialization of those parameters for each given task, and we then fine-tune its proposed initialization via a few steps of gradient descent. Our approach is more parameter-efficient, scalable and adaptable compared to previous methods, and achieves the state-of-the-art on the challenging Meta-Dataset benchmark.

READ FULL TEXT

page 1

page 2

page 3

page 4

08/17/2022

Gradient-Based Meta-Learning Using Uncertainty to Weigh Loss for Few-Shot Learning

Model-Agnostic Meta-Learning (MAML) is one of the most successful meta-l...
03/07/2019

Meta-Dataset: A Dataset of Datasets for Learning to Learn from Few Examples

Few-shot classification refers to learning a classifier for new classes ...
06/21/2020

A Universal Representation Transformer Layer for Few-Shot Image Classification

Few-shot classification aims to recognize unseen classes when presented ...
03/20/2020

Selecting Relevant Features from a Universal Representation for Few-shot Classification

Popular approaches for few-shot classification consist of first learning...
06/30/2021

How to Train Your MAML to Excel in Few-Shot Classification

Model-agnostic meta-learning (MAML) is arguably the most popular meta-le...
09/26/2021

One-shot Key Information Extraction from Document with Deep Partial Graph Matching

Automating the Key Information Extraction (KIE) from documents improves ...
03/16/2022

Structurally Diverse Sampling Reduces Spurious Correlations in Semantic Parsing Datasets

A rapidly growing body of research has demonstrated the inability of NLP...