A New Meta-Baseline for Few-Shot Learning

03/09/2020
by   Yinbo Chen, et al.
11

Meta-learning has become a popular framework for few-shot learning in recent years, with the goal of learning a model from collections of few-shot classification tasks. While more and more novel meta-learning models are being proposed, our research has uncovered simple baselines that have been overlooked. We present a Meta-Baseline method, by pre-training a classifier on all base classes and meta-learning on a nearest-centroid based few-shot classification algorithm, it outperforms recent state-of-the-art methods by a large margin. Why does this simple method work so well? In the meta-learning stage, we observe that a model generalizing better on unseen tasks from base classes can have a decreasing performance on tasks from novel classes, indicating a potential objective discrepancy. We find both pre-training and inheriting a good few-shot classification metric from the pre-trained classifier are important for Meta-Baseline, which potentially helps the model better utilize the pre-trained representations with stronger transferability. Furthermore, we investigate when we need meta-learning in this Meta-Baseline. Our work sets up a new solid benchmark for this field and sheds light on further understanding the phenomenons in the meta-learning framework for few-shot learning.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
03/25/2020

Rethinking Few-Shot Image Classification: a Good Embedding Is All You Need?

The focus of recent meta-learning research has been on the development o...
research
02/03/2020

Revisiting Meta-Learning as Supervised Learning

Recent years have witnessed an abundance of new publications and approac...
research
02/18/2020

Few-Shot Few-Shot Learning and the role of Spatial Attention

Few-shot learning is often motivated by the ability of humans to learn n...
research
10/24/2021

A Closer Look at Few-Shot Video Classification: A New Baseline and Benchmark

The existing few-shot video classification methods often employ a meta-l...
research
05/27/2020

Looking back to lower-level information in few-shot learning

Humans are capable of learning new concepts from small numbers of exampl...
research
07/20/2022

Bitwidth-Adaptive Quantization-Aware Neural Network Training: A Meta-Learning Approach

Deep neural network quantization with adaptive bitwidths has gained incr...
research
01/26/2021

Few-Shot Semantic Parsing for New Predicates

In this work, we investigate the problems of semantic parsing in a few-s...

Please sign up or login with your details

Forgot password? Click here to reset