Does MAML really want feature reuse only?

08/20/2020
by   Jaehoon Oh, et al.
0

Meta-learning, the effort to solve new tasks with only a few samples, has attracted great attention in recent years. Model Agnostic Meta-Learning (MAML) is one of the most representative gradient-based meta-learning algorithms. MAML learns new tasks with a few data samples with inner updates from a meta-initialization point and learns the meta-initialization parameters with outer updates. Recently, it has been hypothesized that feature reuse, which makes little change in efficient representations, is the dominant factor in the performance of meta-initialized model through MAML rather than rapid learning, which makes a big change in representations. In this work, we propose a novel meta-learning algorithm, coined as BOIL (Body Only update in Inner Loop), that updates only the body (extractor) of the model and freezes the head (classifier) of the model during inner loop updates. The BOIL algorithm thus heavily relies on rapid learning. Note that BOIL is the opposite direction to the hypothesis that feature reuse is more efficient than rapid learning. We validate the BOIL algorithm on various data sets and show significant performance improvement over MAML. The results imply that rapid learning in gradient-based meta-learning approaches is necessary.

READ FULL TEXT

page 1

page 2

page 3

page 4

09/19/2019

Rapid Learning or Feature Reuse? Towards Understanding the Effectiveness of MAML

An important research direction in machine learning has centered around ...
09/20/2022

MAC: A Meta-Learning Approach for Feature Learning and Recombination

Optimization-based meta-learning aims to learn an initialization so that...
06/06/2018

Meta Learning by the Baldwin Effect

The scope of the Baldwin effect was recently called into question by two...
06/29/2021

MAML is a Noisy Contrastive Learner

Model-agnostic meta-learning (MAML) is one of the most popular and widel...
05/12/2021

Exploring the Similarity of Representations in Model-Agnostic Meta-Learning

In past years model-agnostic meta-learning (MAML) has been one of the mo...
03/18/2022

Negative Inner-Loop Learning Rates Learn Universal Features

Model Agnostic Meta-Learning (MAML) consists of two optimization loops: ...
10/03/2019

Generalized Inner Loop Meta-Learning

Many (but not all) approaches self-qualifying as "meta-learning" in deep...