Generalization of Model-Agnostic Meta-Learning Algorithms: Recurring and Unseen Tasks

02/07/2021
by   Alireza Fallah, et al.
0

In this paper, we study the generalization properties of Model-Agnostic Meta-Learning (MAML) algorithms for supervised learning problems. We focus on the setting in which we train the MAML model over m tasks, each with n data points, and characterize its generalization error from two points of view: First, we assume the new task at test time is one of the training tasks, and we show that, for strongly convex objective functions, the expected excess population loss is bounded by 𝒪(1/mn). Second, we consider the MAML algorithm's generalization to an unseen task and show that the resulting generalization error depends on the total variation distance between the underlying distributions of the new task and the tasks observed during the training process. Our proof techniques rely on the connections between algorithmic stability and generalization bounds of algorithms. In particular, we propose a new definition of stability for meta-learning algorithms, which allows us to capture the role of both the number of tasks m and number of samples per task n on the generalization error of MAML.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/26/2021

Meta-learning with an Adaptive Task Scheduler

To benefit the learning of a new task, meta-learning has been proposed t...
research
03/20/2020

Weighted Meta-Learning

Meta-learning leverages related source tasks to learn an initialization ...
research
07/29/2020

Meta-Learning with Context-Agnostic Initialisations

Meta-learning approaches have addressed few-shot problems by finding ini...
research
05/01/2023

Model-agnostic Measure of Generalization Difficulty

The measure of a machine learning algorithm is the difficulty of the tas...
research
12/10/2020

Visual Perception Generalization for Vision-and-Language Navigation via Meta-Learning

Vision-and-language navigation (VLN) is a challenging task that requires...
research
08/26/2019

An Introduction to Advanced Machine Learning : Meta Learning Algorithms, Applications and Promises

In [1, 2], we have explored the theoretical aspects of feature extractio...
research
01/21/2021

An Information-Theoretic Analysis of the Impact of Task Similarity on Meta-Learning

Meta-learning aims at optimizing the hyperparameters of a model class or...

Please sign up or login with your details

Forgot password? Click here to reset