Distribution-Agnostic Model-Agnostic Meta-Learning

by   Liam Collins, et al.

The Model-Agnostic Meta-Learning (MAML) algorithm <cit.> has been celebrated for its efficiency and generality, as it has demonstrated success in quickly learning the parameters of an arbitrary learning model. However, MAML implicitly assumes that the tasks come from a particular distribution, and optimizes the expected (or sample average) loss over tasks drawn from this distribution. Here, we amend this limitation of MAML by reformulating the objective function as a min-max problem, where the maximization is over the set of possible distributions over tasks. Our proposed algorithm is the first distribution-agnostic and model-agnostic meta-learning method, and we show that it converges to an ϵ-accurate point at the rate of O(1/ϵ^2) in the convex setting and to an (ϵ, δ)-stationary point at the rate of O(max{1/ϵ^5, 1/δ^5}) in nonconvex settings. We also provide numerical experiments that demonstrate the worst-case superiority of our algorithm in comparison to MAML.



page 1

page 2

page 3

page 4


Is Bayesian Model-Agnostic Meta Learning Better than Model-Agnostic Meta Learning, Provably?

Meta learning aims at learning a model that can quickly adapt to unseen ...

ST-MAML: A Stochastic-Task based Method for Task-Heterogeneous Meta-Learning

Optimization-based meta-learning typically assumes tasks are sampled fro...

On the Convergence Theory of Gradient-Based Model-Agnostic Meta-Learning Algorithms

In this paper, we study the convergence theory of a class of gradient-ba...

Learning to Broadcast for Ultra-Reliable Communication with Differential Quality of Service via the Conditional Value at Risk

Broadcast/multicast communication systems are typically designed to opti...

Compositional ADAM: An Adaptive Compositional Solver

In this paper, we present C-ADAM, the first adaptive solver for composit...

Meta Variational Monte Carlo

An identification is found between meta-learning and the problem of dete...

On the Global Optimality of Model-Agnostic Meta-Learning

Model-agnostic meta-learning (MAML) formulates meta-learning as a bileve...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.