The Advantage of Conditional Meta-Learning for Biased Regularization and Fine-Tuning

08/25/2020
by   Giulia Denevi, et al.
0

Biased regularization and fine-tuning are two recent meta-learning approaches. They have been shown to be effective to tackle distributions of tasks, in which the tasks' target vectors are all close to a common meta-parameter vector. However, these methods may perform poorly on heterogeneous environments of tasks, where the complexity of the tasks' distribution cannot be captured by a single meta-parameter vector. We address this limitation by conditional meta-learning, inferring a conditioning function mapping task's side information into a meta-parameter vector that is appropriate for that task at hand. We characterize properties of the environment under which the conditional approach brings a substantial advantage over standard meta-learning and we highlight examples of environments, such as those with multiple clusters, satisfying these properties. We then propose a convex meta-algorithm providing a comparable advantage also in practice. Numerical experiments confirm our theoretical findings.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
03/30/2021

Conditional Meta-Learning of Linear Representations

Standard meta-learning for representation learning aims to find a common...
research
06/07/2022

On the Effectiveness of Fine-tuning Versus Meta-reinforcement Learning

Intelligent agents should have the ability to leverage knowledge from pr...
research
07/29/2020

Meta-Learning with Context-Agnostic Initialisations

Meta-learning approaches have addressed few-shot problems by finding ini...
research
10/04/2022

Uncertainty-Aware Meta-Learning for Multimodal Task Distributions

Meta-learning or learning to learn is a popular approach for learning ne...
research
07/11/2020

Online Parameter-Free Learning of Multiple Low Variance Tasks

We propose a method to learn a common bias vector for a growing sequence...
research
06/23/2023

Comparing the Efficacy of Fine-Tuning and Meta-Learning for Few-Shot Policy Imitation

In this paper we explore few-shot imitation learning for control problem...
research
11/06/2019

Fair Meta-Learning: Learning How to Learn Fairly

Data sets for fairness relevant tasks can lack examples or be biased acc...

Please sign up or login with your details

Forgot password? Click here to reset