ST-MAML: A Stochastic-Task based Method for Task-Heterogeneous Meta-Learning

09/27/2021
by   Zhe Wang, et al.
1

Optimization-based meta-learning typically assumes tasks are sampled from a single distribution - an assumption oversimplifies and limits the diversity of tasks that meta-learning can model. Handling tasks from multiple different distributions is challenging for meta-learning due to a so-called task ambiguity issue. This paper proposes a novel method, ST-MAML, that empowers model-agnostic meta-learning (MAML) to learn from multiple task distributions. ST-MAML encodes tasks using a stochastic neural network module, that summarizes every task with a stochastic representation. The proposed Stochastic Task (ST) strategy allows a meta-model to get tailored for the current task and enables us to learn a distribution of solutions for an ambiguous task. ST-MAML also propagates the task representation to revise the encoding of input variables. Empirically, we demonstrate that ST-MAML matches or outperforms the state-of-the-art on two few-shot image classification tasks, one curve regression benchmark, one image completion problem, and a real-world temperature prediction application. To the best of authors' knowledge, this is the first time optimization-based meta-learning method being applied on a large-scale real-world task.

READ FULL TEXT
research
07/05/2021

Meta-learning Amidst Heterogeneity and Ambiguity

Meta-learning aims to learn a model that can handle multiple tasks gener...
research
03/23/2022

Multidimensional Belief Quantification for Label-Efficient Meta-Learning

Optimization-based meta-learning offers a promising direction for few-sh...
research
02/20/2021

Meta-Learning Dynamics Forecasting Using Task Inference

Current deep learning models for dynamics forecasting struggle with gene...
research
09/04/2019

Meta Learning with Relational Information for Short Sequences

This paper proposes a new meta-learning method -- named HARMLESS (HAwkes...
research
07/15/2023

Generative Meta-Learning Robust Quality-Diversity Portfolio

This paper proposes a novel meta-learning approach to optimize a robust ...
research
02/12/2020

Distribution-Agnostic Model-Agnostic Meta-Learning

The Model-Agnostic Meta-Learning (MAML) algorithm <cit.> has been celebr...
research
07/08/2022

MACFE: A Meta-learning and Causality Based Feature Engineering Framework

Feature engineering has become one of the most important steps to improv...

Please sign up or login with your details

Forgot password? Click here to reset