Bridging the Gap Between Practice and PAC-Bayes Theory in Few-Shot Meta-Learning

05/28/2021
by   Nan Ding, et al.
0

Despite recent advances in its theoretical understanding, there still remains a significant gap in the ability of existing PAC-Bayesian theories on meta-learning to explain performance improvements in the few-shot learning setting, where the number of training examples in the target tasks is severely limited. This gap originates from an assumption in the existing theories which supposes that the number of training examples in the observed tasks and the number of training examples in the target tasks follow the same distribution, an assumption that rarely holds in practice. By relaxing this assumption, we develop two PAC-Bayesian bounds tailored for the few-shot learning setting and show that two existing meta-learning algorithms (MAML and Reptile) can be derived from our bounds, thereby bridging the gap between practice and PAC-Bayesian theories. Furthermore, we derive a new computationally-efficient PACMAML algorithm, and show it outperforms existing meta-learning algorithms on several few-shot benchmark datasets.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
02/07/2021

PAC-Bayes Bounds for Meta-learning with Data-Dependent Prior

By leveraging experience from previous tasks, meta-learning algorithms c...
research
09/29/2021

Towards a theory of out-of-distribution learning

What is learning? 20^st century formalizations of learning theory – whic...
research
07/06/2022

PAC Prediction Sets for Meta-Learning

Uncertainty quantification is a key component of machine learning models...
research
10/14/2020

Theoretical bounds on estimation error for meta-learning

Machine learning models have traditionally been developed under the assu...
research
12/07/2022

Learn to Explore: on Bootstrapping Interactive Data Exploration with Meta-learning

Interactive data exploration (IDE) is an effective way of comprehending ...
research
03/05/2020

PAC-Bayesian Meta-learning with Implicit Prior

We introduce a new and rigorously-formulated PAC-Bayes few-shot meta-lea...
research
02/13/2020

PACOH: Bayes-Optimal Meta-Learning with PAC-Guarantees

Meta-learning can successfully acquire useful inductive biases from data...

Please sign up or login with your details

Forgot password? Click here to reset