A General framework for PAC-Bayes Bounds for Meta-Learning

06/11/2022
by   Arezou Rezazadeh, et al.
0

Meta learning automatically infers an inductive bias, that includes the hyperparameter of the base-learning algorithm, by observing data from a finite number of related tasks. This paper studies PAC-Bayes bounds on meta generalization gap. The meta-generalization gap comprises two sources of generalization gaps: the environment-level and task-level gaps resulting from observation of a finite number of tasks and data samples per task, respectively. In this paper, by upper bounding arbitrary convex functions, which link the expected and empirical losses at the environment and also per-task levels, we obtain new PAC-Bayes bounds. Using these bounds, we develop new PAC-Bayes meta-learning algorithms. Numerical examples demonstrate the merits of the proposed novel bounds and algorithm in comparison to prior PAC-Bayes bounds for meta-learning.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
02/07/2021

PAC-Bayes Bounds for Meta-learning with Data-Dependent Prior

By leveraging experience from previous tasks, meta-learning algorithms c...
research
11/03/2017

Lifelong Learning by Adjusting Priors

In representational lifelong learning an agent aims to continually learn...
research
02/13/2020

PACOH: Bayes-Optimal Meta-Learning with PAC-Guarantees

Meta-learning can successfully acquire useful inductive biases from data...
research
11/14/2022

PAC-Bayesian Meta-Learning: From Theory to Practice

Meta-Learning aims to accelerate the learning on new tasks by acquiring ...
research
02/23/2023

Bayes meets Bernstein at the Meta Level: an Analysis of Fast Rates in Meta-Learning with PAC-Bayes

Bernstein's condition is a key assumption that guarantees fast rates in ...
research
03/30/2022

Higher-Order Generalization Bounds: Learning Deep Probabilistic Programs via PAC-Bayes Objectives

Deep Probabilistic Programming (DPP) allows powerful models based on rec...
research
02/12/2021

PAC-BUS: Meta-Learning Bounds via PAC-Bayes and Uniform Stability

We are motivated by the problem of providing strong generalization guara...

Please sign up or login with your details

Forgot password? Click here to reset