Information Theoretic Meta Learning with Gaussian Processes

09/07/2020
by   Michalis K. Titsias, et al.
0

We formulate meta learning using information theoretic concepts such as mutual information and the information bottleneck. The idea is to learn a stochastic representation or encoding of the task description, given by a training or support set, that is highly informative about predicting the validation set. By making use of variational approximations to the mutual information we derive a general and tractable framework for meta learning. We particularly develop new memory-based meta learning algorithms based on Gaussian processes and derive extensions that combine memory and gradient based meta learning. We demonstrate our method on few-shot regression and classification by using standard benchmarks such as Omniglot, mini-Imagenet and Augmented Omniglot.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
09/29/2021

Generalization Bounds For Meta-Learning: An Information-Theoretic Analysis

We derive a novel information-theoretic analysis of the generalization p...
research
05/28/2019

Discrete Infomax Codes for Meta-Learning

Learning compact discrete representations of data is itself a key task i...
research
10/21/2021

Bayesian Meta-Learning Through Variational Gaussian Processes

Recent advances in the field of meta-learning have tackled domains consi...
research
06/01/2021

Information-Theoretic Analysis of Epistemic Uncertainty in Bayesian Meta-learning

The overall predictive uncertainty of a trained predictor can be decompo...
research
10/21/2020

Conditional Mutual Information Bound for Meta Generalization Gap

Meta-learning infers an inductive bias—typically in the form of the hype...
research
01/08/2021

An Information-theoretic Progressive Framework for Interpretation

Both brain science and the deep learning communities have the problem of...
research
10/19/2017

Meta-Learning via Feature-Label Memory Network

Deep learning typically requires training a very capable architecture us...

Please sign up or login with your details

Forgot password? Click here to reset