Eliminating Meta Optimization Through Self-Referential Meta Learning

12/29/2022
by   Louis Kirsch, et al.
0

Meta Learning automates the search for learning algorithms. At the same time, it creates a dependency on human engineering on the meta-level, where meta learning algorithms need to be designed. In this paper, we investigate self-referential meta learning systems that modify themselves without the need for explicit meta optimization. We discuss the relationship of such systems to in-context and memory-based meta learning and show that self-referential neural networks require functionality to be reused in the form of parameter sharing. Finally, we propose fitness monotonic execution (FME), a simple approach to avoid explicit meta optimization. A neural network self-modifies to solve bandit and classic control tasks, improves its self-modifications, and learns how to learn, purely by assigning more computational resources to better performing solutions.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/24/2020

Modeling and Optimization Trade-off in Meta-learning

By searching for shared inductive biases across tasks, meta-learning pro...
research
02/25/2022

Meta-Learning for Simple Regret Minimization

We develop a meta-learning framework for simple regret minimization in b...
research
03/16/2023

Arbitrary Order Meta-Learning with Simple Population-Based Evolution

Meta-learning, the notion of learning to learn, enables learning systems...
research
04/12/2023

Meta-Learned Models of Cognition

Meta-learning is a framework for learning learning algorithms through re...
research
02/02/2023

Meta Learning in Decentralized Neural Networks: Towards More General AI

Meta-learning usually refers to a learning algorithm that learns from ot...
research
05/20/2023

Meta Neural Coordination

Meta-learning aims to develop algorithms that can learn from other learn...
research
10/07/2020

A Survey of Deep Meta-Learning

Deep neural networks can achieve great successes when presented with lar...

Please sign up or login with your details

Forgot password? Click here to reset