DeepAI AI Chat
Log In Sign Up

Human-Timescale Adaptation in an Open-Ended Task Space

by   Adaptive Agent Team, et al.

Foundation models have shown impressive adaptation and scalability in supervised and self-supervised learning problems, but so far these successes have not fully translated to reinforcement learning (RL). In this work, we demonstrate that training an RL agent at scale leads to a general in-context learning algorithm that can adapt to open-ended novel embodied 3D problems as quickly as humans. In a vast space of held-out environment dynamics, our adaptive agent (AdA) displays on-the-fly hypothesis-driven exploration, efficient exploitation of acquired knowledge, and can successfully be prompted with first-person demonstrations. Adaptation emerges from three ingredients: (1) meta-reinforcement learning across a vast, smooth and diverse task distribution, (2) a policy parameterised as a large-scale attention-based memory architecture, and (3) an effective automated curriculum that prioritises tasks at the frontier of an agent's capabilities. We demonstrate characteristic scaling laws with respect to network size, memory length, and richness of the training task distribution. We believe our results lay the foundation for increasingly general and adaptive RL agents that perform well across ever-larger open-ended domains.


page 1

page 2

page 4

page 7

page 11

page 12

page 36

page 37


Offline Meta Reinforcement Learning

Consider the following problem, which we term Offline Meta Reinforcement...

URLB: Unsupervised Reinforcement Learning Benchmark

Deep Reinforcement Learning (RL) has emerged as a powerful paradigm to s...

Learn to Effectively Explore in Context-Based Meta-RL

Meta reinforcement learning (meta-RL) provides a principled approach for...

Learning Context-aware Task Reasoning for Efficient Meta-reinforcement Learning

Despite recent success of deep network-based Reinforcement Learning (RL)...

earning Context-aware Task Reasoning for Efficient Meta Reinforcement Learning

Despite recent success of deep network-based Reinforcement Learning (RL)...

BIMRL: Brain Inspired Meta Reinforcement Learning

Sample efficiency has been a key issue in reinforcement learning (RL). A...

Value Driven Representation for Human-in-the-Loop Reinforcement Learning

Interactive adaptive systems powered by Reinforcement Learning (RL) have...