Mix&Match - Agent Curricula for Reinforcement Learning

We introduce Mix&Match (M&M) - a training framework designed to facilitate rapid and effective learning in RL agents, especially those that would be too slow or too challenging to train otherwise. The key innovation is a procedure that allows us to automatically form a curriculum over agents. Through such a curriculum we can progressively train more complex agents by, effectively, bootstrapping from solutions found by simpler agents. In contradistinction to typical curriculum learning approaches, we do not gradually modify the tasks or environments presented, but instead use a process to gradually alter how the policy is represented internally. We show the broad applicability of our method by demonstrating significant performance gains in three different experimental setups: (1) We train an agent able to control more than 700 actions in a challenging 3D first-person task; using our method to progress through an action-space curriculum we achieve both faster training and better final performance than one obtains using traditional methods. (2) We further show that M&M can be used successfully to progress through a curriculum of architectural variants defining an agents internal state. (3) Finally, we illustrate how a variant of our method can be used to improve agent performance in a multitask setting.

READ FULL TEXT
research
06/28/2021

Multi-task curriculum learning in a complex, visual, hard-exploration domain: Minecraft

An important challenge in reinforcement learning is training agents that...
research
12/01/2018

Learning Curriculum Policies for Reinforcement Learning

Curriculum learning in reinforcement learning is a training methodology ...
research
12/30/2022

Reinforcement Learning with Success Induced Task Prioritization

Many challenging reinforcement learning (RL) problems require designing ...
research
06/28/2019

Growing Action Spaces

In complex tasks, such as those with large combinatorial action spaces, ...
research
09/06/2019

From Few to More: Large-scale Dynamic Multiagent Curriculum Learning

A lot of efforts have been devoted to investigating how agents can learn...
research
06/09/2023

Self-Paced Absolute Learning Progress as a Regularized Approach to Curriculum Learning

The usability of Reinforcement Learning is restricted by the large compu...
research
01/28/2019

CLIC: Curriculum Learning and Imitation for feature Control in non-rewarding environments

In this paper, we propose an unsupervised reinforcement learning agent c...

Please sign up or login with your details

Forgot password? Click here to reset