Growing Action Spaces

06/28/2019
by   Gregory Farquhar, et al.
7

In complex tasks, such as those with large combinatorial action spaces, random exploration may be too inefficient to achieve meaningful learning progress. In this work, we use a curriculum of progressively growing action spaces to accelerate learning. We assume the environment is out of our control, but that the agent may set an internal curriculum by initially restricting its action space. Our approach uses off-policy reinforcement learning to estimate optimal value functions for multiple action spaces simultaneously and efficiently transfers data, value estimates, and state representations from restricted action spaces to the full task. We show the efficacy of our approach in proof-of-concept control tasks and on challenging large-scale StarCraft micromanagement tasks with large, multi-agent action spaces.

READ FULL TEXT
research
04/15/2021

Generalising Discrete Action Spaces with Conditional Action Trees

There are relatively few conventions followed in reinforcement learning ...
research
09/19/2022

MAN: Multi-Action Networks Learning

Learning control policies with large action spaces is a challenging prob...
research
08/23/2022

Entropy Enhanced Multi-Agent Coordination Based on Hierarchical Graph Learning for Continuous Action Space

In most existing studies on large-scale multi-agent coordination, the co...
research
10/28/2020

Learning to Represent Action Values as a Hypergraph on the Action Vertices

Action-value estimation is a critical component of many reinforcement le...
research
09/06/2019

From Few to More: Large-scale Dynamic Multiagent Curriculum Learning

A lot of efforts have been devoted to investigating how agents can learn...
research
02/25/2022

Towards neoRL networks; the emergence of purposive graphs

The neoRL framework for purposive AI implements latent learning by emula...
research
06/05/2018

Mix&Match - Agent Curricula for Reinforcement Learning

We introduce Mix&Match (M&M) - a training framework designed to facilita...

Please sign up or login with your details

Forgot password? Click here to reset