Multi-Pass Q-Networks for Deep Reinforcement Learning with Parameterised Action Spaces

05/10/2019
by   Craig J. Bester, et al.
0

Parameterised actions in reinforcement learning are composed of discrete actions with continuous action-parameters. This provides a framework for solving complex domains that require combining high-level actions with flexible control. The recent P-DQN algorithm extends deep Q-networks to learn over such action spaces. However, it treats all action-parameters as a single joint input to the Q-network, invalidating its theoretical foundations. We analyse the issues with this approach and propose a novel method, multi-pass deep Q-networks, or MP-DQN, to address them. We empirically demonstrate that MP-DQN significantly outperforms P-DQN and other previous algorithms in terms of data efficiency and converged policy performance on the Platform, Robot Soccer Goal, and Half Field Offense domains.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
09/05/2015

Reinforcement Learning with Parameterized Actions

We introduce a model-free algorithm for learning in Markov decision proc...
research
10/07/2021

Design Strategy Network: A deep hierarchical framework to represent generative design strategies in complex action spaces

Generative design problems often encompass complex action spaces that ma...
research
04/13/2021

Learning and Planning in Complex Action Spaces

Many important real-world problems have action spaces that are high-dime...
research
03/03/2022

Implicit Kinematic Policies: Unifying Joint and Cartesian Action Spaces in End-to-End Robot Learning

Action representation is an important yet often overlooked aspect in end...
research
05/11/2022

Characterizing the Action-Generalization Gap in Deep Q-Learning

We study the action generalization ability of deep Q-learning in discret...
research
03/07/2019

Learning Hierarchical Teaching in Cooperative Multiagent Reinforcement Learning

Heterogeneous knowledge naturally arises among different agents in coope...
research
01/11/2018

Model-Based Action Exploration

Deep reinforcement learning has great stride in solving challenging moti...

Please sign up or login with your details

Forgot password? Click here to reset