Parametrized Deep Q-Networks Learning: Reinforcement Learning with Discrete-Continuous Hybrid Action Space

10/10/2018

∙

Most existing deep reinforcement learning (DRL) frameworks consider either discrete action space or continuous action space solely. Motivated by applications in computer games, we consider the scenario with discrete-continuous hybrid action space. To handle hybrid action space, previous works either approximate the hybrid space by discretization, or relax it into a continuous set. In this paper, we propose a parametrized deep Q-network (P- DQN) framework for the hybrid action space without approximation or relaxation. Our algorithm combines the spirits of both DQN (dealing with discrete action space) and DDPG (dealing with continuous action space) by seamlessly integrating them. Empirical results on a simulation example, scoring a goal in simulated RoboCup soccer and the solo mode in game King of Glory (KOG) validate the efficiency and effectiveness of our method.

READ FULL TEXT

Parametrized Deep Q-Networks Learning: Reinforcement Learning with Discrete-Continuous Hybrid Action Space

HyAR: Addressing Discrete-Continuous Action Reinforcement Learning via Hybrid Action Representation

A0C: Alpha Zero in Continuous Action Space

Deep Jump Q-Evaluation for Offline Policy Evaluation in Continuous Action Space

Discrete linear-complexity reinforcement learning in continuous action spaces for Q-learning algorithms

Reinforcement learning for traffic signal control in hybrid action space

Learn Continuously, Act Discretely: Hybrid Action-Space Reinforcement Learning For Optimal Execution

Deep Reinforcement Learning with Discrete Normalized Advantage Functions for Resource Management in Network Slicing

Parametrized Deep Q-Networks Learning: Reinforcement Learning with Discrete-Continuous Hybrid Action Space

Related Research

HyAR: Addressing Discrete-Continuous Action Reinforcement Learning via Hybrid Action Representation

A0C: Alpha Zero in Continuous Action Space

Deep Jump Q-Evaluation for Offline Policy Evaluation in Continuous Action Space

Discrete linear-complexity reinforcement learning in continuous action spaces for Q-learning algorithms

Reinforcement learning for traffic signal control in hybrid action space

Learn Continuously, Act Discretely: Hybrid Action-Space Reinforcement Learning For Optimal Execution

Deep Reinforcement Learning with Discrete Normalized Advantage Functions for Resource Management in Network Slicing