Action Space Shaping in Deep Reinforcement Learning

04/02/2020
by   Anssi Kanervisto, et al.
0

Reinforcement learning (RL) has been successful in training agents in various learning environments, including video-games. However, such work modifies and shrinks the action space from the game's original. This is to avoid trying "pointless" actions and to ease the implementation. Currently, this is mostly done based on intuition, with little systematic research supporting the design decisions. In this work, we aim to gain insight on these action space modifications by conducting extensive experiments in video-game environments. Our results show how domain-specific removal of actions and discretization of continuous actions can be crucial for successful learning. With these insights, we hope to ease the use of RL in new environments, by clarifying what action-spaces are easy to learn.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
09/06/2018

Learn What Not to Learn: Action Elimination with Deep Reinforcement Learning

Learning how to act when there are many available actions in each state ...
research
06/25/2022

Towards Modern Card Games with Large-Scale Action Spaces Through Action Representation

Axie infinity is a complicated card game with a huge-scale action space....
research
05/02/2019

From Video Game to Real Robot: The Transfer between Action Spaces

Training agents with reinforcement learning based techniques requires th...
research
10/26/2019

Comparing Observation and Action Representations for Deep Reinforcement Learning in MicroRTS

This paper presents a preliminary study comparing different observation ...
research
01/24/2019

Combinational Q-Learning for Dou Di Zhu

Deep reinforcement learning (DRL) has gained a lot of attention in recen...
research
04/08/2022

Data-Driven Evaluation of Training Action Space for Reinforcement Learning

Training action space selection for reinforcement learning (RL) is confl...
research
07/01/2021

Distilling Reinforcement Learning Tricks for Video Games

Reinforcement learning (RL) research focuses on general solutions that c...

Please sign up or login with your details

Forgot password? Click here to reset