Emergency action termination for immediate reaction in hierarchical reinforcement learning

by   Michał Bortkiewicz, et al.

Hierarchical decomposition of control is unavoidable in large dynamical systems. In reinforcement learning (RL), it is usually solved with subgoals defined at higher policy levels and achieved at lower policy levels. Reaching these goals can take a substantial amount of time, during which it is not verified whether they are still worth pursuing. However, due to the randomness of the environment, these goals may become obsolete. In this paper, we address this gap in the state-of-the-art approaches and propose a method in which the validity of higher-level actions (thus lower-level goals) is constantly verified at the higher level. If the actions, i.e. lower level goals, become inadequate, they are replaced by more appropriate ones. This way we combine the advantages of hierarchical RL, which is fast training, and flat RL, which is immediate reactivity. We study our approach experimentally on seven benchmark environments.


page 6

page 7


Learning Functionally Decomposed Hierarchies for Continuous Control Tasks

Solving long-horizon sequential decision making tasks in environments wi...

Hierarchical Reinforcement Learning for Deep Goal Reasoning: An Expressiveness Analysis

Hierarchical DQN (h-DQN) is a two-level architecture of feedforward neur...

Data-Efficient Hierarchical Reinforcement Learning

Hierarchical reinforcement learning (HRL) is a promising approach to ext...

A Crash Course on Reinforcement Learning

The emerging field of Reinforcement Learning (RL) has led to impressive ...

Diversity-Driven Extensible Hierarchical Reinforcement Learning

Hierarchical reinforcement learning (HRL) has recently shown promising a...

Many-Goals Reinforcement Learning

All-goals updating exploits the off-policy nature of Q-learning to updat...

Following Instructions by Imagining and Reaching Visual Goals

While traditional methods for instruction-following typically assume pri...

Please sign up or login with your details

Forgot password? Click here to reset