Hierarchical Reinforcement Learning in Complex 3D Environments

02/28/2023
by   Bernardo Ávila Pires, et al.
0

Hierarchical Reinforcement Learning (HRL) agents have the potential to demonstrate appealing capabilities such as planning and exploration with abstraction, transfer, and skill reuse. Recent successes with HRL across different domains provide evidence that practical, effective HRL agents are possible, even if existing agents do not yet fully realize the potential of HRL. Despite these successes, visually complex partially observable 3D environments remained a challenge for HRL agents. We address this issue with Hierarchical Hybrid Offline-Online (H2O2), a hierarchical deep reinforcement learning agent that discovers and learns to use options from scratch using its own experience. We show that H2O2 is competitive with a strong non-hierarchical Muesli baseline in the DeepMind Hard Eight tasks and we shed new light on the problem of learning hierarchical agents in complex environments. Our empirical study of H2O2 reveals previously unnoticed practical challenges and brings new perspective to the current understanding of hierarchical agents in complex domains.

READ FULL TEXT

page 3

page 21

research
05/18/2018

Hierarchical Reinforcement Learning with Deep Nested Agents

Deep hierarchical reinforcement learning has gained a lot of attention i...
research
06/23/2021

Evolving Hierarchical Memory-Prediction Machines in Multi-Task Reinforcement Learning

A fundamental aspect of behaviour is the ability to encode salient featu...
research
06/12/2022

Matching options to tasks using Option-Indexed Hierarchical Reinforcement Learning

The options framework in Hierarchical Reinforcement Learning breaks down...
research
03/23/2022

Possibility Before Utility: Learning And Using Hierarchical Affordances

Reinforcement learning algorithms struggle on tasks with complex hierarc...
research
04/20/2016

Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivation

Learning goal-directed behavior in environments with sparse feedback is ...
research
05/31/2021

Procedural Content Generation: Better Benchmarks for Transfer Reinforcement Learning

The idea of transfer in reinforcement learning (TRL) is intriguing: bein...
research
11/27/2015

Multiagent Cooperation and Competition with Deep Reinforcement Learning

Multiagent systems appear in most social, economical, and political situ...

Please sign up or login with your details

Forgot password? Click here to reset