Automatic formation of the structure of abstract machines in hierarchical reinforcement learning with state clustering

06/13/2018
by   Aleksandr I. Panov, et al.
0

We introduce a new approach to hierarchy formation and task decomposition in hierarchical reinforcement learning. Our method is based on the Hierarchy Of Abstract Machines (HAM) framework because HAM approach is able to design efficient controllers that will realize specific behaviors in real robots. The key to our algorithm is the introduction of the internal or "mental" environment in which the state represents the structure of the HAM hierarchy. The internal action in this environment leads to changes the hierarchy of HAMs. We propose the classical Q-learning procedure in the internal environment which allows the agent to obtain an optimal hierarchy. We extends the HAM framework by adding on-model approach to select the appropriate sub-machine to execute action sequences for certain class of external environment states. Preliminary experiments demonstrated the prospects of the method.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/16/2023

Creating Multi-Level Skill Hierarchies in Reinforcement Learning

What is a useful skill hierarchy for an autonomous agent? We propose an ...
research
09/14/2021

Continuous Homeostatic Reinforcement Learning for Self-Regulated Autonomous Agents

Homeostasis is a prevalent process by which living beings maintain their...
research
04/21/2022

Learning how to Interact with a Complex Interface using Hierarchical Reinforcement Learning

Hierarchical Reinforcement Learning (HRL) allows interactive agents to d...
research
07/13/2018

Exploring Hierarchy-Aware Inverse Reinforcement Learning

We introduce a new generative model for human planning under the Bayesia...
research
10/18/2021

Provable Hierarchy-Based Meta-Reinforcement Learning

Hierarchical reinforcement learning (HRL) has seen widespread interest a...
research
10/21/2021

Variational Predictive Routing with Nested Subjective Timescales

Discovery and learning of an underlying spatiotemporal hierarchy in sequ...
research
08/04/2022

Developmental Network Two, Its Optimality, and Emergent Turing Machines

Strong AI requires the learning engine to be task non-specific and to au...

Please sign up or login with your details

Forgot password? Click here to reset