Hierarchical Subtask Discovery With Non-Negative Matrix Factorization

08/01/2017
by   Adam C. Earle, et al.
0

Hierarchical reinforcement learning methods offer a powerful means of planning flexible behavior in complicated domains. However, learning an appropriate hierarchical decomposition of a domain into subtasks remains a substantial challenge. We present a novel algorithm for subtask discovery, based on the recently introduced multitask linearly-solvable Markov decision process (MLMDP) framework. The MLMDP can perform never-before-seen tasks by representing them as a linear combination of a previously learned basis set of tasks. In this setting, the subtask discovery problem can naturally be posed as finding an optimal low-rank approximation of the set of tasks the agent will face in a domain. We use non-negative matrix factorization to discover this minimal basis set of tasks, and show that the technique learns intuitive decompositions in a variety of domains. Our method has several qualitatively desirable features: it is not limited to learning subtasks with single goal states, instead learning distributed patterns of preferred states; it learns qualitatively different hierarchical decompositions in the same domain depending on the ensemble of tasks the agent will face; and it may be straightforwardly iterated to obtain deeper hierarchical decompositions.

READ FULL TEXT

page 3

page 5

research
03/10/2016

Hierarchical Linearly-Solvable Markov Decision Problems

We present a hierarchical reinforcement learning framework that formulat...
research
09/12/2022

Population-Based Hierarchical Non-negative Matrix Factorization for Survey Data

Motivated by the problem of identifying potential hierarchical populatio...
research
11/02/2021

Learning Multiresolution Matrix Factorization and its Wavelet Networks on Graphs

Multiresolution Matrix Factorization (MMF) is unusual amongst fast matri...
research
07/29/2019

Perspectives on CUR Decompositions

This note discusses an interesting matrix factorization called the CUR D...
research
12/08/2016

Hierarchy through Composition with Linearly Solvable Markov Decision Processes

Hierarchical architectures are critical to the scalability of reinforcem...
research
09/01/2022

Unsupervised EHR-based Phenotyping via Matrix and Tensor Decompositions

Computational phenotyping allows for unsupervised discovery of subgroups...
research
09/28/2021

Unsupervised Learning for Submarket Modeling: A Proxy for Neighborhood Change

This study focused on submarket modeling with unsupervised learning and ...

Please sign up or login with your details

Forgot password? Click here to reset