DeepAI AI Chat
Log In Sign Up

Option Encoder: A Framework for Discovering a Policy Basis in Reinforcement Learning

09/09/2019
by   Arjun Manoharan, et al.
Indian Institute Of Technology, Madras
6

Option discovery and skill acquisition frameworks are integral to the functioning of a Hierarchically organized Reinforcement learning agent. However, such techniques often yield a large number of options or skills, which can potentially be represented succinctly by filtering out any redundant information. Such a reduction can reduce the required computation while also improving the performance on a target task. In order to compress an array of option policies, we attempt to find a policy basis that accurately captures the set of all options. In this work, we propose Option Encoder, an auto-encoder based framework with intelligently constrained weights, that helps discover a collection of basis policies. The policy basis can be used as a proxy for the original set of skills in a suitable hierarchically organized framework. We demonstrate the efficacy of our method on a collection of grid-worlds and on the high-dimensional Fetch-Reach robotic manipulation task by evaluating the obtained policy basis on a set of downstream tasks.

READ FULL TEXT

page 5

page 6

05/14/2019

Successor Options: An Option Discovery Framework for Reinforcement Learning

The options framework in reinforcement learning models the notion of a s...
11/21/2016

Options Discovery with Budgeted Reinforcement Learning

We consider the problem of learning hierarchical policies for Reinforcem...
06/27/2021

Unsupervised Skill Discovery with Bottleneck Option Learning

Having the ability to acquire inherent skills from environments without ...
08/22/2017

Reinforcement Learning in POMDPs with Memoryless Options and Option-Observation Initiation Sets

Many real-world reinforcement learning problems have a hierarchical natu...
07/26/2018

Variational Option Discovery Algorithms

We explore methods for option discovery based on variational inference a...
06/10/2021

Adversarial Option-Aware Hierarchical Imitation Learning

It has been a challenge to learning skills for an agent from long-horizo...
05/17/2016

Option Discovery in Hierarchical Reinforcement Learning using Spatio-Temporal Clustering

This paper introduces an automated skill acquisition framework in reinfo...