DeepAI AI Chat
Log In Sign Up

Options of Interest: Temporal Abstraction with Interest Functions

by   Khimya Khetarpal, et al.
Montréal Institute of Learning Algorithms
Stanford University
McGill University

Temporal abstraction refers to the ability of an agent to use behaviours of controllers which act for a limited, variable amount of time. The options framework describes such behaviours as consisting of a subset of states in which they can initiate, an internal policy and a stochastic termination condition. However, much of the subsequent work on option discovery has ignored the initiation set, because of difficulty in learning it from data. We provide a generalization of initiation sets suitable for general function approximation, by defining an interest function associated with an option. We derive a gradient-based learning algorithm for interest functions, leading to a new interest-option-critic architecture. We investigate how interest functions can be leveraged to learn interpretable and reusable temporal abstractions. We demonstrate the efficacy of the proposed approach through quantitative and qualitative results, in both discrete and continuous environments.


page 4

page 5

page 6

page 7


The Option-Critic Architecture

Temporal abstraction is key to scaling up learning and planning in reinf...

Data-efficient Hindsight Off-policy Option Learning

Solutions to most complex tasks can be decomposed into simpler, intermed...

Diversity-Enriched Option-Critic

Temporal abstraction allows reinforcement learning agents to represent k...

The Termination Critic

In this work, we consider the problem of autonomously discovering behavi...

Variable-Decision Frequency Option Critic

In classic reinforcement learning algorithms, agents make decisions at d...

Natural Option Critic

The recently proposed option-critic architecture Bacon et al. provide a ...

Variational Intrinsic Control

In this paper we introduce a new unsupervised reinforcement learning met...

Code Repositories


Options of Interest: Temporal Abstraction with Interest Functions AAAI 2020

view repo