Learning Routines for Effective Off-Policy Reinforcement Learning

06/05/2021
by   Edoardo Cetin, et al.
7

The performance of reinforcement learning depends upon designing an appropriate action space, where the effect of each action is measurable, yet, granular enough to permit flexible behavior. So far, this process involved non-trivial user choices in terms of the available actions and their execution frequency. We propose a novel framework for reinforcement learning that effectively lifts such constraints. Within our framework, agents learn effective behavior over a routine space: a new, higher-level action space, where each routine represents a set of 'equivalent' sequences of granular actions with arbitrary length. Our routine space is learned end-to-end to facilitate the accomplishment of underlying off-policy reinforcement learning objectives. We apply our framework to two state-of-the-art off-policy algorithms and show that the resulting agents obtain relevant performance improvements while requiring fewer interactions with the environment per episode, improving computational efficiency.

READ FULL TEXT

page 8

page 15

page 16

page 17

page 18

research
10/23/2018

Hierarchical Approaches for Reinforcement Learning in Parameterized Action Space

We explore Deep Reinforcement Learning in a parameterized action space. ...
research
06/07/2015

A Framework for Constrained and Adaptive Behavior-Based Agents

Behavior Trees are commonly used to model agents for robotics and games,...
research
02/19/2022

A Regularized Implicit Policy for Offline Reinforcement Learning

Offline reinforcement learning enables learning from a fixed dataset, wi...
research
09/20/2022

Towards Task-Prioritized Policy Composition

Combining learned policies in a prioritized, ordered manner is desirable...
research
08/09/2022

Generalized Reinforcement Learning: Experience Particles, Action Operator, Reinforcement Field, Memory Association, and Decision Concepts

Learning a control policy that involves time-varying and evolving system...
research
03/29/2021

LASER: Learning a Latent Action Space for Efficient Reinforcement Learning

The process of learning a manipulation task depends strongly on the acti...
research
08/08/2019

Incremental Reinforcement Learning --- a New Continuous Reinforcement Learning Frame Based on Stochastic Differential Equation methods

Continuous reinforcement learning such as DDPG and A3C are widely used i...

Please sign up or login with your details

Forgot password? Click here to reset