Control Frequency Adaptation via Action Persistence in Batch Reinforcement Learning

02/17/2020
by   Alberto Maria Metelli, et al.
0

The choice of the control frequency of a system has a relevant impact on the ability of reinforcement learning algorithms to learn a highly performing policy. In this paper, we introduce the notion of action persistence that consists in the repetition of an action for a fixed number of decision steps, having the effect of modifying the control frequency. We start analyzing how action persistence affects the performance of the optimal policy, and then we present a novel algorithm, Persistent Fitted Q-Iteration (PFQI), that extends FQI, with the goal of learning the optimal value function at a given persistence. After having provided a theoretical study of PFQI and a heuristic approach to identify the optimal persistence, we present an experimental campaign on benchmark domains to show the advantages of action persistence and proving the effectiveness of our persistence selection method.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

04/10/2019

Persistence-perfect discrete gradient vector fields and multi-parameter persistence

The main objective of this paper is to introduce and study a notion of p...
01/02/2021

Notes on pivot pairings

We present a row reduction algorithm to compute the barcode decompositio...
11/24/2021

A comment on stabilizing reinforcement learning

This is a short comment on the paper "Asymptotically Stable Adaptive-Opt...
04/06/2020

On the Persistence of Persistent Identifiers of the Scholarly Web

Scholarly resources, just like any other resources on the web, are subje...
11/28/2020

Multidimensional Persistence Module Classification via Lattice-Theoretic Convolutions

Multiparameter persistent homology has been largely neglected as an inpu...
03/06/2013

Possibilistic decreasing persistence

A key issue in the handling of temporal data is the treatment of persist...
02/14/2020

Frequency-based Search-control in Dyna

Model-based reinforcement learning has been empirically demonstrated as ...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.