Control Frequency Adaptation via Action Persistence in Batch Reinforcement Learning

by   Alberto Maria Metelli, et al.

The choice of the control frequency of a system has a relevant impact on the ability of reinforcement learning algorithms to learn a highly performing policy. In this paper, we introduce the notion of action persistence that consists in the repetition of an action for a fixed number of decision steps, having the effect of modifying the control frequency. We start analyzing how action persistence affects the performance of the optimal policy, and then we present a novel algorithm, Persistent Fitted Q-Iteration (PFQI), that extends FQI, with the goal of learning the optimal value function at a given persistence. After having provided a theoretical study of PFQI and a heuristic approach to identify the optimal persistence, we present an experimental campaign on benchmark domains to show the advantages of action persistence and proving the effectiveness of our persistence selection method.



There are no comments yet.


page 1

page 2

page 3

page 4


Persistence-perfect discrete gradient vector fields and multi-parameter persistence

The main objective of this paper is to introduce and study a notion of p...

Notes on pivot pairings

We present a row reduction algorithm to compute the barcode decompositio...

A comment on stabilizing reinforcement learning

This is a short comment on the paper "Asymptotically Stable Adaptive-Opt...

On the Persistence of Persistent Identifiers of the Scholarly Web

Scholarly resources, just like any other resources on the web, are subje...

Multidimensional Persistence Module Classification via Lattice-Theoretic Convolutions

Multiparameter persistent homology has been largely neglected as an inpu...

Possibilistic decreasing persistence

A key issue in the handling of temporal data is the treatment of persist...

Frequency-based Search-control in Dyna

Model-based reinforcement learning has been empirically demonstrated as ...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.