A framework for reinforcement learning with autocorrelated actions

09/10/2020

∙

The subject of this paper is reinforcement learning. Policies are considered here that produce actions based on states and random elements autocorrelated in subsequent time instants. Consequently, an agent learns from experiments that are distributed over time and potentially give better clues to policy improvement. Also, physical implementation of such policies, e.g. in robotics, is less problematic, as it avoids making robots shake. This is in opposition to most RL algorithms which add white noise to control causing unwanted shaking of the robots. An algorithm is introduced here that approximately optimizes the aforementioned policy. Its efficiency is verified for four simulated learning control problems (Ant, HalfCheetah, Hopper, and Walker2D) against three other methods (PPO, SAC, ACER). The algorithm outperforms others in three of these problems.

READ FULL TEXT

A framework for reinforcement learning with autocorrelated actions

ACERAC: Efficient reinforcement learning in fine time discretization

Control with adaptive Q-learning

Learning to Play Soccer by Reinforcement and Applying Sim-to-Real to Compete in the Real World

Robust Predictable Control

Learning Task-Driven Control Policies via Information Bottlenecks

Composable Energy Policies for Reactive Motion Generation and Reinforcement Learning

Moral reinforcement learning using actual causation

A framework for reinforcement learning with autocorrelated actions

Related Research

ACERAC: Efficient reinforcement learning in fine time discretization

Control with adaptive Q-learning

Learning to Play Soccer by Reinforcement and Applying Sim-to-Real to Compete in the Real World

Robust Predictable Control

Learning Task-Driven Control Policies via Information Bottlenecks

Composable Energy Policies for Reactive Motion Generation and Reinforcement Learning

Moral reinforcement learning using actual causation