An experimental study of two predictive reinforcement learning methods and comparison with model-predictive control

08/10/2021
by   Dmitrii Dobriborsci, et al.
0

Reinforcement learning (RL) has been successfully used in various simulations and computer games. Industry-related applications, such as autonomous mobile robot motion control, are somewhat challenging for RL up to date though. This paper presents an experimental evaluation of predictive RL controllers for optimal mobile robot motion control. As a baseline for comparison, model-predictive control (MPC) is used. Two RL methods are tested: a roll-out Q-learning, which may be considered as MPC with terminal cost being a Q-function approximation, and a so-called stacked Q-learning, which in turn is like MPC with the running cost substituted for a Q-function approximation. The experimental foundation is a mobile robot with a differential drive (Robotis Turtlebot3). Experimental results showed that both RL methods beat the baseline in terms of the accumulated cost, whereas the stacked variant performed best. Provided the series of previous works on stacked Q-learning, this particular study supports the idea that MPC with a running cost adaptation inspired by Q-learning possesses potential of performance boost while retaining the nice properties of MPC.

READ FULL TEXT
research
08/23/2021

A generalized stacked reinforcement learning method for sampled systems

A common setting of reinforcement learning (RL) is a Markov decision pro...
research
03/06/2020

Practical Reinforcement Learning For MPC: Learning from sparse objectives in under an hour on a real robot

Model Predictive Control (MPC) is a powerful control technique that hand...
research
02/27/2020

Reinforcement Learning Based Compensation Methods for Robot Manipulators

Smart robotics will be a core feature while migrating from Industry 3.0 ...
research
09/15/2021

Infusing model predictive control into meta-reinforcement learning for mobile robots in dynamic environments

The successful operation of mobile robots requires them to rapidly adapt...
research
05/23/2022

Model Predictive Control of Non-Holonomic Vehicles: Beyond Differential-Drive

Non-holonomic vehicles are of immense practical value and increasingly s...
research
10/27/2017

Declarative vs Rule-based Control for Flocking Dynamics

The popularity of rule-based flocking models, such as Reynolds' classic ...
research
08/29/2023

On the improvement of model-predictive controllers

This article investigates synthetic model-predictive control (MPC) probl...

Please sign up or login with your details

Forgot password? Click here to reset