Reinforcement-Learning-based Adaptive Optimal Control for Arbitrary Reference Tracking

by   Florian Köpf, et al.

Model-free control based on the idea of Reinforcement Learning is a promising control approach that has recently gained extensive attention. However, most Reinforcement-Learning-based control methods solely focus on the regulation problem or learn to track a reference that is generated by a time-invariant exo-system. In order to overcome these limitations, we develop a new Reinforcement-Learning-based adaptive optimal control method that is able to generalize to arbitrary reference trajectories. Therefore, we propose a novel Q-function that incorporates a given reference trajectory on a moving horizon. We show that only the Q-function needs to be determined in order to solve the optimal tracking problem. The analytical solution of our Q-function provides insights into its structure and allows us to choose basis functions suited for Q-function approximation purposes. Based on that, the optimal solution to the moving horizon linear-quadratic tracking problem with arbitrary reference trajectories is learned by means of a temporal difference learning method without knowledge of the system. We furthermore prove convergence of our algorithm to the optimal Q-function as well as the optimal control law. Finally, simulation examples demonstrate the effectiveness of our developed method.


Adaptive Dynamic Programming for Model-free Tracking of Trajectories with Time-varying Parameters

In order to autonomously learn to control unknown systems optimally w.r....

A Reinforcement Learning Approach to Health Aware Control Strategy

Health-aware control (HAC) has emerged as one of the domains where contr...

Neural Network Algorithm for Intercepting Targets Moving Along Known Trajectories by a Dubins' Car

The task of intercepting a target moving along a rectilinear or circular...

Continuum Swarm Tracking Control: A Geometric Perspective in Wasserstein Space

We consider a setting in which one swarm of agents is to service or trac...

Markov Data-Based Reference Tracking of Tensegrity Morphing Airfoils

This letter presents a data-based control design for reference tracking ...

Discontinuity-Sensitive Optimal Control Learning by Mixture of Experts

This paper proposes a discontinuity-sensitive approach to learn the solu...

Trajectory Tracking of Underactuated Sea Vessels With Uncertain Dynamics: An Integral Reinforcement Learning Approach

Underactuated systems like sea vessels have degrees of motion that are i...

Please sign up or login with your details

Forgot password? Click here to reset