Adaptive Dynamic Programming for Model-free Tracking of Trajectories with Time-varying Parameters

by   Florian Köpf, et al.

In order to autonomously learn to control unknown systems optimally w.r.t. an objective function, Adaptive Dynamic Programming (ADP) is well-suited to adapt controllers based on experience from interaction with the system. In recent years, many researchers focused on the tracking case, where the aim is to follow a desired trajectory. So far, ADP tracking controllers assume that the reference trajectory follows time-invariant exo-system dynamics-an assumption that does not hold for many applications. In order to overcome this limitation, we propose a new Q-function which explicitly incorporates a parametrized approximation of the reference trajectory. This allows to learn to track a general class of trajectories by means of ADP. Once our Q-function has been learned, the associated controller copes with time-varying reference trajectories without need of further training and independent of exo-system dynamics. After proposing our general model-free off-policy tracking method, we provide analysis of the important special case of linear quadratic tracking. We conclude our paper with an example which demonstrates that our new method successfully learns the optimal tracking controller and outperforms existing approaches in terms of tracking error and cost.


page 1

page 2

page 3

page 4


Adaptive Optimal Trajectory Tracking Control Applied to a Large-Scale Ball-on-Plate System

While many theoretical works concerning Adaptive Dynamic Programming (AD...

Reinforcement-Learning-based Adaptive Optimal Control for Arbitrary Reference Tracking

Model-free control based on the idea of Reinforcement Learning is a prom...

Time-Varying Quasi-Closed-Phase Analysis for Accurate Formant Tracking in Speech Signals

In this paper, we propose a new method for the accurate estimation and t...

Transfer Learning for High-Precision Trajectory Tracking Through L_1 Adaptive Feedback and Iterative Learning

Robust and adaptive control strategies are needed when robots or automat...

Deep Neural Networks for Improved, Impromptu Trajectory Tracking of Quadrotors

Trajectory tracking control for quadrotors is important for applications...

Model-free tracking control of complex dynamical trajectories with machine learning

Nonlinear tracking control enabling a dynamical system to track a desire...

On the Controllers Based on Time Delay Estimation for Robotic Manipulators

Assurance of asymptotic trajectory tracking in robotic manipulators with...

Please sign up or login with your details

Forgot password? Click here to reset