Model-free two-step design for improving transient learning performance in nonlinear optimal regulator problems

by   Yuka Masumoto, et al.

Reinforcement learning (RL) provides a model-free approach to designing an optimal controller for nonlinear dynamical systems. However, the learning process requires a considerable number of trial-and-error experiments using the poorly controlled system, and accumulates wear and tear on the plant. Thus, it is desirable to maintain some degree of control performance during the learning process. In this paper, we propose a model-free two-step design approach to improve the transient learning performance of RL in an optimal regulator design problem for unknown nonlinear systems. Specifically, a linear control law pre-designed in a model-free manner is used in parallel with online RL to ensure a certain level of performance at the early stage of learning. Numerical simulations show that the proposed method improves the transient learning performance and efficiency in hyperparameter tuning of RL.



There are no comments yet.


page 6


Model-Free μ Synthesis via Adversarial Reinforcement Learning

Motivated by the recent empirical success of policy-based reinforcement ...

Bridging Model-based Safety and Model-free Reinforcement Learning through System Identification of Low Dimensional Linear Models

Bridging model-based safety and model-free reinforcement learning (RL) f...

Who Are the Best Adopters? User Selection Model for Free Trial Item Promotion

With the increasingly fierce market competition, offering a free trial h...

Model-Free Optimal Control of Linear Multi-Agent Systems via Decomposition and Hierarchical Approximation

Designing the optimal linear quadratic regulator (LQR) for a large-scale...

Hybrid Learning for Orchestrating Deep Learning Inference in Multi-user Edge-cloud Networks

Deep-learning-based intelligent services have become prevalent in cyber-...

Lyapunov-Regularized Reinforcement Learning for Power System Transient Stability

Transient stability of power systems is becoming increasingly important ...

Model-Free Control of Dynamical Systems with Deep Reservoir Computing

We propose and demonstrate a nonlinear control method that can be applie...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.