Ternary Policy Iteration Algorithm for Nonlinear Robust Control

07/14/2020
by   Jie Li, et al.
0

The uncertainties in plant dynamics remain a challenge for nonlinear control problems. This paper develops a ternary policy iteration (TPI) algorithm for solving nonlinear robust control problems with bounded uncertainties. The controller and uncertainty of the system are considered as game players, and the robust control problem is formulated as a two-player zero-sum differential game. In order to solve the differential game, the corresponding Hamilton-Jacobi-Isaacs (HJI) equation is then derived. Three loss functions and three update phases are designed to match the identity equation, minimization and maximization of the HJI equation, respectively. These loss functions are defined by the expectation of the approximate Hamiltonian in a generated state set to prevent operating all the states in the entire state set concurrently. The parameters of value function and policies are directly updated by diminishing the designed loss functions using the gradient descent method. Moreover, zero-initialization can be applied to the parameters of the control policy. The effectiveness of the proposed TPI algorithm is demonstrated through two simulation studies. The simulation results show that the TPI algorithm can converge to the optimal solution for the linear plant, and has high resistance to disturbances for the nonlinear plant.

READ FULL TEXT
research
10/05/2021

Continuous-Time Fitted Value Iteration for Robust Policies

Solving the Hamilton-Jacobi-Bellman equation is important in many domain...
research
11/24/2013

Off-policy reinforcement learning for H_∞ control design

The H_∞ control design problem is considered for nonlinear systems with ...
research
11/28/2020

Approximate Midpoint Policy Iteration for Linear Quadratic Control

We present a midpoint policy iteration algorithm to solve linear quadrat...
research
11/26/2019

Deep adaptive dynamic programming for nonaffine nonlinear optimal control problem with state constraints

This paper presents a constrained deep adaptive dynamic programming (CDA...
research
08/28/2019

Networked Control of Nonlinear Systems under Partial Observation Using Continuous Deep Q-Learning

In this paper, we propose a design of a model-free networked controller ...
research
05/25/2021

Robust Value Iteration for Continuous Control Tasks

When transferring a control policy from simulation to a physical system,...
research
12/28/2020

Analytical and numerical solutions to ergodic control problems arising in environmental management

Environmental management should be based on a long-run sustainable viewp...

Please sign up or login with your details

Forgot password? Click here to reset