Single Time-scale Actor-critic Method to Solve the Linear Quadratic Regulator with Convergence Guarantees

01/31/2022

∙

by   Mo Zhou, et al.

∙

5

∙

share

We propose a single time-scale actor-critic algorithm to solve the linear quadratic regulator (LQR) problem. A least squares temporal difference (LSTD) method is applied to the critic and a natural policy gradient method is used for the actor. We give a proof of convergence with sample complexity (^-1log(^-1)^2). The method in the proof is applicable to general single time-scale bilevel optimization problem. We also numerically validate our theoretical results on the convergence.