Explore More and Improve Regret in Linear Quadratic Regulators

by   Sahin Lale, et al.

Stabilizing the unknown dynamics of a control system and minimizing regret in control of an unknown system are among the main goals in control theory and reinforcement learning. In this work, we pursue both these goals for adaptive control of linear quadratic regulators (LQR). Prior works accomplish either one of these goals at the cost of the other one. The algorithms that are guaranteed to find a stabilizing controller suffer from high regret, whereas algorithms that focus on achieving low regret assume the presence of a stabilizing controller at the early stages of agent-environment interaction. In the absence of such a stabilizing controller, at the early stages, the lack of reasonable model estimates needed for (i) strategic exploration and (ii) design of controllers that stabilize the system, results in regret that scales exponentially in the problem dimensions. We propose a framework for adaptive control that exploits the characteristics of linear dynamical systems and deploys additional exploration in the early stages of agent-environment interaction to guarantee sooner design of stabilizing controllers. We show that for the classes of controllable and stabilizable LQRs, where the latter is a generalization of prior work, these methods achieve 𝒊Ėƒ(√(T)) regret with a polynomial dependence in the problem dimensions.


page 1

page 2

page 3

page 4

∙ 03/25/2020

Logarithmic Regret Bound in Partially Observable Linear Dynamical Systems

We study the problem of adaptive control in partially observable linear ...
∙ 06/17/2022

Thompson Sampling Achieves Õ(√(T)) Regret in Linear Quadratic Control

Thompson Sampling (TS) is an efficient method for decision-making under ...
∙ 08/26/2021

Finite-time System Identification and Adaptive Control in Autoregressive Exogenous Systems

Autoregressive exogenous (ARX) systems are the general class of input-ou...
∙ 10/25/2020

Geometric Exploration for Online Control

We study the control of an unknown linear dynamical system under general...
∙ 01/13/2023

Almost Surely √(T) Regret Bound for Adaptive LQR

The Linear-Quadratic Regulation (LQR) problem with unknown system parame...
∙ 03/24/2013

Efficient Reinforcement Learning for High Dimensional Linear Quadratic Systems

We study the problem of adaptive control of a high dimensional linear qu...
∙ 02/24/2020

Robust Learning-Based Control via Bootstrapped Multiplicative Noise

Despite decades of research and recent progress in adaptive control and ...

Please sign up or login with your details

Forgot password? Click here to reset