Explore More and Improve Regret in Linear Quadratic Regulators

07/23/2020
∙
by   Sahin Lale, et al.
∙
27
∙

Stabilizing the unknown dynamics of a control system and minimizing regret in control of an unknown system are among the main goals in control theory and reinforcement learning. In this work, we pursue both these goals for adaptive control of linear quadratic regulators (LQR). Prior works accomplish either one of these goals at the cost of the other one. The algorithms that are guaranteed to find a stabilizing controller suffer from high regret, whereas algorithms that focus on achieving low regret assume the presence of a stabilizing controller at the early stages of agent-environment interaction. In the absence of such a stabilizing controller, at the early stages, the lack of reasonable model estimates needed for (i) strategic exploration and (ii) design of controllers that stabilize the system, results in regret that scales exponentially in the problem dimensions. We propose a framework for adaptive control that exploits the characteristics of linear dynamical systems and deploys additional exploration in the early stages of agent-environment interaction to guarantee sooner design of stabilizing controllers. We show that for the classes of controllable and stabilizable LQRs, where the latter is a generalization of prior work, these methods achieve 𝒊Ėƒ(√(T)) regret with a polynomial dependence in the problem dimensions.

READ FULL TEXT

Authors

page 1

page 2

page 3

page 4

∙ 03/25/2020

Logarithmic Regret Bound in Partially Observable Linear Dynamical Systems

We study the problem of adaptive control in partially observable linear ...
∙ 06/17/2022

Thompson Sampling Achieves Õ(√(T)) Regret in Linear Quadratic Control

Thompson Sampling (TS) is an efficient method for decision-making under ...
∙ 08/26/2021

Finite-time System Identification and Adaptive Control in Autoregressive Exogenous Systems

Autoregressive exogenous (ARX) systems are the general class of input-ou...
∙ 10/20/2020

Regret-optimal control in dynamic environments

We consider the control of linear time-varying dynamical systems from th...
∙ 03/24/2013

Efficient Reinforcement Learning for High Dimensional Linear Quadratic Systems

We study the problem of adaptive control of a high dimensional linear qu...
∙ 02/24/2020

Robust Learning-Based Control via Bootstrapped Multiplicative Noise

Despite decades of research and recent progress in adaptive control and ...
∙ 05/31/2020

Adaptive Digital PID Control of a Quadcopter with Unknown Dynamics

This paper develops an adaptive autopilot for quadcopters with unknown d...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.