DeepAI AI Chat
Log In Sign Up

Efficient Reinforcement Learning for High Dimensional Linear Quadratic Systems

by   Morteza Ibrahimi, et al.
Stanford University

We study the problem of adaptive control of a high dimensional linear quadratic (LQ) system. Previous work established the asymptotic convergence to an optimal controller for various adaptive control schemes. More recently, for the average cost LQ problem, a regret bound of O(√(T)) was shown, apart form logarithmic factors. However, this bound scales exponentially with p, the dimension of the state space. In this work we consider the case where the matrices describing the dynamic of the LQ system are sparse and their dimensions are large. We present an adaptive control scheme that achieves a regret bound of O(p √(T)), apart from logarithmic factors. In particular, our algorithm has an average cost of (1+) times the optimum cost after T = (p) O(1/^2). This is in comparison to previous work on the dense dynamics where the algorithm requires time that scales exponentially with dimension in order to achieve regret of times the optimal cost. We believe that our result has prominent applications in the emerging area of computational advertising, in particular targeted online advertising and advertising in social networks.


page 1

page 2

page 3

page 4


Regret Bound of Adaptive Control in Linear Quadratic Gaussian (LQG) Systems

We study the problem of adaptive control in partially observable linear ...

Logarithmic Regret for Online Control

We study optimal regret bounds for control in linear dynamical systems u...

Pseudonorm Approachability and Applications to Regret Minimization

Blackwell's celebrated approachability theory provides a general framewo...

Towards a Dimension-Free Understanding of Adaptive Linear Control

We study the problem of adaptive control of the linear quadratic regulat...

Thompson Sampling Achieves Õ(√(T)) Regret in Linear Quadratic Control

Thompson Sampling (TS) is an efficient method for decision-making under ...

Explore More and Improve Regret in Linear Quadratic Regulators

Stabilizing the unknown dynamics of a control system and minimizing regr...

Finite Time Analysis of Optimal Adaptive Policies for Linear-Quadratic Systems

We consider the classical problem of control of linear systems with quad...