Learning Without Mixing: Towards A Sharp Analysis of Linear System Identification

02/22/2018
by   Max Simchowitz, et al.
0

We prove that the ordinary least-squares (OLS) estimator attains nearly minimax optimal performance for the identification of linear dynamical systems from a single observed trajectory. Our upper bound relies on a generalization of Mendelson's small-ball method to dependent data, eschewing the use of standard mixing-time arguments. Our lower bounds reveal that these upper bounds match up to logarithmic factors. In particular, we capture the correct signal-to-noise behavior of the problem, showing that more unstable linear systems are easier to estimate. This behavior is qualitatively different from arguments which rely on mixing-time calculations that suggest that unstable systems are more difficult to estimate. We generalize our technique to provide bounds for a more general class of linear response time-series.

READ FULL TEXT

page 1

page 2

page 3

page 4

02/03/2012

Minimax Rates of Estimation for Sparse PCA in High Dimensions

We study sparse principal components analysis in the high-dimensional se...
06/03/2011

Rademacher complexity of stationary sequences

We show how to control the generalization error of time series models wh...
12/23/2019

Exact minimax risk for linear least squares, and the lower tail of sample covariance matrices

The first part of this paper is devoted to the decision-theoretic analys...
06/16/2022

Learning with little mixing

We study square loss in a realizable time-series framework with martinga...
10/17/2019

Sharper bounds for uniformly stable algorithms

The generalization bounds for stable algorithms is a classical question ...
03/17/2020

Finite-time Identification of Stable Linear Systems: Optimality of the Least-Squares Estimator

We provide a new finite-time analysis of the estimation error of stable ...
06/17/2021

Minimax Estimation of Partially-Observed Vector AutoRegressions

To understand the behavior of large dynamical systems like transportatio...