Learning Linear Dynamical Systems with Semi-Parametric Least Squares

02/02/2019
by   Max Simchowitz, et al.
20

We analyze a simple prefiltered variation of the least squares estimator for the problem of estimation with biased, semi-parametric noise, an error model studied more broadly in causal statistics and active learning. We prove an oracle inequality which demonstrates that this procedure provably mitigates the variance introduced by long-term dependencies. We then demonstrate that prefiltered least squares yields, to our knowledge, the first algorithm that provably estimates the parameters of partially-observed linear systems that attains rates which do not not incur a worst-case dependence on the rate at which these dependencies decay. The algorithm is provably consistent even for systems which satisfy the weaker marginal stability condition obeyed by many classical models based on Newtonian mechanics. In this context, our semi-parametric framework yields guarantees for both stochastic and worst-case noise.

READ FULL TEXT
research
02/02/2020

Active Learning for Identification of Linear Dynamical Systems

We propose an algorithm to actively estimate the parameters of a linear ...
research
09/04/2023

Challenges of the inconsistency regime: Novel debiasing methods for missing data models

We study semi-parametric estimation of the population mean when data is ...
research
02/04/2019

Minimax experimental design: Bridging the gap between statistical and worst-case approaches to least squares regression

In experimental design, we are given a large collection of vectors, each...
research
01/03/2013

Follow the Leader If You Can, Hedge If You Must

Follow-the-Leader (FTL) is an intuitive sequential prediction strategy t...
research
08/02/2023

Compressed and distributed least-squares regression: convergence rates with applications to Federated Learning

In this paper, we investigate the impact of compression on stochastic gr...
research
07/11/2023

Non-parametric estimation of the reaction term in semi-linear SPDEs with spatial ergodicity

The non-parametric estimation of a non-linear reaction term in a semi-li...
research
03/31/2022

Learning from many trajectories

We initiate a study of supervised learning from many independent sequenc...

Please sign up or login with your details

Forgot password? Click here to reset