Online Forecasting of Total-Variation-bounded Sequences

06/08/2019
by   Dheeraj Baby, et al.
5

We consider the problem of online forecasting of sequences of length n with total-variation at most C_n using observations contaminated by independent σ-subgaussian noise. We design an O(n n)-time algorithm that achieves a cumulative square error of Õ(n^1/3C_n^2/3σ^4/3) with high probability. The result is rate-optimal as it matches the known minimax rate for the offline nonparametric estimation of the same class [Mammen and van de Geer, 1997]. To the best of our knowledge, this is the first polynomial-time algorithm that optimally forecasts total variation bounded sequences. Our proof techniques leverage the special localized structure of Haar wavelet basis and adaptivity to unknown smoothness parameter in the classical wavelet smoothing [Donoho et al., 1998]. We also compare our model to the rich literature of dynamic regret minimization and nonstationary stochastic optimization, where our problem can be treated as a special case. We show that the workhorse in those settings --- online gradient descent and its variants with a fixed restarting schedule --- are instances of a class of linear forecasters that require a suboptimal regret of Ω̃(√(n)). This implies that the use of more adaptive algorithms are necessary to obtain the optimal rate.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
01/23/2021

An Optimal Reduction of TV-Denoising to Adaptive Online Learning

We consider the problem of estimating a function from n noisy samples wh...
research
09/30/2020

Adaptive Online Estimation of Piecewise Polynomial Trends

We consider the framework of non-stationary stochastic optimization [Bes...
research
06/18/2022

Optimal Dynamic Regret in LQR Control

We consider the problem of nonstochastic control with a sequence of quad...
research
05/26/2016

Total Variation Classes Beyond 1d: Minimax Rates, and the Limitations of Linear Smoothers

We consider the problem of estimating a function defined over n location...
research
11/13/2020

Non-stationary Online Regression

Online forecasting under a changing environment has been a problem of in...
research
02/23/2023

Beyond Moments: Robustly Learning Affine Transformations with Asymptotically Optimal Error

We present a polynomial-time algorithm for robustly learning an unknown ...
research
05/21/2019

Total variation multiscale estimators for linear inverse problems

Even though the statistical theory of linear inverse problems is a well-...

Please sign up or login with your details

Forgot password? Click here to reset