Dynamic Local Regret for Non-convex Online Forecasting

10/16/2019
by   Sergul Aydore, et al.
0

We consider online forecasting problems for non-convex machine learning models. Forecasting introduces several challenges such as (i) frequent updates are necessary to deal with concept drift issues since the dynamics of the environment change over time, and (ii) the state of the art models are non-convex models. We address these challenges with a novel regret framework. Standard regret measures commonly do not consider both dynamic environment and non-convex models. We introduce a local regret for non-convex models in a dynamic environment. We present an update rule incurring a cost, according to our proposed local regret, which is sublinear in time T. Our update uses time-smoothed gradients. Using a real-world dataset we show that our time-smoothed approach yields several benefits when compared with state-of-the-art competitors: results are more stable against new data; training is more robust to hyperparameter selection; and our approach is more computationally efficient than the alternatives.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
05/21/2019

Time-Smoothed Gradients for Online Forecasting

Here, we study different update rules in stochastic gradient descent (SG...
research
06/03/2022

Rate-Optimal Online Convex Optimization in Adaptive Linear Control

We consider the problem of controlling an unknown linear dynamical syste...
research
09/21/2022

Distributed Online Non-convex Optimization with Composite Regret

Regret has been widely adopted as the metric of choice for evaluating th...
research
02/20/2001

Non-convex cost functionals in boosting algorithms and methods for panel selection

In this document we propose a new improvement for boosting techniques as...
research
10/05/2015

On the Online Frank-Wolfe Algorithms for Convex and Non-convex Optimizations

In this paper, the online variants of the classical Frank-Wolfe algorith...
research
09/20/2022

MPC with Sensor-Based Online Cost Adaptation

Model predictive control is a powerful tool to generate complex motions ...
research
09/13/2021

Zeroth-order non-convex learning via hierarchical dual averaging

We propose a hierarchical version of dual averaging for zeroth-order onl...

Please sign up or login with your details

Forgot password? Click here to reset