Improved Analysis for Dynamic Regret of Strongly Convex and Smooth Functions

06/10/2020
by   Peng Zhao, et al.
0

In this paper, we present an improved analysis for dynamic regret of strongly convex and smooth functions. Specifically, we investigate the Online Multiple Gradient Descent (OMGD) algorithm proposed by Zhang et al. (2017). The original analysis shows that the dynamic regret of OMGD is at most O(min{P_T,S_T}), where P_T and S_T are path-length and squared path-length that measures the cumulative movement of minimizers of the online functions. We demonstrate that by an improved analysis, the dynamic regret of OMGD can be improved to O(min{P_T,S_T,V_T}), where V_T is the function variation of the online functions. Note that the quantities of P_T, S_T, V_T essentially reflect different aspects of environmental non-stationarity—they are not comparable in general and are favored in different scenarios. Therefore, the dynamic regret presented in this paper actually achieves a best-of-three-worlds guarantee and is strictly tighter than previous results.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/06/2020

Unconstrained Online Optimization: Dynamic Regret Analysis of Strongly Convex and Smooth Problems

The regret bound of dynamic online learning algorithms is often expresse...
research
12/26/2018

Dynamic Online Gradient Descent with Improved Query Complexity: A Theoretical Revisit

We provide a new theoretical analysis framework to investigate online gr...
research
02/11/2023

Improved Dynamic Regret for Online Frank-Wolfe

To deal with non-stationary online problems with complex constraints, we...
research
07/07/2020

Dynamic Regret of Convex and Smooth Functions

We investigate online convex optimization in non-stationary environments...
research
02/09/2023

Optimistic Online Mirror Descent for Bridging Stochastic and Adversarial Online Convex Optimization

Stochastically Extended Adversarial (SEA) model is introduced by Sachs e...
research
05/20/2023

Non-stationary Online Convex Optimization with Arbitrary Delays

Online convex optimization (OCO) with arbitrary delays, in which gradien...
research
01/07/2013

Dynamical Models and Tracking Regret in Online Convex Programming

This paper describes a new online convex optimization method which incor...

Please sign up or login with your details

Forgot password? Click here to reset