Complexity Guarantees for Polyak Steps with Momentum

02/03/2020
by   Mathieu Barré, et al.
0

In smooth strongly convex optimization, or in the presence of Hölderian error bounds, knowledge of the curvature parameter is critical for obtaining simple methods with accelerated rates. In this work, we study a class of methods, based on Polyak steps, where this knowledge is substituted by that of the optimal value, f_*. We first show slightly improved convergence bounds than previously known for the classical case of simple gradient descent with Polyak steps, we then derive an accelerated gradient method with Polyak steps and momentum, along with convergence guarantees.

READ FULL TEXT

Please sign up or login with your details

Forgot password? Click here to reset