Boosting First-order Methods by Shifting Objective: New Schemes with Faster Worst Case Rates

05/25/2020 ∙ by Kaiwen Zhou, et al. ∙ 20

We propose a new methodology to design first-order methods for unconstrained strongly convex problems, i.e., to design for a shifted objective function. Several technical lemmas are provided as the building blocks for designing new methods. By shifting objective, the analysis is tightened, which leaves space for faster rates, and also simplified. Following this methodology, we derived several new accelerated schemes for problems that equipped with various first-order oracles, and all of the derived methods have faster worst case convergence rates than their existing counterparts. Experiments on machine learning tasks are conducted to evaluate the new methods.



There are no comments yet.


page 1

page 2

page 3

page 4

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.