Gauges and Accelerated Optimization over Smooth and/or Strongly Convex Sets

by   Ning Liu, et al.

We consider feasibility and constrained optimization problems defined over smooth and/or strongly convex sets. These notions mirror their popular function counterparts but are much less explored in the first-order optimization literature. We propose new scalable, projection-free, accelerated first-order methods in these settings. Our methods avoid linear optimization or projection oracles, only using cheap one-dimensional linesearches and normal vector computations. Despite this, we derive optimal accelerated convergence guarantees of O(1/T) for strongly convex problems, O(1/T^2) for smooth problems, and accelerated linear convergence given both. Our algorithms and analysis are based on novel characterizations of the Minkowski gauge of smooth and/or strongly convex sets, which may be of independent interest: although the gauge is neither smooth nor strongly convex, we show the gauge squared inherits any structure present in the set.


page 1

page 2

page 3

page 4


Accelerated Stochastic Mirror Descent Algorithms For Composite Non-strongly Convex Optimization

We consider the problem of minimizing the sum of an average function of ...

Transient growth of accelerated first-order methods for strongly convex optimization problems

Optimization algorithms are increasingly being used in applications with...

Smooth Strongly Convex Regression

Convex regression (CR) is the problem of fitting a convex function to a ...

Efficient First-order Methods for Convex Optimization with Strongly Convex Function Constraints

Convex function constrained optimization has received growing research i...

Understanding Accelerated Gradient Methods: Lyapunov Analyses and Hamiltonian Assisted Interpretations

We formulate two classes of first-order algorithms more general than pre...

Online and Bandit Algorithms for Nonstationary Stochastic Saddle-Point Optimization

Saddle-point optimization problems are an important class of optimizatio...

Distributed Accelerated Projection-Based Consensus Decomposition

With the development of machine learning and Big Data, the concepts of l...

Please sign up or login with your details

Forgot password? Click here to reset