Local Minimax Complexity of Stochastic Convex Optimization

05/24/2016
by   Yuancheng Zhu, et al.
0

We extend the traditional worst-case, minimax analysis of stochastic convex optimization by introducing a localized form of minimax complexity for individual functions. Our main result gives function-specific lower and upper bounds on the number of stochastic subgradient evaluations needed to optimize either the function or its "hardest local alternative" to a given numerical precision. The bounds are expressed in terms of a localized and computational analogue of the modulus of continuity that is central to statistical minimax analysis. We show how the computational modulus of continuity can be explicitly calculated in concrete cases, and relates to the curvature of the function at the optimum. We also prove a superefficiency result that demonstrates it is a meaningful benchmark, acting as a computational analogue of the Fisher information in statistical estimation. The nature and practical implications of the results are demonstrated in simulations.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

09/03/2010

Information-theoretic lower bounds on the oracle complexity of stochastic convex optimization

Relative to the large literature on upper bounds on complexity of convex...
03/22/2018

Optimization of Smooth Functions with Noisy Observations: Local Minimax Rates

We consider the problem of global optimization of an unknown non-convex ...
06/14/2018

The Right Complexity Measure in Locally Private Estimation: It is not the Fisher Information

We identify fundamental tradeoffs between statistical utility and privac...
08/05/2021

Adapting to Function Difficulty and Growth Conditions in Private Optimization

We develop algorithms for private stochastic convex optimization that ad...
12/25/2013

A Convex Formulation for Mixed Regression with Two Components: Minimax Optimal Rates

We consider the mixed regression problem with two components, under adve...
04/01/2018

Near-Optimality Recovery of Linear and N-Convex Functions on Unions of Convex Sets

In this paper, following the line of research on "statistical inference ...
04/01/2018

Near-Optimal Recovery of Linear and N-Convex Functions on Unions of Convex Sets

In this paper, following the line of research on "statistical inference ...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.