DeepAI AI Chat
Log In Sign Up

Lower Bounds for Parallel and Randomized Convex Optimization

by   Jelena Diakonikolas, et al.
berkeley college
Pontificia Universidad Católica de Chile

We study the question of whether parallelization in the exploration of the feasible set can be used to speed up convex optimization, in the local oracle model of computation. We show that the answer is negative for both deterministic and randomized algorithms applied to essentially any of the interesting geometries and nonsmooth, weakly-smooth, or smooth objective functions. In particular, we show that it is not possible to obtain a polylogarithmic (in the sequential complexity of the problem) number of parallel rounds with a polynomial (in the dimension) number of queries per round. In the majority of these settings and when the dimension of the space is polynomial in the inverse target accuracy, our lower bounds match the oracle complexity of sequential convex optimization, up to at most a logarithmic factor in the dimension, which makes them (nearly) tight. Prior to our work, lower bounds for parallel convex optimization algorithms were only known in a small fraction of the settings considered in this paper, mainly applying to Euclidean (ℓ_2) and ℓ_∞ spaces. It is unclear whether the arguments used in this prior work can be extended to general ℓ_p spaces. Hence, our work provides a more general approach for proving lower bounds in the setting of parallel convex optimization. Moreover, as a consequence of our proof techniques, we obtain new anti-concentration bounds for convex combinations of Rademacher sequences that may be of independent interest.


page 1

page 2

page 3

page 4


Complexity of Highly Parallel Non-Smooth Convex Optimization

A landmark result of non-smooth convex optimization is that gradient des...

Optimal Methods for Higher-Order Smooth Monotone Variational Inequalities

In this work, we present new simple and optimal algorithms for solving t...

Majorizing Measures for the Optimizer

The theory of majorizing measures, extensively developed by Fernique, Ta...

Tight Query Complexity Lower Bounds for PCA via Finite Sample Deformed Wigner Law

We prove a query complexity lower bound for approximating the top r dime...

Analysis of Work-Stealing and Parallel Cache Complexity

Parallelism has become extremely popular over the past decade, and there...