DeepAI AI Chat
Log In Sign Up

The Sample Complexities of Global Lipschitz Optimization

by   François Bachoc, et al.

We study the problem of black-box optimization of a Lipschitz function f defined on a compact subset X of R^d, both via algorithms that certify the accuracy of their recommendations and those that do not. We investigate their sample complexities, i.e., the number of samples needed to either reach or certify a given accuracy epsilon. We start by proving a tighter bound for the well-known DOO algorithm [Perevozchikov, 1990, Munos, 2011] that matches the best existing upper bounds for (more computationally challenging) non-certified algorithms. We then introduce and analyze a new certified version of DOO and prove a matching f-dependent lower bound (up to logarithmic terms) for all certified algorithms. Afterwards, we show that this optimal quantity is proportional to ∫_X dx/(max(f) - f(x) + epsilon)^d, solving as a corollary a three-decade-old conjecture by Hansen et al. [1991]. Finally, we show how to control the sample complexity of state-of-the-art non-certified algorithms with an integral reminiscent of the Dudley-entropy integral.


page 1

page 2

page 3

page 4


Almost Optimal Variance-Constrained Best Arm Identification

We design and analyze VA-LUCB, a parameter-free algorithm, for identifyi...

The (1+1) Elitist Black-Box Complexity of LeadingOnes

One important goal of black-box complexity theory is the development of ...

Õptimal Differentially Private Learning of Thresholds and Quasi-Concave Optimization

The problem of learning threshold functions is a fundamental one in mach...

Measuring Quantum Entropy

The entropy of a quantum system is a measure of its randomness, and has ...

On the Approximation Ratio of the k-Opt and Lin-Kernighan Algorithm for Metric TSP

The k-Opt and Lin-Kernighan algorithm are two of the most important loca...

Sequential Probability Assignment with Binary Alphabets and Large Classes of Experts

We analyze the problem of sequential probability assignment for binary o...

Robustifying Algorithms of Learning Latent Trees with Vector Variables

We consider learning the structures of Gaussian latent tree models with ...