The (1+1)-ES Reliably Overcomes Saddle Points

12/01/2021
by   Tobias Glasmachers, et al.
0

It is known that step size adaptive evolution strategies (ES) do not converge (prematurely) to regular points of continuously differentiable objective functions. Among critical points, convergence to minima is desired, and convergence to maxima is easy to exclude. However, surprisingly little is known on whether ES can get stuck at a saddle point. In this work we establish that even the simple (1+1)-ES reliably overcomes most saddle points under quite mild regularity conditions. Our analysis is based on drift with tail bounds. It is non-standard in that we do not even aim to estimate hitting times based on drift. Rather, in our case it suffices to show that the relevant time is finite with full probability.

READ FULL TEXT
research
11/15/2010

Non-Existence of Linear Universal Drift Functions

Drift analysis has become a powerful tool to prove bounds on the runtime...
research
06/09/2017

Global Convergence of the (1+1) Evolution Strategy

We establish global convergence of the (1+1)-ES algorithm, i.e., converg...
research
06/26/2020

Well-posedness and numerical schemes for McKean-Vlasov equations and interacting particle systems with discontinuous drift

In this paper, we first establish well-posedness results of McKean-Vlaso...
research
01/06/2022

On the geometric convergence for MALA under verifiable conditions

While the Metropolis Adjusted Langevin Algorithm (MALA) is a popular and...
research
11/05/2018

Non-ergodic Convergence Analysis of Heavy-Ball Algorithms

In this paper, we revisit the convergence of the Heavy-ball method, and ...
research
04/24/2023

Asymptotics of large deviations of finite difference method for stochastic Cahn–Hilliard equation

In this work, we establish the Freidlin–Wentzell large deviations princi...
research
01/22/2020

Chirotopes of Random Points in Space are Realizable on a Small Integer Grid

We prove that with high probability, a uniform sample of n points in a c...

Please sign up or login with your details

Forgot password? Click here to reset