Dimension-free convergence rates for gradient Langevin dynamics in RKHS

02/29/2020
by   Boris Muzellec, et al.
12

Gradient Langevin dynamics (GLD) and stochastic GLD (SGLD) have attracted considerable attention lately, as a way to provide convergence guarantees in a non-convex setting. However, the known rates grow exponentially with the dimension of the space. In this work, we provide a convergence analysis of GLD and SGLD when the optimization space is an infinite dimensional Hilbert space. More precisely, we derive non-asymptotic, dimension-free convergence rates for GLD/SGLD when performing regularized non-convex optimization in a reproducing kernel Hilbert space. Amongst others, the convergence analysis relies on the properties of a stochastic differential equation, its discrete time Galerkin approximation and the geometric ergodicity of the associated Markov chains.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
12/11/2020

Recent Theoretical Advances in Non-Convex Optimization

Motivated by recent increased interest in optimization algorithms for no...
research
06/01/2020

Factorial Powers for Stochastic Optimization

The convergence rates for convex and non-convex optimization methods dep...
research
10/25/2018

Uniform Convergence of Gradients for Non-Convex Learning and Optimization

We investigate 1) the rate at which refined properties of the empirical ...
research
04/11/2022

Non-Convex Optimization with Certificates and Fast Rates Through Kernel Sums of Squares

We consider potentially non-convex optimization problems, for which opti...
research
06/13/2022

The Kolmogorov Infinite Dimensional Equation in a Hilbert space Via Deep Learning Methods

We consider the nonlinear Kolmogorov equation posed in a Hilbert space H...
research
01/18/2011

Convergence rates of efficient global optimization algorithms

Efficient global optimization is the problem of minimizing an unknown fu...
research
02/02/2022

HMC and Langevin united in the unadjusted and convex case

We consider a family of unadjusted HMC samplers, which includes standard...

Please sign up or login with your details

Forgot password? Click here to reset