DeepAI
Log In Sign Up

Logsmooth Gradient Concentration and Tighter Runtimes for Metropolized Hamiltonian Monte Carlo

02/10/2020
by   Yin Tat Lee, et al.
0

We show that the gradient norm ∇ f(x) for x ∼(-f(x)), where f is strongly convex and smooth, concentrates tightly around its mean. This removes a barrier in the prior state-of-the-art analysis for the well-studied Metropolized Hamiltonian Monte Carlo (HMC) algorithm for sampling from a strongly logconcave distribution <cit.>. We correspondingly demonstrate that Metropolized HMC mixes in Õ(κ d) iterations[%s], improving upon the Õ(κ^1.5√(d) + κ d) runtime of <cit.> by a factor (κ/d)^1/2 when the condition number κ is large. Our mixing time analysis introduces several techniques which to our knowledge have not appeared in the literature and may be of independent interest, including restrictions to a nonconvex set with good conductance behavior, and a new reduction technique for boosting a constant-accuracy total variation guarantee under weak warmness assumptions. This is the first mixing time result for logconcave distributions using only first-order function information which achieves linear dependence on κ; we also give evidence that this dependence is likely to be necessary for standard Metropolized first-order methods.

READ FULL TEXT

page 1

page 2

page 3

page 4

08/23/2017

Rapid Mixing of Hamiltonian Monte Carlo on Strongly Log-Concave Distributions

We obtain several quantitative bounds on the mixing properties of the Ha...
05/07/2019

Optimal Convergence Rate of Hamiltonian Monte Carlo for Strongly Logconcave Distributions

We study Hamiltonian Monte Carlo (HMC) for sampling from a strongly logc...
10/13/2022

Condition-number-independent Convergence Rate of Riemannian Hamiltonian Monte Carlo with Numerical Integrators

We study the convergence rate of discretized Riemannian Hamiltonian Mont...
05/03/2021

Mixing Time Guarantees for Unadjusted Hamiltonian Monte Carlo

We provide quantitative upper bounds on the total variation mixing time ...
07/05/2022

Accelerating Hamiltonian Monte Carlo via Chebyshev Integration Time

Hamiltonian Monte Carlo (HMC) is a popular method in sampling. While the...
02/24/2020

Weakly smooth Langevin Monte Carlo using p-generalized Gaussian smoothing

Langevin Monte Carlo (LMC) is an iterative process for sampling from a d...
05/28/2021

Influence of sampling on the convergence rates of greedy algorithms for parameter-dependent random variables

The main focus of this article is to provide a mathematical study of the...