Log In Sign Up

Logsmooth Gradient Concentration and Tighter Runtimes for Metropolized Hamiltonian Monte Carlo

by   Yin Tat Lee, et al.

We show that the gradient norm ∇ f(x) for x ∼(-f(x)), where f is strongly convex and smooth, concentrates tightly around its mean. This removes a barrier in the prior state-of-the-art analysis for the well-studied Metropolized Hamiltonian Monte Carlo (HMC) algorithm for sampling from a strongly logconcave distribution <cit.>. We correspondingly demonstrate that Metropolized HMC mixes in Õ(κ d) iterations[%s], improving upon the Õ(κ^1.5√(d) + κ d) runtime of <cit.> by a factor (κ/d)^1/2 when the condition number κ is large. Our mixing time analysis introduces several techniques which to our knowledge have not appeared in the literature and may be of independent interest, including restrictions to a nonconvex set with good conductance behavior, and a new reduction technique for boosting a constant-accuracy total variation guarantee under weak warmness assumptions. This is the first mixing time result for logconcave distributions using only first-order function information which achieves linear dependence on κ; we also give evidence that this dependence is likely to be necessary for standard Metropolized first-order methods.


page 1

page 2

page 3

page 4


Rapid Mixing of Hamiltonian Monte Carlo on Strongly Log-Concave Distributions

We obtain several quantitative bounds on the mixing properties of the Ha...

Optimal Convergence Rate of Hamiltonian Monte Carlo for Strongly Logconcave Distributions

We study Hamiltonian Monte Carlo (HMC) for sampling from a strongly logc...

Condition-number-independent Convergence Rate of Riemannian Hamiltonian Monte Carlo with Numerical Integrators

We study the convergence rate of discretized Riemannian Hamiltonian Mont...

Mixing Time Guarantees for Unadjusted Hamiltonian Monte Carlo

We provide quantitative upper bounds on the total variation mixing time ...

Accelerating Hamiltonian Monte Carlo via Chebyshev Integration Time

Hamiltonian Monte Carlo (HMC) is a popular method in sampling. While the...

Weakly smooth Langevin Monte Carlo using p-generalized Gaussian smoothing

Langevin Monte Carlo (LMC) is an iterative process for sampling from a d...

Influence of sampling on the convergence rates of greedy algorithms for parameter-dependent random variables

The main focus of this article is to provide a mathematical study of the...