Stochastic Gradient Hamiltonian Monte Carlo for Non-Convex Learning in the Big Data Regime

03/25/2019

∙

Stochastic Gradient Hamiltonian Monte Carlo (SGHMC) is a momentum version of stochastic gradient descent with properly injected Gaussian noise to find a global minimum. In this paper, non-asymptotic convergence analysis of SGHMC is given in the context of non-convex optimization, where subsampling techniques are used over an i.i.d dataset for gradient updates. Our results complement those of [RRT17] and improve on those of [GGZ18].

READ FULL TEXT

Stochastic Gradient Hamiltonian Monte Carlo for Non-Convex Learning in the Big Data Regime

Global Convergence of Stochastic Gradient Hamiltonian Monte Carlo for Non-Convex Stochastic Optimization: Non-Asymptotic Performance Bounds and Momentum-Based Acceleration

Nonasymptotic analysis of Stochastic Gradient Hamiltonian Monte Carlo under local conditions for nonconvex optimization

Convergence of stochastic gradient descent on parameterized sphere with applications to variational Monte Carlo simulation

Analysis of Langevin Monte Carlo via convex optimization

Non-Asymptotic Analysis of Fractional Langevin Monte Carlo for Non-Convex Optimization

Projected Stochastic Gradient Langevin Algorithms for Constrained Sampling and Non-Convex Learning

Non-convex shape optimization by dissipative Hamiltonian flows

Stochastic Gradient Hamiltonian Monte Carlo for Non-Convex Learning in the Big Data Regime

Related Research

Global Convergence of Stochastic Gradient Hamiltonian Monte Carlo for Non-Convex Stochastic Optimization: Non-Asymptotic Performance Bounds and Momentum-Based Acceleration

Nonasymptotic analysis of Stochastic Gradient Hamiltonian Monte Carlo under local conditions for nonconvex optimization

Convergence of stochastic gradient descent on parameterized sphere with applications to variational Monte Carlo simulation

Analysis of Langevin Monte Carlo via convex optimization

Non-Asymptotic Analysis of Fractional Langevin Monte Carlo for Non-Convex Optimization

Projected Stochastic Gradient Langevin Algorithms for Constrained Sampling and Non-Convex Learning

Non-convex shape optimization by dissipative Hamiltonian flows