Stochastic Langevin Differential Inclusions with Applications to Machine Learning

by   Fabio V. Difonzo, et al.

Stochastic differential equations of Langevin-diffusion form have received significant recent, thanks to their foundational role in both Bayesian sampling algorithms and optimization in machine learning. In the latter, they serve as a conceptual model of the stochastic gradient flow in training over-parametrized models. However, the literature typically assumes smoothness of the potential, whose gradient is the drift term. Nevertheless, there are many problems, for which the potential function is not continuously differentiable, and hence the drift is not Lipschitz-continuous everywhere. This is exemplified by robust losses and Rectified Linear Units in regression problems. In this paper, we show some foundational results regarding the flow and asymptotic properties of Langevin-type Stochastic Differential Inclusions under assumptions appropriate to the machine-learning settings. In particular, we show strong existence of the solution, as well as asymptotic minimization of the canonical Free Energy Functional.


page 1

page 2

page 3

page 4


On Explicit Milstein-type Scheme for Mckean-Vlasov Stochastic Differential Equations with Super-linear Drift Coefficient

We develop an explicit Milstein-type scheme for McKean-Vlasov stochastic...

On approximation of solutions of stochastic delay differential equations via randomized Euler scheme

We investigate existence, uniqueness and approximation of solutions to s...

Drift parameter estimation for nonlinear reflected stochastic differential equations

We study the maximum likehood estimator and least squares estimator for ...

Asymptotic Analysis via Stochastic Differential Equations of Gradient Descent Algorithms in Statistical and Computational Paradigms

This paper investigates asymptotic behaviors of gradient descent algorit...

Error Bounds of the Invariant Statistics in Machine Learning of Ergodic Itô Diffusions

This paper studies the theoretical underpinnings of machine learning of ...

Please sign up or login with your details

Forgot password? Click here to reset