A Diffusion Process Perspective on Posterior Contraction Rates for Parameters

09/03/2019
by   Wenlong Mou, et al.
0

We show that diffusion processes can be exploited to study the posterior contraction rates of parameters in Bayesian models. By treating the posterior distribution as a stationary distribution of a stochastic differential equation (SDE), posterior convergence rates can be established via control of the moments of the corresponding SDE. Our results depend on the structure of the population log-likelihood function, obtained in the limit of an infinite sample sample size, and stochastic perturbation bounds between the population and sample log-likelihood functions. When the population log-likelihood is strongly concave, we establish posterior convergence of a d-dimensional parameter at the optimal rate (d/n)^1/ 2. In the weakly concave setting, we show that the convergence rate is determined by the unique solution of a non-linear equation that arises from the interplay between the degree of weak concavity and the stochastic perturbation bounds. We illustrate this general theory by deriving posterior convergence rates for three concrete examples: Bayesian logistic regression models, Bayesian single index models, and over-specified Bayesian mixture models.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
01/17/2022

Finite samples inference and critical dimension for stochastically linear models

The aim of this note is to state a couple of general results about the p...
research
01/21/2021

Optimal convergence rates for the invariant density estimation of jump-diffusion processes

We aim at estimating the invariant density associated to a stochastic di...
research
09/23/2022

Posterior Probabilities: Nonmonotonicity, Asymptotic Rates, Log-Concavity, and Turán's Inequality

In the standard Bayesian framework data are assumed to be generated by a...
research
06/14/2023

Contraction Rate Estimates of Stochastic Gradient Kinetic Langevin Integrators

In previous work, we introduced a method for determining convergence rat...
research
02/12/2021

Stochastic Convergence Rates and Applications of Adaptive Quadrature in Bayesian Inference

We provide the first stochastic convergence rates for adaptive Gauss–Her...
research
04/29/2023

Representing Additive Gaussian Processes by Sparse Matrices

Among generalized additive models, additive Matérn Gaussian Processes (G...
research
11/23/2022

Efficient sampling of non log-concave posterior distributions with mixture of noises

This paper focuses on a challenging class of inverse problems that is of...

Please sign up or login with your details

Forgot password? Click here to reset