Convergence Rates of Empirical Bayes Posterior Distributions: A Variational Perspective

by   Fengshuo Zhang, et al.

We study the convergence rates of empirical Bayes posterior distributions for nonparametric and high-dimensional inference. We show that as long as the hyperparameter set is discrete, the empirical Bayes posterior distribution induced by the maximum marginal likelihood estimator can be regarded as a variational approximation to a hierarchical Bayes posterior distribution. This connection between empirical Bayes and variational Bayes allows us to leverage the recent results in the variational Bayes literature, and directly obtains the convergence rates of empirical Bayes posterior distributions from a variational perspective. For a more general hyperparameter set that is not necessarily discrete, we introduce a new technique called "prior decomposition" to deal with prior distributions that can be written as convex combinations of probability measures whose supports are low-dimensional subspaces. This leads to generalized versions of the classical "prior mass and testing" conditions for the convergence rates of empirical Bayes. Our theory is applied to a number of statistical estimation problems including nonparametric density estimation and sparse linear regression.


page 1

page 2

page 3

page 4


Convergence Rates of Variational Posterior Distributions

We study convergence rates of variational posterior distributions for no...

Nonparametric empirical Bayes estimation based on generalized Laguerre series

In this work, we delve into the nonparametric empirical Bayes theory and...

An Empirical Bayes Approach for High Dimensional Classification

We propose an empirical Bayes estimator based on Dirichlet process mixtu...

Neural Empirical Bayes: Source Distribution Estimation and its Applications to Simulation-Based Inference

We revisit empirical Bayes in the absence of a tractable likelihood func...

Where Bayes tweaks Gauss: Conditionally Gaussian priors for stable multi-dipole estimation

We present a very simple yet powerful generalization of a previously des...

Decoupling Learning Rates Using Empirical Bayes Priors

In this work, we propose an Empirical Bayes approach to decouple the lea...