Convergence Rates of Empirical Bayes Posterior Distributions: A Variational Perspective

09/08/2020
by   Fengshuo Zhang, et al.
13

We study the convergence rates of empirical Bayes posterior distributions for nonparametric and high-dimensional inference. We show that as long as the hyperparameter set is discrete, the empirical Bayes posterior distribution induced by the maximum marginal likelihood estimator can be regarded as a variational approximation to a hierarchical Bayes posterior distribution. This connection between empirical Bayes and variational Bayes allows us to leverage the recent results in the variational Bayes literature, and directly obtains the convergence rates of empirical Bayes posterior distributions from a variational perspective. For a more general hyperparameter set that is not necessarily discrete, we introduce a new technique called "prior decomposition" to deal with prior distributions that can be written as convex combinations of probability measures whose supports are low-dimensional subspaces. This leads to generalized versions of the classical "prior mass and testing" conditions for the convergence rates of empirical Bayes. Our theory is applied to a number of statistical estimation problems including nonparametric density estimation and sparse linear regression.

READ FULL TEXT

page 1

page 2

page 3

page 4

12/07/2017

Convergence Rates of Variational Posterior Distributions

We study convergence rates of variational posterior distributions for no...
12/16/2021

Nonparametric empirical Bayes estimation based on generalized Laguerre series

In this work, we delve into the nonparametric empirical Bayes theory and...
02/16/2017

An Empirical Bayes Approach for High Dimensional Classification

We propose an empirical Bayes estimator based on Dirichlet process mixtu...
11/11/2020

Neural Empirical Bayes: Source Distribution Estimation and its Applications to Simulation-Based Inference

We revisit empirical Bayes in the absence of a tractable likelihood func...
06/07/2020

Where Bayes tweaks Gauss: Conditionally Gaussian priors for stable multi-dipole estimation

We present a very simple yet powerful generalization of a previously des...
02/04/2020

Decoupling Learning Rates Using Empirical Bayes Priors

In this work, we propose an Empirical Bayes approach to decouple the lea...