Reference Bayesian analysis for hierarchical models

04/25/2019
by   Thaís C. O. Fonseca, et al.
0

This paper proposes an alternative approach for constructing invariant Jeffreys prior distributions tailored for hierarchical or multilevel models. In particular, our proposal is based on a flexible decomposition of the Fisher information for hierarchical models which overcomes the marginalization step of the likelihood of model parameters. The Fisher information matrix for the hierarchical model is derived from the Hessian of the Kullback-Liebler (KL) divergence for the model in a neighborhood of the parameter value of interest. Properties of the KL divergence are used to prove the proposed decomposition. Our proposal takes advantage of the hierarchy and leads to an alternative way of computing Jeffreys priors for the hyperparameters and an upper bound for the prior information. While the Jeffreys prior gives the minimum information about parameters, the proposed bound gives an upper limit for the information put in any prior distribution. A prior with information above that limit may be considered too informative. From a practical point of view, the proposed prior may be evaluated computationally as part of a MCMC algorithm. This property might be essential for modeling setups with many levels in which analytic marginalization is not feasible. We illustrate the usefulness of our proposal with examples in mixture models, in model selection priors such as lasso and in the Student-t model.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
09/15/2021

How to use KL-divergence to construct conjugate priors, with well-defined non-informative limits, for the multivariate Gaussian

The Wishart distribution is the standard conjugate prior for the precisi...
research
04/26/2018

Tight MMSE Bounds for the AGN Channel Under KL Divergence Constraints on the Input Distribution

Tight bounds on the minimum mean square error for the additive Gaussian ...
research
12/02/2019

Stochastic Variational Inference via Upper Bound

Stochastic variational inference (SVI) plays a key role in Bayesian deep...
research
03/15/2023

Translating predictive distributions into informative priors

When complex Bayesian models exhibit implausible behaviour, one solution...
research
12/24/2017

Bayesian Nonparametric Causal Inference: Information Rates and Learning Algorithms

We investigate the problem of estimating the causal effect of a treatmen...
research
05/18/2018

Bayesian model reduction

This paper reviews recent developments in statistical structure learning...
research
10/23/2020

The Wasserstein Impact Measure (WIM): a generally applicable, practical tool for quantifying prior impact in Bayesian statistics

The prior distribution is a crucial building block in Bayesian analysis,...

Please sign up or login with your details

Forgot password? Click here to reset