Asymptotic Behavior of Free Energy When Optimal Probability Distribution Is Not Unique

12/15/2020
by   Shuya Nagayasu, et al.
0

Bayesian inference is a widely used statistical method. The free energy and generalization loss, which are used to estimate the accuracy of Bayesian inference, are known to be small in singular models that do not have a unique optimal parameter. However, their characteristics are not yet known when there are multiple optimal probability distributions. In this paper, we theoretically derive the asymptotic behaviors of the generalization loss and free energy in the case that the optimal probability distributions are not unique and show that they contain asymptotically different terms from those of the conventional asymptotic analysis.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
03/14/2022

Asymptotic Behavior of Bayesian Generalization Error in Multinomial Mixtures

Multinomial mixtures are widely used in the information engineering fiel...
research
04/14/2020

Computing Bayes: Bayesian Computation from 1763 to the 21st Century

The Bayesian statistical paradigm uses the language of probability to ex...
research
08/04/2020

The Exact Asymptotic Form of Bayesian Generalization Error in Latent Dirichlet Allocation

Latent Dirichlet allocation (LDA) obtains essential information from dat...
research
06/01/2021

Separated-Spectral-Distribution Estimation Based on Bayesian Inference with Single RGB Camera

In this paper, we propose a novel method for separately estimating spect...
research
11/25/2020

Free Energy Minimization: A Unified Framework for Modelling, Inference, Learning,and Optimization

The goal of these lecture notes is to review the problem of free energy ...
research
12/09/2021

Times Square sampling: an adaptive algorithm for free energy estimation

Estimating free energy differences, an important problem in computationa...
research
05/17/2022

Bayesian Inference for Non-Parametric Extreme Value Theory

Statistical inference for extreme values of random events is difficult i...

Please sign up or login with your details

Forgot password? Click here to reset