On the Equivalence of Factorized Information Criterion Regularization and the Chinese Restaurant Process Prior

06/30/2015
by   Shaohua Li, et al.
0

Factorized Information Criterion (FIC) is a recently developed information criterion, based on which a novel model selection methodology, namely Factorized Asymptotic Bayesian (FAB) Inference, has been developed and successfully applied to various hierarchical Bayesian models. The Dirichlet Process (DP) prior, and one of its well known representations, the Chinese Restaurant Process (CRP), derive another line of model selection methods. FIC can be viewed as a prior distribution over the latent variable configurations. Under this view, we prove that when the parameter dimensionality D_c=2, FIC is equivalent to CRP. We argue that when D_c>2, FIC avoids an inherent problem of DP/CRP, i.e. the data likelihood will dominate the impact of the prior, and thus the model selection capability will weaken as D_c increases. However, FIC overestimates the data likelihood. As a result, FIC may be overly biased towards models with less components. We propose a natural generalization of FIC, which finds a middle ground between CRP and FIC, and may yield more accurate model selection results than FIC.

READ FULL TEXT

page 1

page 2

page 3

research
04/22/2015

Rebuilding Factorized Information Criterion: Asymptotically Accurate Marginal Likelihood

Factorized information criterion (FIC) is a recently developed approxima...
research
06/19/2015

A simple application of FIC to model selection

We have recently proposed a new information-based approach to model sele...
research
05/27/2018

BIC extensions for order-constrained model selection

The Schwarz or Bayesian information criterion (BIC) is one of the most w...
research
02/27/2013

Three Approaches to Probability Model Selection

This paper compares three approaches to the problem of selecting among p...
research
10/23/2021

Prior Intensified Information Criterion

The widely applicable information criterion (WAIC) has been used as a mo...
research
03/08/2017

Exact Dimensionality Selection for Bayesian PCA

We present a Bayesian model selection approach to estimate the intrinsic...
research
05/22/2018

Multi-model inference through projections in model space

Information criteria have had a profound impact on modern ecological sci...

Please sign up or login with your details

Forgot password? Click here to reset