Rebuilding Factorized Information Criterion: Asymptotically Accurate Marginal Likelihood

04/22/2015
by   Kohei Hayashi, et al.
0

Factorized information criterion (FIC) is a recently developed approximation technique for the marginal log-likelihood, which provides an automatic model selection framework for a few latent variable models (LVMs) with tractable inference algorithms. This paper reconsiders FIC and fills theoretical gaps of previous FIC studies. First, we reveal the core idea of FIC that allows generalization for a broader class of LVMs, including continuous LVMs, in contrast to previous FICs, which are applicable only to binary LVMs. Second, we investigate the model selection mechanism of the generalized FIC. Our analysis provides a formal justification of FIC as a model selection criterion for LVMs and also a systematic procedure for pruning redundant latent variables that have been removed heuristically in previous studies. Third, we provide an interpretation of FIC as a variational free energy and uncover a few previously-unknown their relationships. A demonstrative study on Bayesian principal component analysis is provided and numerical experiments support our theoretical results.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
04/28/2019

Schwartz type model selection for ergodic stochastic differential equation models

We study the construction of the theoretical foundation of model compari...
research
06/30/2015

On the Equivalence of Factorized Information Criterion Regularization and the Chinese Restaurant Process Prior

Factorized Information Criterion (FIC) is a recently developed informati...
research
02/06/2016

A Tractable Fully Bayesian Method for the Stochastic Block Model

The stochastic block model (SBM) is a generative model revealing macrosc...
research
08/28/2019

On the overestimation of widely applicable Bayesian information criterion

A widely applicable Bayesian information criterion (Watanabe, 2013) is a...
research
06/04/2019

Estimating Real Log Canonical Thresholds

Evaluation of the marginal likelihood plays an important role in model s...
research
06/19/2015

A simple application of FIC to model selection

We have recently proposed a new information-based approach to model sele...
research
12/29/2014

Marginal likelihood and model selection for Gaussian latent tree and forest models

Gaussian latent tree models, or more generally, Gaussian latent forest m...

Please sign up or login with your details

Forgot password? Click here to reset