Bridging Information Criteria and Parameter Shrinkage for Model Selection

07/08/2013
by   Kun Zhang, et al.
0

Model selection based on classical information criteria, such as BIC, is generally computationally demanding, but its properties are well studied. On the other hand, model selection based on parameter shrinkage by ℓ_1-type penalties is computationally efficient. In this paper we make an attempt to combine their strengths, and propose a simple approach that penalizes the likelihood with data-dependent ℓ_1 penalties as in adaptive Lasso and exploits a fixed penalization parameter. Even for finite samples, its model selection results approximately coincide with those based on information criteria; in particular, we show that in some special cases, this approach and the corresponding information criterion produce exactly the same model. One can also consider this approach as a way to directly determine the penalization parameter in adaptive Lasso to achieve information criteria-like model selection. As extensions, we apply this idea to complex models including Gaussian mixture model and mixture of factor analyzers, whose model selection is traditionally difficult to do; by adopting suitable penalties, we provide continuous approximators to the corresponding information criteria, which are easy to optimize and enable efficient model selection.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
11/27/2017

Bayesian Model Averaging By Mixture Modeling

A new and numerically efficient method for Bayes factor computation and ...
research
11/27/2012

A LASSO-Penalized BIC for Mixture Model Selection

The efficacy of family-based approaches to mixture model-based clusterin...
research
01/23/2018

MCMC methods for inference in a mathematical model of pulmonary circulation

This study performs parameter inference in a partial differential equati...
research
06/19/2015

A simple application of FIC to model selection

We have recently proposed a new information-based approach to model sele...
research
10/23/2014

Model Selection for Topic Models via Spectral Decomposition

Topic models have achieved significant successes in analyzing large-scal...
research
06/24/2023

Information criteria for structured parameter selection in high dimensional tree and graph models

Parameter selection in high-dimensional models is typically finetuned in...
research
02/23/2023

Detecting Signs of Model Change with Continuous Model Selection Based on Descriptive Dimensionality

We address the issue of detecting changes of models that lie behind a da...

Please sign up or login with your details

Forgot password? Click here to reset