Analyzing Relevance Vector Machines using a single penalty approach

by   Anand Dixit, et al.

Relevance vector machine (RVM) is a popular sparse Bayesian learning model typically used for prediction. Recently it has been shown that improper priors assumed on multiple penalty parameters in RVM may lead to an improper posterior. Currently in the literature, the sufficient conditions for posterior propriety of RVM do not allow improper priors over the multiple penalty parameters. In this article, we propose a single penalty relevance vector machine (SPRVM) model in which multiple penalty parameters are replaced by a single penalty and we consider a semi Bayesian approach for fitting the SPRVM. The necessary and sufficient conditions for posterior propriety of SPRVM are more liberal than those of RVM and allow for several improper priors over the penalty parameter. Additionally, we also prove the geometric ergodicity of the Gibbs sampler used to analyze the SPRVM model and hence can estimate the asymptotic standard errors associated with the Monte Carlo estimate of the means of the posterior predictive distribution. Such a Monte Carlo standard error cannot be computed in the case of RVM, since the rate of convergence of the Gibbs sampler used to analyze RVM is not known. The predictive performance of RVM and SPRVM is compared by analyzing three real life datasets.


page 1

page 2

page 3

page 4


Posterior Impropriety of some Sparse Bayesian Learning Models

Sparse Bayesian learning models are typically used for prediction in dat...

Posterior properties of the Weibull distribution for censored data

The Weibull distribution is one of the most used tools in reliability an...

Convergence Analysis of a Collapsed Gibbs Sampler for Bayesian Vector Autoregressions

We propose a collapsed Gibbs sampler for Bayesian vector autoregressions...

On spike-and-slab priors for Bayesian equation discovery of nonlinear dynamical systems via sparse linear regression

This paper presents the use of spike-and-slab (SS) priors for discoverin...

Bayesian Inference of Local Projections with Roughness Penalty Priors

A local projection is a statistical framework that accounts for the rela...

Relevance Vector Machine with Weakly Informative Hyperprior and Extended Predictive Information Criterion

In the variational relevance vector machine, the gamma distribution is r...

Posterior Consistency for Bayesian Relevance Vector Machines

Statistical modeling and inference problems with sample sizes substantia...