Relevance Vector Machine with Weakly Informative Hyperprior and Extended Predictive Information Criterion

by   Kazuaki Murayama, et al.

In the variational relevance vector machine, the gamma distribution is representative as a hyperprior over the noise precision of automatic relevance determination prior. Instead of the gamma hyperprior, we propose to use the inverse gamma hyperprior with a shape parameter close to zero and a scale parameter not necessary close to zero. This hyperprior is associated with the concept of a weakly informative prior. The effect of this hyperprior is investigated through regression to non-homogeneous data. Because it is difficult to capture the structure of such data with a single kernel function, we apply the multiple kernel method, in which multiple kernel functions with different widths are arranged for input data. We confirm that the degrees of freedom in a model is controlled by adjusting the scale parameter and keeping the shape parameter close to zero. A candidate for selecting the scale parameter is the predictive information criterion. However the estimated model using this criterion seems to cause over-fitting. This is because the multiple kernel method makes the model a situation where the dimension of the model is larger than the data size. To select an appropriate scale parameter even in such a situation, we also propose an extended prediction information criterion. It is confirmed that a multiple kernel relevance vector regression model with good predictive accuracy can be obtained by selecting the scale parameter minimizing extended prediction information criterion.



page 12

page 17

page 19


Variational Relevance Vector Machines

The Support Vector Machine (SVM) of Vapnik (1998) has become widely esta...

Analyzing insurance data with an exponentiated composite Inverse-Gamma Pareto model

Exponentiated models have been widely used in modeling various types of ...

Maximum Correntropy Criterion with Variable Center

Correntropy is a local similarity measure defined in kernel space and th...

Analyzing Relevance Vector Machines using a single penalty approach

Relevance vector machine (RVM) is a popular sparse Bayesian learning mod...

A principled distance-based prior for the shape of the Weibull model

The use of flat or weakly informative priors is popular due to the objec...

Softplus Regressions and Convex Polytopes

To construct flexible nonlinear predictive distributions, the paper intr...

Proposing a Localized Relevance Vector Machine for Pattern Classification

Relevance vector machine (RVM) can be seen as a probabilistic version of...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.