Relevance Vector Machine with Weakly Informative Hyperprior and Extended Predictive Information Criterion

05/07/2020
by   Kazuaki Murayama, et al.
6

In the variational relevance vector machine, the gamma distribution is representative as a hyperprior over the noise precision of automatic relevance determination prior. Instead of the gamma hyperprior, we propose to use the inverse gamma hyperprior with a shape parameter close to zero and a scale parameter not necessary close to zero. This hyperprior is associated with the concept of a weakly informative prior. The effect of this hyperprior is investigated through regression to non-homogeneous data. Because it is difficult to capture the structure of such data with a single kernel function, we apply the multiple kernel method, in which multiple kernel functions with different widths are arranged for input data. We confirm that the degrees of freedom in a model is controlled by adjusting the scale parameter and keeping the shape parameter close to zero. A candidate for selecting the scale parameter is the predictive information criterion. However the estimated model using this criterion seems to cause over-fitting. This is because the multiple kernel method makes the model a situation where the dimension of the model is larger than the data size. To select an appropriate scale parameter even in such a situation, we also propose an extended prediction information criterion. It is confirmed that a multiple kernel relevance vector regression model with good predictive accuracy can be obtained by selecting the scale parameter minimizing extended prediction information criterion.

READ FULL TEXT

Authors

page 12

page 17

page 19

01/16/2013

Variational Relevance Vector Machines

The Support Vector Machine (SVM) of Vapnik (1998) has become widely esta...
08/14/2021

Analyzing insurance data with an exponentiated composite Inverse-Gamma Pareto model

Exponentiated models have been widely used in modeling various types of ...
04/13/2019

Maximum Correntropy Criterion with Variable Center

Correntropy is a local similarity measure defined in kernel space and th...
07/05/2021

Analyzing Relevance Vector Machines using a single penalty approach

Relevance vector machine (RVM) is a popular sparse Bayesian learning mod...
02/16/2020

A principled distance-based prior for the shape of the Weibull model

The use of flat or weakly informative priors is popular due to the objec...
08/23/2016

Softplus Regressions and Convex Polytopes

To construct flexible nonlinear predictive distributions, the paper intr...
04/07/2019

Proposing a Localized Relevance Vector Machine for Pattern Classification

Relevance vector machine (RVM) can be seen as a probabilistic version of...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.