Variational Relevance Vector Machines

by   Christopher M. Bishop, et al.

The Support Vector Machine (SVM) of Vapnik (1998) has become widely established as one of the leading approaches to pattern recognition and machine learning. It expresses predictions in terms of a linear combination of kernel functions centred on a subset of the training data, known as support vectors. Despite its widespread success, the SVM suffers from some important limitations, one of the most significant being that it makes point predictions rather than generating predictive distributions. Recently Tipping (1999) has formulated the Relevance Vector Machine (RVM), a probabilistic model whose functional form is equivalent to the SVM. It achieves comparable recognition accuracy to the SVM, yet provides a full predictive distribution, and also requires substantially fewer kernel functions. The original treatment of the RVM relied on the use of type II maximum likelihood (the `evidence framework') to provide point estimates of the hyperparameters which govern model sparsity. In this paper we show how the RVM can be formulated and solved within a completely Bayesian paradigm through the use of variational inference, thereby giving a posterior distribution over both parameters and hyperparameters. We demonstrate the practicality and performance of the variational RVM using both synthetic and real world examples.


page 1

page 2

page 3

page 4


Support vector machines/relevance vector machine for remote sensing classification: A review

Kernel-based machine learning algorithms are based on mapping data from ...

Sparse Polynomial Chaos expansions using Variational Relevance Vector Machines

The challenges for non-intrusive methods for Polynomial Chaos modeling l...

Relevance Vector Machine with Weakly Informative Hyperprior and Extended Predictive Information Criterion

In the variational relevance vector machine, the gamma distribution is r...

Learning from Distributions via Support Measure Machines

This paper presents a kernel-based discriminative learning framework on ...

Proposing a Localized Relevance Vector Machine for Pattern Classification

Relevance vector machine (RVM) can be seen as a probabilistic version of...

On Linear Separability under Linear Compression with Applications to Hard Support Vector Machine

This paper investigates the theoretical problem of maintaining linear se...

A Weighted Solution to SVM Actionability and Interpretability

Research in machine learning has successfully developed algorithms to bu...