Binary Linear Classification and Feature Selection via Generalized Approximate Message Passing

01/05/2014
by   Justin Ziniel, et al.
0

For the problem of binary linear classification and feature selection, we propose algorithmic approaches to classifier design based on the generalized approximate message passing (GAMP) algorithm, recently proposed in the context of compressive sensing. We are particularly motivated by problems where the number of features greatly exceeds the number of training examples, but where only a few features suffice for accurate classification. We show that sum-product GAMP can be used to (approximately) minimize the classification error rate and max-sum GAMP can be used to minimize a wide variety of regularized loss functions. Furthermore, we describe an expectation-maximization (EM)-based scheme to learn the associated model parameters online, as an alternative to cross-validation, and we show that GAMP's state-evolution framework can be used to accurately predict the misclassification rate. Finally, we present a detailed numerical study to confirm the accuracy, speed, and flexibility afforded by our GAMP-based approaches to binary linear classification and feature selection.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
09/15/2015

Sparse Multinomial Logistic Regression via Approximate Message Passing

For the problem of multi-class linear classification and feature selecti...
research
09/20/2020

Expectation propagation for the diluted Bayesian classifier

Efficient feature selection from high-dimensional datasets is a very imp...
research
06/26/2018

An Expectation-Maximization Approach to Tuning Generalized Vector Approximate Message Passing

Generalized Vector Approximate Message Passing (GVAMP) is an efficient i...
research
06/26/2022

Prediction Errors for Penalized Regressions based on Generalized Approximate Message Passing

We discuss the prediction accuracy of assumed statistical models in term...
research
10/30/2021

Optimizing Binary Symptom Checkers via Approximate Message Passing

Symptom checkers have been widely adopted as an intelligent e-healthcare...
research
08/12/2011

Compressive Imaging using Approximate Message Passing and a Markov-Tree Prior

We propose a novel algorithm for compressive imaging that exploits both ...

Please sign up or login with your details

Forgot password? Click here to reset