Fast Sparse Classification for Generalized Linear and Additive Models

02/23/2022
by   Jiachang Liu, et al.
6

We present fast classification techniques for sparse generalized linear and additive models. These techniques can handle thousands of features and thousands of observations in minutes, even in the presence of many highly correlated features. For fast sparse logistic regression, our computational speed-up over other best-subset search techniques owes to linear and quadratic surrogate cuts for the logistic loss that allow us to efficiently screen features for elimination, as well as use of a priority queue that favors a more uniform exploration of features. As an alternative to the logistic loss, we propose the exponential loss, which permits an analytical solution to the line search at each iteration. Our algorithms are generally 2 to 5 times faster than previous approaches. They produce interpretable models that have accuracy comparable to black box models on challenging datasets.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
04/29/2020

Neural Additive Models: Interpretable Machine Learning with Neural Nets

Deep neural networks (DNNs) are powerful black-box predictors that have ...
research
04/05/2019

Logitron: Perceptron-augmented classification model based on an extended logistic loss function

Classification is the most important process in data analysis. However, ...
research
10/12/2022

FasterRisk: Fast and Accurate Interpretable Risk Scores

Over the last century, risk scores have been the most popular form of pr...
research
11/16/2021

A Unified and Fast Interpretable Model for Predictive Analytics

In this paper, we propose FXAM (Fast and eXplainable Additive Model), a ...
research
03/08/2017

Sparse Quadratic Logistic Regression in Sub-quadratic Time

We consider support recovery in the quadratic logistic regression settin...
research
01/28/2021

Low Complexity Approximate Bayesian Logistic Regression for Sparse Online Learning

Theoretical results show that Bayesian methods can achieve lower bounds ...
research
07/29/2019

A Factored Generalized Additive Model for Clinical Decision Support in the Operating Room

Logistic regression (LR) is widely used in clinical prediction because i...

Please sign up or login with your details

Forgot password? Click here to reset