DeepAI AI Chat
Log In Sign Up

High-Dimensional Regularized Discriminant Analysis

by   John A. Ramey, et al.

Regularized discriminant analysis (RDA), proposed by Friedman (1989), is a widely popular classifier that lacks interpretability and is impractical for high-dimensional data sets. Here, we present an interpretable and computationally efficient classifier called high-dimensional RDA (HDRDA), designed for the small-sample, high-dimensional setting. For HDRDA, we show that each training observation, regardless of class, contributes to the class covariance matrix, resulting in an interpretable estimator that borrows from the pooled sample covariance matrix. Moreover, we show that HDRDA is equivalent to a classifier in a reduced-feature space with dimension approximately equal to the training sample size. As a result, the matrix operations employed by HDRDA are computationally linear in the number of features, making the classifier well-suited for high-dimensional classification in practice. We demonstrate that HDRDA is often superior to several sparse and regularized classifiers in terms of classification accuracy with three artificial and six real high-dimensional data sets. Also, timing comparisons between our HDRDA implementation in the sparsediscrim R package and the standard RDA formulation in the klaR R package demonstrate that as the number of features increases, the computational runtime of HDRDA is drastically smaller than that of RDA.


Classification of high-dimensional data with spiked covariance matrix structure

We study the classification problem for high-dimensional data with n obs...

diproperm: An R Package for the DiProPerm Test

High-dimensional low sample size (HDLSS) data sets emerge frequently in ...

Asymptotic Analysis of an Ensemble of Randomly Projected Linear Discriminants

Datasets from the fields of bioinformatics, chemometrics, and face recog...

A Large Dimensional Analysis of Regularized Discriminant Analysis Classifiers

This article carries out a large dimensional analysis of standard regula...

Distance-based classifier by data transformation for high-dimension, strongly spiked eigenvalue models

We consider classifiers for high-dimensional data under the strongly spi...

A Compressive Classification Framework for High-Dimensional Data

We propose a compressive classification framework for settings where the...

Code Repositories


Sparse and Regularized Discriminant Analysis in R

view repo


:exclamation: This is a read-only mirror of the CRAN R package repository. sparsediscrim — Sparse and Regularized Discriminant Analysis. Homepage:,

view repo