DeepAI AI Chat
Log In Sign Up

Randomized Iterative Algorithms for Fisher Discriminant Analysis

by   Agniva Chowdhury, et al.
Purdue University

Fisher discriminant analysis (FDA) is a widely used method for classification and dimensionality reduction. When the number of predictor variables greatly exceeds the number of observations, one of the alternatives for conventional FDA is regularized Fisher discriminant analysis (RFDA). In this paper, we present a simple, iterative, sketching-based algorithm for RFDA that comes with provable accuracy guarantees when compared to the conventional approach. Our analysis builds upon two simple structural results that boil down to randomized matrix multiplication, a fundamental and well-understood primitive of randomized linear algebra. We analyze the behavior of RFDA when the ridge leverage and the standard leverage scores are used to select predictor variables and we prove that accurate approximations can be achieved by a sample whose size depends on the effective degrees of freedom of the RFDA problem. Our results yield significant improvements over existing approaches and our empirical evaluations support our theoretical analyses.


page 1

page 2

page 3

page 4


Fisher's Pioneering work on Discriminant Analysis and its Impact on AI

Fisher opened many new areas in Multivariate Analysis, and the one which...

On the trace ratio method and Fisher's discriminant analysis for robust multigroup classification

We compare two different linear dimensionality reduction strategies for ...

Grassmann Iterative Linear Discriminant Analysis with Proxy Matrix Optimization

Linear Discriminant Analysis (LDA) is commonly used for dimensionality r...

A Compressive Classification Framework for High-Dimensional Data

We propose a compressive classification framework for settings where the...

Bayesian Fisher's Discriminant for Functional Data

We propose a Bayesian framework of Gaussian process in order to extend F...

Asymptotic Generalization Bound of Fisher's Linear Discriminant Analysis

Fisher's linear discriminant analysis (FLDA) is an important dimension r...