Orthogonal Least Squares Based Fast Feature Selection for Linear Classification

01/21/2021
by   Sikai Zhang, et al.
0

An Orthogonal Least Squares (OLS) based feature selection method is proposed for both binomial and multinomial classification. The novel Squared Orthogonal Correlation Coefficient (SOCC) is defined based on Error Reduction Ratio (ERR) in OLS and used as the feature ranking criterion. The equivalence between the canonical correlation coefficient, Fisher's criterion, and the sum of the SOCCs is revealed, which unveils the statistical implication of ERR in OLS for the first time. It is also shown that the OLS based feature selection method has speed advantages when applied for greedy search. The proposed method is comprehensively compared with the mutual information based feature selection methods in 2 synthetic and 7 real world datasets. The results show that the proposed method is always in the top 5 among the 10 candidate methods. Besides, the proposed method can be directly applied to continuous features without discretisation, which is another significant advantage over mutual information based methods.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/15/2021

Canonical-Correlation-Based Fast Feature Selection

This paper proposes a canonical-correlation-based filter method for feat...
research
02/01/2015

Feature Selection with Redundancy-complementariness Dispersion

Feature selection has attracted significant attention in data mining and...
research
10/03/2017

Multi-layer architecture for efficient steganalysis of Undermp3cover in multi-encoder scenario

Mp3 is a very popular audio format and hence it can be a good host for c...
research
06/10/2021

A concise method for feature selection via normalized frequencies

Feature selection is an important part of building a machine learning mo...
research
12/19/2022

An Extension of Fisher's Criterion: Theoretical Results with a Neural Network Realization

Fisher's criterion is a widely used tool in machine learning for feature...
research
02/19/2019

An entropic feature selection method in perspective of Turing formula

Health data are generally complex in type and small in sample size. Such...
research
04/06/2023

SLM: End-to-end Feature Selection via Sparse Learnable Masks

Feature selection has been widely used to alleviate compute requirements...

Please sign up or login with your details

Forgot password? Click here to reset