Dimension Reduction by Mutual Information Feature Extraction

by   Ali Shadvar, et al.

During the past decades, to study high-dimensional data in a large variety of problems, researchers have proposed many Feature Extraction algorithms. One of the most effective approaches for optimal feature extraction is based on mutual information (MI). However it is not always easy to get an accurate estimation for high dimensional MI. In terms of MI, the optimal feature extraction is creating a feature set from the data which jointly have the largest dependency on the target class and minimum redundancy. In this paper, a component-by-component gradient ascent method is proposed for feature extraction which is based on one-dimensional MI estimates. We will refer to this algorithm as Mutual Information Feature Extraction (MIFX). The performance of this proposed method is evaluated using UCI databases. The results indicate that MIFX provides a robust performance over different data sets which are almost always the best or comparable to the best ones.



page 1

page 2

page 3

page 4


Dimension Reduction by Mutual Information Discriminant Analysis

In the past few decades, researchers have proposed many discriminant ana...

Relational Autoencoder for Feature Extraction

Feature extraction becomes increasingly important as data grows high dim...

Sliced Mutual Information: A Scalable Measure of Statistical Dependence

Mutual information (MI) is a fundamental measure of statistical dependen...

Sirius: A Mutual Information Tool for Exploratory Visualization of Mixed Data

Data scientists across disciplines are increasingly in need of explorato...

The Information Mutual Information Ratio for Counting Image Features and Their Matches

Feature extraction and description is an important topic of computer vis...

Impact of the composition of feature extraction and class sampling in medicare fraud detection

With healthcare being critical aspect, health insurance has become an im...

Stochastic Mutual Information Gradient Estimation for Dimensionality Reduction Networks

Feature ranking and selection is a widely used approach in various appli...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.