Stochastic Mutual Information Gradient Estimation for Dimensionality Reduction Networks

05/01/2021
by   Ozan Özdenizci, et al.
0

Feature ranking and selection is a widely used approach in various applications of supervised dimensionality reduction in discriminative machine learning. Nevertheless there exists significant evidence on feature ranking and selection algorithms based on any criterion leading to potentially sub-optimal solutions for class separability. In that regard, we introduce emerging information theoretic feature transformation protocols as an end-to-end neural network training approach. We present a dimensionality reduction network (MMINet) training procedure based on the stochastic estimate of the mutual information gradient. The network projects high-dimensional features onto an output feature space where lower dimensional representations of features carry maximum mutual information with their associated class labels. Furthermore, we formulate the training objective to be estimated non-parametrically with no distributional assumptions. We experimentally evaluate our method with applications to high-dimensional biological data sets, and relate it to conventional feature selection algorithms to form a special case of our approach.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

03/28/2019

Information Theoretic Feature Transformation Learning for Brain Interfaces

Objective: A variety of pattern analysis techniques for model training i...
06/09/2021

Sirius: A Mutual Information Tool for Exploratory Visualization of Mixed Data

Data scientists across disciplines are increasingly in need of explorato...
07/14/2012

Dimension Reduction by Mutual Information Feature Extraction

During the past decades, to study high-dimensional data in a large varie...
12/13/2020

Active Feature Selection for the Mutual Information Criterion

We study active feature selection, a novel feature selection setting in ...
11/18/2020

Accelerating Text Mining Using Domain-Specific Stop Word Lists

Text preprocessing is an essential step in text mining. Removing words t...
01/23/2021

ReliefE: Feature Ranking in High-dimensional Spaces via Manifold Embeddings

Feature ranking has been widely adopted in machine learning applications...
06/27/2012

Communications Inspired Linear Discriminant Analysis

We study the problem of supervised linear dimensionality reduction, taki...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.