Communications Inspired Linear Discriminant Analysis

by   Minhua Chen, et al.

We study the problem of supervised linear dimensionality reduction, taking an information-theoretic viewpoint. The linear projection matrix is designed by maximizing the mutual information between the projected signal and the class label (based on a Shannon entropy measure). By harnessing a recent theoretical result on the gradient of mutual information, the above optimization problem can be solved directly using gradient descent, without requiring simplification of the objective function. Theoretical analysis and empirical comparison are made between the proposed method and two closely related methods (Linear Discriminant Analysis and Information Discriminant Analysis), and comparisons are also made with a method in which Renyi entropy is used to define the mutual information (in this case the gradient may be computed simply, under a special parameter setting). Relative to these alternative approaches, the proposed method achieves promising results on real datasets.



There are no comments yet.


page 1

page 2

page 3

page 4


Grassmann Iterative Linear Discriminant Analysis with Proxy Matrix Optimization

Linear Discriminant Analysis (LDA) is commonly used for dimensionality r...

Theoretical Evaluation of Feature Selection Methods based on Mutual Information

Feature selection methods are usually evaluated by wrapping specific cla...

Information theoretic learning of robust deep representations

We propose a novel objective function for learning robust deep represent...

Dimension Reduction by Mutual Information Discriminant Analysis

In the past few decades, researchers have proposed many discriminant ana...

Stochastic Mutual Information Gradient Estimation for Dimensionality Reduction Networks

Feature ranking and selection is a widely used approach in various appli...

A Practical Consideration on Convex Mutual Information

In this paper, we focus on the convex mutual information, which was foun...

Communication in Plants: Comparison of Multiple Action Potential and Mechanosensitive Signals with Experiments

Both action potentials and mechanosensitive signalling are an important ...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.