Manifold Elastic Net: A Unified Framework for Sparse Dimension Reduction

07/21/2010
by   Tianyi Zhou, et al.
0

It is difficult to find the optimal sparse solution of a manifold learning based dimensionality reduction algorithm. The lasso or the elastic net penalized manifold learning based dimensionality reduction is not directly a lasso penalized least square problem and thus the least angle regression (LARS) (Efron et al. LARS), one of the most popular algorithms in sparse learning, cannot be applied. Therefore, most current approaches take indirect ways or have strict settings, which can be inconvenient for applications. In this paper, we proposed the manifold elastic net or MEN for short. MEN incorporates the merits of both the manifold learning based dimensionality reduction and the sparse learning based dimensionality reduction. By using a series of equivalent transformations, we show MEN is equivalent to the lasso penalized least square problem and thus LARS is adopted to obtain the optimal sparse solution of MEN. In particular, MEN has the following advantages for subsequent classification: 1) the local geometry of samples is well preserved for low dimensional data representation, 2) both the margin maximization and the classification error minimization are considered for sparse projection calculation, 3) the projection matrix of MEN improves the parsimony in computation, 4) the elastic net penalty reduces the over-fitting problem, and 5) the projection matrix of MEN can be interpreted psychologically and physiologically. Experimental evidence on face recognition over various popular datasets suggests that MEN is superior to top level dimensionality reduction algorithms.

READ FULL TEXT

page 18

page 26

page 27

page 28

research
09/29/2008

An Information Geometric Framework for Dimensionality Reduction

This report concerns the problem of dimensionality reduction through inf...
research
07/29/2016

Dimensionality reduction based on Distance Preservation to Local Mean (DPLM) for SPD matrices and its application in BCI

In this paper, we propose a nonlinear dimensionality reduction algorithm...
research
09/29/2015

Foundations of Coupled Nonlinear Dimensionality Reduction

In this paper we introduce and analyze the learning scenario of coupled ...
research
09/07/2022

Dimensionality Reduction using Elastic Measures

With the recent surge in big data analytics for hyper-dimensional data t...
research
11/11/2013

Toward a unified theory of sparse dimensionality reduction in Euclidean space

Let Φ∈R^m× n be a sparse Johnson-Lindenstrauss transform [KN14] with s n...
research
06/05/2023

Kinodynamic FMT* with Dimensionality Reduction Heuristics and Neural Network Controllers

This paper proposes a new sampling-based kinodynamic motion planning alg...
research
06/27/2012

Regularizers versus Losses for Nonlinear Dimensionality Reduction: A Factored View with New Convex Relaxations

We demonstrate that almost all non-parametric dimensionality reduction m...

Please sign up or login with your details

Forgot password? Click here to reset