Log In Sign Up

ReliefE: Feature Ranking in High-dimensional Spaces via Manifold Embeddings

by   Blaž Škrlj, et al.

Feature ranking has been widely adopted in machine learning applications such as high-throughput biology and social sciences. The approaches of the popular Relief family of algorithms assign importances to features by iteratively accounting for nearest relevant and irrelevant instances. Despite their high utility, these algorithms can be computationally expensive and not-well suited for high-dimensional sparse input spaces. In contrast, recent embedding-based methods learn compact, low-dimensional representations, potentially facilitating down-stream learning capabilities of conventional learners. This paper explores how the Relief branch of algorithms can be adapted to benefit from (Riemannian) manifold-based embeddings of instance and target spaces, where a given embedding's dimensionality is intrinsic to the dimensionality of the considered data set. The developed ReliefE algorithm is faster and can result in better feature rankings, as shown by our evaluation on 20 real-life data sets for multi-class and multi-label classification tasks. The utility of ReliefE for high-dimensional data sets is ensured by its implementation that utilizes sparse matrix algebraic operations. Finally, the relation of ReliefE to other ranking algorithms is studied via the Fuzzy Jaccard Index.


page 1

page 2

page 3

page 4


Group Preserving Label Embedding for Multi-Label Classification

Multi-label learning is concerned with the classification of data with m...

High–Dimensional Brain in a High-Dimensional World: Blessing of Dimensionality

High-dimensional data and high-dimensional representations of reality ar...

Learning a Deep Part-based Representation by Preserving Data Distribution

Unsupervised dimensionality reduction is one of the commonly used techni...

Feature Ranking for Semi-supervised Learning

The data made available for analysis are becoming more and more complex ...

FCA2VEC: Embedding Techniques for Formal Concept Analysis

Embedding large and high dimensional data into low dimensional vector sp...

Symmetric Spaces for Graph Embeddings: A Finsler-Riemannian Approach

Learning faithful graph representations as sets of vertex embeddings has...

Laplacian Eigenmaps from Sparse, Noisy Similarity Measurements

Manifold learning and dimensionality reduction techniques are ubiquitous...

Code Repositories


Manifold-based feature ranking

view repo