DeepAI
Log In Sign Up

ReliefE: Feature Ranking in High-dimensional Spaces via Manifold Embeddings

01/23/2021
by   Blaž Škrlj, et al.
0

Feature ranking has been widely adopted in machine learning applications such as high-throughput biology and social sciences. The approaches of the popular Relief family of algorithms assign importances to features by iteratively accounting for nearest relevant and irrelevant instances. Despite their high utility, these algorithms can be computationally expensive and not-well suited for high-dimensional sparse input spaces. In contrast, recent embedding-based methods learn compact, low-dimensional representations, potentially facilitating down-stream learning capabilities of conventional learners. This paper explores how the Relief branch of algorithms can be adapted to benefit from (Riemannian) manifold-based embeddings of instance and target spaces, where a given embedding's dimensionality is intrinsic to the dimensionality of the considered data set. The developed ReliefE algorithm is faster and can result in better feature rankings, as shown by our evaluation on 20 real-life data sets for multi-class and multi-label classification tasks. The utility of ReliefE for high-dimensional data sets is ensured by its implementation that utilizes sparse matrix algebraic operations. Finally, the relation of ReliefE to other ranking algorithms is studied via the Fuzzy Jaccard Index.

READ FULL TEXT

page 1

page 2

page 3

page 4

12/24/2018

Group Preserving Label Embedding for Multi-Label Classification

Multi-label learning is concerned with the classification of data with m...
01/14/2020

High–Dimensional Brain in a High-Dimensional World: Blessing of Dimensionality

High-dimensional data and high-dimensional representations of reality ar...
09/17/2020

Learning a Deep Part-based Representation by Preserving Data Distribution

Unsupervised dimensionality reduction is one of the commonly used techni...
08/10/2020

Feature Ranking for Semi-supervised Learning

The data made available for analysis are becoming more and more complex ...
11/26/2019

FCA2VEC: Embedding Techniques for Formal Concept Analysis

Embedding large and high dimensional data into low dimensional vector sp...
06/09/2021

Symmetric Spaces for Graph Embeddings: A Finsler-Riemannian Approach

Learning faithful graph representations as sets of vertex embeddings has...
03/12/2016

Laplacian Eigenmaps from Sparse, Noisy Similarity Measurements

Manifold learning and dimensionality reduction techniques are ubiquitous...

Code Repositories

reliefe

Manifold-based feature ranking


view repo