SNEkhorn: Dimension Reduction with Symmetric Entropic Affinities

by   Hugues van Assel, et al.

Many approaches in machine learning rely on a weighted graph to encode the similarities between samples in a dataset. Entropic affinities (EAs), which are notably used in the popular Dimensionality Reduction (DR) algorithm t-SNE, are particular instances of such graphs. To ensure robustness to heterogeneous sampling densities, EAs assign a kernel bandwidth parameter to every sample in such a way that the entropy of each row in the affinity matrix is kept constant at a specific value, whose exponential is known as perplexity. EAs are inherently asymmetric and row-wise stochastic, but they are used in DR approaches after undergoing heuristic symmetrization methods that violate both the row-wise constant entropy and stochasticity properties. In this work, we uncover a novel characterization of EA as an optimal transport problem, allowing a natural symmetrization that can be computed efficiently using dual ascent. The corresponding novel affinity matrix derives advantages from symmetric doubly stochastic normalization in terms of clustering performance, while also effectively controlling the entropy of each row thus making it particularly robust to varying noise levels. Following, we present a new DR algorithm, SNEkhorn, that leverages this new affinity matrix. We show its clear superiority to state-of-the-art approaches with several indicators on both synthetic and real-world datasets.


page 1

page 2

page 3

page 4


Doubly-Stochastic Normalization of the Gaussian Kernel is Robust to Heteroskedastic Noise

A fundamental step in many data-analysis techniques is the construction ...

Taming Wild High Dimensional Text Data with a Fuzzy Lash

The bag of words (BOW) represents a corpus in a matrix whose elements ar...

LaneAF: Robust Multi-Lane Detection with Affinity Fields

This study presents an approach to lane detection involving the predicti...

A Probabilistic Graph Coupling View of Dimension Reduction

Most popular dimension reduction (DR) methods like t-SNE and UMAP are ba...

Path-entropy maximized Markov chains for dimensionality reduction

Stochastic kernel based dimensionality reduction methods have become pop...

A Latent Variable Model for Two-Dimensional Canonical Correlation Analysis and its Variational Inference

Describing the dimension reduction (DR) techniques by means of probabili...

Efficient and Robust Bayesian Selection of Hyperparameters in Dimension Reduction for Visualization

We introduce an efficient and robust auto-tuning framework for hyperpara...

Please sign up or login with your details

Forgot password? Click here to reset