Avoiding unwanted results in locally linear embedding: A new understanding of regularization

08/28/2021
by   Liren Lin, et al.
0

We demonstrate that locally linear embedding (LLE) inherently admits some unwanted results when no regularization is used, even for cases in which regularization is not supposed to be needed in the original algorithm. The existence of one special type of result, which we call “projection pattern”, is mathematically proved in the situation that an exact local linear relation is achieved in each neighborhood of the data. These special patterns as well as some other bizarre results that may occur in more general situations are shown by numerical examples on the Swiss roll with a hole embedded in a high dimensional space. It is observed that all these bad results can be effectively prevented by using regularization.

READ FULL TEXT
research
08/06/2008

LLE with low-dimensional neighborhood representation

The local linear embedding algorithm (LLE) is a non-linear dimension-red...
research
12/16/2021

A new locally linear embedding scheme in light of Hessian eigenmap

We provide a new interpretation of Hessian locally linear embedding (HLL...
research
03/15/2021

A new interpretation of (Tikhonov) regularization

Tikhonov regularization with square-norm penalty for linear forward oper...
research
03/29/2020

High-dimensional Neural Feature using Rectified Linear Unit and Random Matrix Instance

We design a ReLU-based multilayer neural network to generate a rich high...
research
09/22/2021

Index t-SNE: Tracking Dynamics of High-Dimensional Datasets with Coherent Embeddings

t-SNE is an embedding method that the data science community has widely ...
research
10/10/2018

The Andoni--Krauthgamer--Razenshteyn characterization of sketchable norms fails for sketchable metrics

Andoni, Krauthgamer and Razenshteyn (AKR) proved (STOC 2015) that a fini...
research
06/10/2021

A Variational View on Statistical Multiscale Estimation

We present a unifying view on various statistical estimation techniques ...

Please sign up or login with your details

Forgot password? Click here to reset