LLE with low-dimensional neighborhood representation

08/06/2008
by   Yair Goldberg, et al.
0

The local linear embedding algorithm (LLE) is a non-linear dimension-reducing technique, widely used due to its computational simplicity and intuitive approach. LLE first linearly reconstructs each input point from its nearest neighbors and then preserves these neighborhood relations in the low-dimensional embedding. We show that the reconstruction weights computed by LLE capture the high-dimensional structure of the neighborhoods, and not the low-dimensional manifold structure. Consequently, the weight vectors are highly sensitive to noise. Moreover, this causes LLE to converge to a linear projection of the input, as opposed to its non-linear embedding goal. To overcome both of these problems, we propose to compute the weight vectors using a low-dimensional neighborhood representation. We prove theoretically that this straightforward and computationally simple modification of LLE reduces LLE's sensitivity to noise. This modification also removes the need for regularization when the number of neighbors is larger than the dimension of the input. We present numerical examples demonstrating both the perturbation and linear projection problems, and the improved outputs using the low-dimensional neighborhood representation.

READ FULL TEXT

page 6

page 13

research
08/23/2016

On Clustering and Embedding Mixture Manifolds using a Low Rank Neighborhood Approach

Samples from intimate (non-linear) mixtures are generally modeled as bei...
research
08/28/2021

Avoiding unwanted results in locally linear embedding: A new understanding of regularization

We demonstrate that locally linear embedding (LLE) inherently admits som...
research
09/09/2017

Simultaneously Learning Neighborship and Projection Matrix for Supervised Dimensionality Reduction

Explicitly or implicitly, most of dimensionality reduction methods need ...
research
03/16/2020

G-LBM:Generative Low-dimensional Background Model Estimation from Video Sequences

In this paper, we propose a computationally tractable and theoretically ...
research
03/01/2018

Minimax rates for cost-sensitive learning on manifolds with approximate nearest neighbours

We study the approximate nearest neighbour method for cost-sensitive cla...
research
09/22/2021

The Curse Revisited: a Newly Quantified Concept of Meaningful Distances for Learning from High-Dimensional Noisy Data

Distances between data points are widely used in point cloud representat...
research
09/22/2021

Index t-SNE: Tracking Dynamics of High-Dimensional Datasets with Coherent Embeddings

t-SNE is an embedding method that the data science community has widely ...

Please sign up or login with your details

Forgot password? Click here to reset