A Unifying Probabilistic Perspective for Spectral Dimensionality Reduction: Insights and New Models

10/22/2010
by   Neil D. Lawrence, et al.
0

We introduce a new perspective on spectral dimensionality reduction which views these methods as Gaussian Markov random fields (GRFs). Our unifying perspective is based on the maximum entropy principle which is in turn inspired by maximum variance unfolding. The resulting model, which we call maximum entropy unfolding (MEU) is a nonlinear generalization of principal component analysis. We relate the model to Laplacian eigenmaps and isomap. We show that parameter fitting in the locally linear embedding (LLE) is approximate maximum likelihood MEU. We introduce a variant of LLE that performs maximum likelihood exactly: Acyclic LLE (ALLE). We show that MEU and ALLE are competitive with the leading spectral approaches on a robot navigation visualization and a human motion capture data set. Finally the maximum likelihood perspective allows us to introduce a new approach to dimensionality reduction based on L1 regularization of the Gaussian random field via the graphical lasso.

READ FULL TEXT

page 22

page 23

research
11/10/2016

Policy Search with High-Dimensional Context Variables

Direct contextual policy search methods learn to improve policy paramete...
research
06/11/2015

Random Maxout Features

In this paper, we propose and study random maxout features, which are co...
research
01/22/2016

A Mathematical Formalization of Hierarchical Temporal Memory's Spatial Pooler

Hierarchical temporal memory (HTM) is an emerging machine learning algor...
research
08/31/2012

On the convergence of maximum variance unfolding

Maximum Variance Unfolding is one of the main methods for (nonlinear) di...
research
06/17/2021

Disentangling Identifiable Features from Noisy Data with Structured Nonlinear ICA

We introduce a new general identifiable framework for principled disenta...
research
09/03/2019

Mixture Probabilistic Principal GeodesicAnalysis

Dimensionality reduction on Riemannian manifolds is challenging due to t...
research
10/09/2015

New Optimisation Methods for Machine Learning

A thesis submitted for the degree of Doctor of Philosophy of The Austral...

Please sign up or login with your details

Forgot password? Click here to reset