Intrinsic dimension estimation of data by principal component analysis

02/10/2010
by   Mingyu Fan, et al.
0

Estimating intrinsic dimensionality of data is a classic problem in pattern recognition and statistics. Principal Component Analysis (PCA) is a powerful tool in discovering dimensionality of data sets with a linear structure; it, however, becomes ineffective when data have a nonlinear structure. In this paper, we propose a new PCA-based method to estimate intrinsic dimension of data with nonlinear structures. Our method works by first finding a minimal cover of the data set, then performing PCA locally on each subset in the cover and finally giving the estimation result by checking up the data variance on all small neighborhood regions. The proposed method utilizes the whole data set to estimate its intrinsic dimension and is convenient for incremental learning. In addition, our new PCA procedure can filter out noise in data and converge to a stable estimation with the neighborhood region size increasing. Experiments on synthetic and real world data sets show effectiveness of the proposed method.

READ FULL TEXT

page 1

page 2

page 3

page 4

page 5

page 6

page 7

page 8

research
07/02/2012

Robust Principal Component Analysis Using Statistical Estimators

Principal Component Analysis (PCA) finds a linear mapping and maximizes ...
research
12/20/2019

Big Data Approaches to Knot Theory: Understanding the Structure of the Jones Polynomial

We examine the structure and dimensionality of the Jones polynomial usin...
research
06/25/2021

Self-paced Principal Component Analysis

Principal Component Analysis (PCA) has been widely used for dimensionali...
research
09/26/2022

On Projections to Linear Subspaces

The merit of projecting data onto linear subspaces is well known from, e...
research
07/13/2018

Conditional Masking to Numerical Data

Protecting the privacy of data-sets has become hugely important these da...
research
11/29/2022

Approximating Intersections and Differences Between Statistical Shape Models

To date, the comparison of Statistical Shape Models (SSMs) is often sole...
research
11/30/2012

A recursive divide-and-conquer approach for sparse principal component analysis

In this paper, a new method is proposed for sparse PCA based on the recu...

Please sign up or login with your details

Forgot password? Click here to reset