DeepAI AI Chat
Log In Sign Up

Sparse Generalized Principal Component Analysis for Large-scale Applications beyond Gaussianity

12/12/2015
by   Qiaoya Zhang, et al.
0

Principal Component Analysis (PCA) is a dimension reduction technique. It produces inconsistent estimators when the dimensionality is moderate to high, which is often the problem in modern large-scale applications where algorithm scalability and model interpretability are difficult to achieve, not to mention the prevalence of missing values. While existing sparse PCA methods alleviate inconsistency, they are constrained to the Gaussian assumption of classical PCA and fail to address algorithm scalability issues. We generalize sparse PCA to the broad exponential family distributions under high-dimensional setup, with built-in treatment for missing values. Meanwhile we propose a family of iterative sparse generalized PCA (SG-PCA) algorithms such that despite the non-convexity and non-smoothness of the optimization task, the loss function decreases in every iteration. In terms of ease and intuitive parameter tuning, our sparsity-inducing regularization is far superior to the popular Lasso. Furthermore, to promote overall scalability, accelerated gradient is integrated for fast convergence, while a progressive screening technique gradually squeezes out nuisance dimensions of a large-scale problem for feasible optimization. High-dimensional simulation and real data experiments demonstrate the efficiency and efficacy of SG-PCA.

READ FULL TEXT

page 1

page 2

page 3

page 4

07/03/2019

Generalized Principal Component Analysis

Generalized principal component analysis (GLM-PCA) facilitates dimension...
03/27/2019

An Alternating Manifold Proximal Gradient Method for Sparse PCA and Sparse CCA

Sparse principal component analysis (PCA) and sparse canonical correlati...
01/30/2021

Spike and slab Bayesian sparse principal component analysis

Sparse principal component analysis (PCA) is a popular tool for dimensio...
07/02/2020

High Dimensional Bayesian Optimization Assisted by Principal Component Analysis

Bayesian Optimization (BO) is a surrogate-assisted global optimization t...
11/08/2017

Penalized Orthogonal Iteration for Sparse Estimation of Generalized Eigenvalue Problem

We propose a new algorithm for sparse estimation of eigenvectors in gene...
10/16/2018

Fast Randomized PCA for Sparse Data

Principal component analysis (PCA) is widely used for dimension reductio...
09/16/2020

PCA Reduced Gaussian Mixture Models with Applications in Superresolution

Despite the rapid development of computational hardware, the treatment o...