Generalized probabilistic principal component analysis of correlated data

by   Mengyang Gu, et al.

Principal component analysis (PCA) is a well-established tool in machine learning and data processing. tipping1999probabilistic proposed a probabilistic formulation of PCA (PPCA) by showing that the principal axes in PCA are equivalent to the maximum marginal likelihood estimator of the factor loading matrix in a latent factor model for the observed data, assuming that the latent factors are independently distributed as standard normal distributions. However, the independence assumption may be unrealistic for many scenarios such as modeling multiple time series, spatial processes, and functional data, where the output variables are correlated. In this paper, we introduce the generalized probabilistic principal component analysis (GPPCA) to study the latent factor model of multiple correlated outcomes, where each factor is modeled by a Gaussian process. The proposed method provides a probabilistic solution of the latent factor model with the scalable computation. In particular, we derive the maximum marginal likelihood estimator of the factor loading matrix and the predictive distribution of the output. Based on the explicit expression of the precision matrix in the marginal likelihood, the number of the computational operations is linear to the number of output variables. Moreover, with the use of the Matérn covariance function, the number of the computational operations is also linear to the number of time points for modeling the multiple time series without any approximation to the likelihood function. We discuss the connection of the GPPCA with other approaches such as the PCA and PPCA, and highlight the advantage of GPPCA in terms of the practical relevance, estimation accuracy and computational convenience. Numerical studies confirm the excellent finite-sample performance of the proposed approach.


page 1

page 2

page 3

page 4


MML Probabilistic Principal Component Analysis

Principal component analysis (PCA) is perhaps the most widely method for...

Multivariate time-series modeling with generative neural networks

Generative moment matching networks (GMMNs) are introduced as dependence...

Generalized Matrix Factorization

Unmeasured or latent variables are often the cause of correlations betwe...

AugmentedPCA: A Python Package of Supervised and Adversarial Linear Factor Models

Deep autoencoders are often extended with a supervised or adversarial lo...

Consistent estimation of high-dimensional factor models when the factor number is over-estimated

A high-dimensional r-factor model for an n-dimensional vector time serie...

Factor-guided functional PCA for high-dimensional functional data

The literature on high-dimensional functional data focuses on either the...

Residual Component Analysis

Probabilistic principal component analysis (PPCA) seeks a low dimensiona...

Please sign up or login with your details

Forgot password? Click here to reset