DeepAI AI Chat
Log In Sign Up

Distributed Estimation for Principal Component Analysis: a Gap-free Approach

by   Xi Chen, et al.

The growing size of modern data sets brings many challenges to the existing statistical estimation approaches, which calls for new distributed methodologies. This paper studies distributed estimation for a fundamental statistical machine learning problem, principal component analysis (PCA). Despite the massive literature on top eigenvector estimation, much less is presented for the top-L-dim (L > 1) eigenspace estimation, especially in a distributed manner. We propose a novel multi-round algorithm for constructing top-L-dim eigenspace for distributed data. Our algorithm takes advantage of shift-and-invert preconditioning and convex optimization. Our estimator is communication-efficient and achieves a fast convergence rate. In contrast to the existing divide-and-conquer algorithm, our approach has no restriction on the number of machines. Theoretically, we establish a gap-free error bound and abandon the assumption on the sharp eigengap between the L-th and the (L+1)-th eigenvalues. Our distributed algorithm can be applied to a wide range of statistical problems based on PCA. In particular, this paper illustrates two important applications, principal component regression and single index model, where our distributed algorithm can be extended. Finally, We provide simulation studies to demonstrate the performance of the proposed distributed estimator.


page 1

page 2

page 3

page 4


Distributed Robust Principal Component Analysis

We study the robust principal component analysis (RPCA) problem in a dis...

Distributed Inference for Linear Support Vector Machine

The growing size of modern data brings many new challenges to existing s...

Distributed Learning for Principle Eigenspaces without Moment Constraints

Distributed Principal Component Analysis (PCA) has been studied to deal ...

Distributed Estimation and Inference for Semi-parametric Binary Response Models

The development of modern technology has enabled data collection of unpr...

Communication-efficient distributed eigenspace estimation

Distributed computing is a standard way to scale up machine learning and...

An Improved Gap-Dependency Analysis of the Noisy Power Method

We consider the noisy power method algorithm, which has wide application...