Distributed Estimation for Principal Component Analysis: a Gap-free Approach

04/05/2020
by   Xi Chen, et al.
10

The growing size of modern data sets brings many challenges to the existing statistical estimation approaches, which calls for new distributed methodologies. This paper studies distributed estimation for a fundamental statistical machine learning problem, principal component analysis (PCA). Despite the massive literature on top eigenvector estimation, much less is presented for the top-L-dim (L > 1) eigenspace estimation, especially in a distributed manner. We propose a novel multi-round algorithm for constructing top-L-dim eigenspace for distributed data. Our algorithm takes advantage of shift-and-invert preconditioning and convex optimization. Our estimator is communication-efficient and achieves a fast convergence rate. In contrast to the existing divide-and-conquer algorithm, our approach has no restriction on the number of machines. Theoretically, we establish a gap-free error bound and abandon the assumption on the sharp eigengap between the L-th and the (L+1)-th eigenvalues. Our distributed algorithm can be applied to a wide range of statistical problems based on PCA. In particular, this paper illustrates two important applications, principal component regression and single index model, where our distributed algorithm can be extended. Finally, We provide simulation studies to demonstrate the performance of the proposed distributed estimator.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

05/06/2020

A Communication-Efficient Distributed Algorithm for Kernel Principal Component Analysis

Principal Component Analysis (PCA) is a fundamental technology in machin...
11/29/2018

Distributed Inference for Linear Support Vector Machine

The growing size of modern data brings many new challenges to existing s...
09/05/2020

Communication-efficient distributed eigenspace estimation

Distributed computing is a standard way to scale up machine learning and...
02/23/2016

An Improved Gap-Dependency Analysis of the Noisy Power Method

We consider the noisy power method algorithm, which has wide application...
11/04/2019

Visualization of Multi-Objective Switched Reluctance Machine Optimization at Multiple Operating Conditions with t-SNE

The optimization of electric machines at multiple operating points is cr...
10/28/2017

A Geometric Perspective on the Power of Principal Component Association Tests in Multiple Phenotype Studies

Joint analysis of multiple phenotypes can increase statistical power in ...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.