
A CommunicationEfficient Distributed Algorithm for Kernel Principal Component Analysis
Principal Component Analysis (PCA) is a fundamental technology in machin...
read it

Distributed Estimation for Principal Component Analysis: a Gapfree Approach
The growing size of modern data sets brings many challenges to the exist...
read it

Rank/Norm Regularization with ClosedForm Solutions: Application to Subspace Clustering
When data is sampled from an unknown subspace, principal component analy...
read it

Federated PCA with Adaptive Rank Estimation
In many online machine learning and data science tasks such as data summ...
read it

Scale Invariant Power Iteration
Power iteration has been generalized to solve many interesting problems ...
read it

Avoiding Communication in Proximal Methods for Convex Optimization Problems
The fast iterative soft thresholding algorithm (FISTA) is used to solve ...
read it

Incrementally Updated Spectral Embeddings
Several fundamental tasks in data science rely on computing an extremal ...
read it
Communicationefficient distributed eigenspace estimation
Distributed computing is a standard way to scale up machine learning and data science algorithms to process large amounts of data. In such settings, avoiding communication amongst machines is paramount for achieving high performance. Rather than distribute the computation of existing algorithms, a common practice for avoiding communication is to compute local solutions or parameter estimates on each machine and then combine the results; in many convex optimization problems, even simple averaging of local solutions can work well. However, these schemes do not work when the local solutions are not unique. Spectral methods are a collection of such problems, where solutions are orthonormal bases of the leading invariant subspace of an associated data matrix, which are only unique up to rotation and reflections. Here, we develop a communicationefficient distributed algorithm for computing the leading invariant subspace of a data matrix. Our algorithm uses a novel alignment scheme that minimizes the Procrustean distance between local solutions and a reference solution, and only requires a single round of communication. For the important case of principal component analysis (PCA), we show that our algorithm achieves a similar error rate to that of a centralized estimator. We present numerical experiments demonstrating the efficacy of our proposed algorithm for distributed PCA, as well as other problems where solutions exhibit rotational symmetry, such as node embeddings for graph data and spectral initialization for quadratic sensing.
READ FULL TEXT
Comments
There are no comments yet.