A Distributed Frank-Wolfe Framework for Learning Low-Rank Matrices with the Trace Norm

12/20/2017
by   Wenjie Zheng, et al.
0

We consider the problem of learning a high-dimensional but low-rank matrix from a large-scale dataset distributed over several machines, where low-rankness is enforced by a convex trace norm constraint. We propose DFW-Trace, a distributed Frank-Wolfe algorithm which leverages the low-rank structure of its updates to achieve efficiency in time, memory and communication usage. The step at the heart of DFW-Trace is solved approximately using a distributed version of the power method. We provide a theoretical analysis of the convergence of DFW-Trace, showing that we can ensure sublinear convergence in expectation to an optimal solution with few power iterations per epoch. We implement DFW-Trace in the Apache Spark distributed programming framework and validate the usefulness of our approach on synthetic and real data, including the ImageNet dataset with high-dimensional features extracted from a deep neural network.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
02/18/2011

Concentration-Based Guarantees for Low-Rank Matrix Reconstruction

We consider the problem of approximately reconstructing a partially-obse...
research
06/27/2017

Reexamining Low Rank Matrix Factorization for Trace Norm Regularization

Trace norm regularization is a widely used approach for learning low ran...
research
01/04/2015

A generalization error bound for sparse and low-rank multivariate Hawkes processes

We consider the problem of unveiling the implicit network structure of u...
research
11/29/2021

Rank-Constrained Least-Squares: Prediction and Inference

In this work, we focus on the high-dimensional trace regression model wi...
research
08/15/2023

Analysis of stochastic probing methods for estimating the trace of functions of sparse symmetric matrices

We consider the problem of estimating the trace of a matrix function f(A...
research
07/02/2020

Partial Trace Regression and Low-Rank Kraus Decomposition

The trace regression model, a direct extension of the well-studied linea...

Please sign up or login with your details

Forgot password? Click here to reset