Randomly Initialized Alternating Least Squares: Fast Convergence for Matrix Sensing

04/25/2022
by   Kiryung Lee, et al.
0

We consider the problem of reconstructing rank-one matrices from random linear measurements, a task that appears in a variety of problems in signal processing, statistics, and machine learning. In this paper, we focus on the Alternating Least Squares (ALS) method. While this algorithm has been studied in a number of previous works, most of them only show convergence from an initialization close to the true solution and thus require a carefully designed initialization scheme. However, random initialization has often been preferred by practitioners as it is model-agnostic. In this paper, we show that ALS with random initialization converges to the true solution with ε-accuracy in O(log n + log (1/ε)) iterations using only a near-optimal amount of samples, where we assume the measurement matrices to be i.i.d. Gaussian and where by n we denote the ambient dimension. Key to our proof is the observation that the trajectory of the ALS iterates only depends very mildly on certain entries of the random measurement matrices. Numerical experiments corroborate our theoretical predictions.

READ FULL TEXT

Authors

page 21

05/25/2018

Guaranteed Simultaneous Asymmetric Tensor Decomposition via Orthogonalized Alternating Least Squares

We consider the asymmetric orthogonal tensor decomposition problem, and ...
12/31/2020

Fast Global Convergence for Low-rank Matrix Recovery via Riemannian Gradient Descent with Random Initialization

In this paper, we propose a new global analysis framework for a class of...
11/25/2019

The Epsilon-Alternating Least Squares for Orthogonal Low-Rank Tensor Approximation and Its Global Convergence

The epsilon alternating least squares (ϵ-ALS) is developed and analyzed ...
07/12/2021

Quantile-Based Random Kaczmarz for corrupted linear systems of equations

We consider linear systems Ax = b where A ∈ℝ^m × n consists of normalize...
03/07/2019

Rigorous Analysis of Spectral Methods for Random Orthogonal Matrices

Phase retrieval refers to algorithmic methods for recovering a signal fr...
03/06/2019

Spectral Method for Phase Retrieval: an Expectation Propagation Perspective

Phase retrieval refers to the problem of recovering a signal x_∈C^n from...
11/20/2018

Analytic Network Learning

Based on the property that solving the system of linear matrix equations...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.