Fast spectral algorithms from sum-of-squares proofs: tensor decomposition and planted sparse vectors

12/08/2015
by   Samuel B. Hopkins, et al.
0

We consider two problems that arise in machine learning applications: the problem of recovering a planted sparse vector in a random linear subspace and the problem of decomposing a random low-rank overcomplete 3-tensor. For both problems, the best known guarantees are based on the sum-of-squares method. We develop new algorithms inspired by analyses of the sum-of-squares method. Our algorithms achieve the same or similar guarantees as sum-of-squares for these problems but the running time is significantly faster. For the planted sparse vector problem, we give an algorithm with running time nearly linear in the input size that approximately recovers a planted sparse vector with up to constant relative sparsity in a random subspace of R^n of dimension up to Ω̃(√(n)). These recovery guarantees match the best known ones of Barak, Kelner, and Steurer (STOC 2014) up to logarithmic factors. For tensor decomposition, we give an algorithm with running time close to linear in the input size (with exponent ≈ 1.086) that approximately recovers a component of a random 3-tensor over R^n of rank up to Ω̃(n^4/3). The best previous algorithm for this problem due to Ge and Ma (RANDOM 2015) works up to rank Ω̃(n^3/2) but requires quasipolynomial time.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/27/2017

Fast and robust tensor decomposition with applications to dictionary learning

We develop fast spectral algorithms for tensor decomposition that match ...
research
07/06/2014

Dictionary Learning and Tensor Decomposition via the Sum-of-Squares Method

We give a new approach to the dictionary learning (also known as "sparse...
research
02/14/2022

Fast algorithm for overcomplete order-3 tensor decomposition

We develop the first fast spectral algorithm to decompose a random third...
research
01/05/2021

SoS Degree Reduction with Applications to Clustering and Robust Moment Estimation

We develop a general framework to significantly reduce the degree of sum...
research
03/05/2022

A Robust Spectral Algorithm for Overcomplete Tensor Decomposition

We give a spectral algorithm for decomposing overcomplete order-4 tensor...
research
07/12/2021

In-Database Regression in Input Sparsity Time

Sketching is a powerful dimensionality reduction technique for accelerat...
research
02/25/2018

A New Algorithm for Finding Closest Pair of Vectors

Given n vectors x_0, x_1, ..., x_n-1 in {0,1}^m, how to find two vectors...

Please sign up or login with your details

Forgot password? Click here to reset