The Kikuchi Hierarchy and Tensor PCA

04/08/2019
by   Alexander S. Wein, et al.
0

For the tensor PCA (principal component analysis) problem, we propose a new hierarchy of algorithms that are increasingly powerful yet require increasing runtime. Our hierarchy is analogous to the sum-of-squares (SOS) hierarchy but is instead inspired by statistical physics and related algorithms such as belief propagation and AMP (approximate message passing). Our level-ℓ algorithm can be thought of as a (linearized) message-passing algorithm that keeps track of ℓ-wise dependencies among the hidden variables. Specifically, our algorithms are spectral methods based on the Kikuchi Hessian matrix, which generalizes the well-studied Bethe Hessian matrix to the higher-order Kikuchi free energies. It is known that AMP, the flagship algorithm of statistical physics, has substantially worse performance than SOS for tensor PCA. In this work we `redeem' the statistical physics approach by showing that our hierarchy gives a polynomial-time algorithm matching the performance of SOS. Our hierarchy also yields a continuum of subexponential-time algorithms, and we prove that these achieve the same (conjecturally optimal) tradeoff between runtime and statistical power as SOS. Our results hold for even-order tensors, and we conjecture that they also hold for odd-order tensors. Our methods suggest a new avenue for systematically obtaining optimal algorithms for Bayesian inference problems, and our results constitute a step toward unifying the statistical physics and sum-of-squares approaches to algorithm design.

READ FULL TEXT
research
04/21/2015

Decomposing Overcomplete 3rd Order Tensors using Sum-of-Squares Algorithms

Tensor rank and low-rank tensor decompositions have many applications in...
research
05/05/2021

A unifying tutorial on Approximate Message Passing

Over the last decade or so, Approximate Message Passing (AMP) algorithms...
research
10/13/2017

The power of sum-of-squares for detecting hidden structures

We study planted problems---finding hidden structures in random noisy in...
research
06/25/2013

Fourier PCA and Robust Tensor Decomposition

Fourier PCA is Principal Component Analysis of a matrix obtained from hi...
research
11/04/2014

A statistical model for tensor PCA

We consider the Principal Component Analysis problem for large tensors o...
research
07/30/2019

Classical and Quantum Algorithms for Tensor Principal Component Analysis

We present classical and quantum algorithms based on spectral methods fo...
research
07/05/2022

Hierarchies of Minion Tests for PCSPs through Tensors

We provide a unified framework to study hierarchies of relaxations for C...

Please sign up or login with your details

Forgot password? Click here to reset