Spectral Methods for Data Science: A Statistical Perspective

12/15/2020
by   Yuxin Chen, et al.
18

Spectral methods have emerged as a simple yet surprisingly effective approach for extracting information from massive, noisy and incomplete data. In a nutshell, spectral methods refer to a collection of algorithms built upon the eigenvalues (resp. singular values) and eigenvectors (resp. singular vectors) of some properly designed matrices constructed from data. A diverse array of applications have been found in machine learning, data science, and signal processing. Due to their simplicity and effectiveness, spectral methods are not only used as a stand-alone estimator, but also frequently employed to initialize other more sophisticated algorithms to improve performance. While the studies of spectral methods can be traced back to classical matrix perturbation theory and methods of moments, the past decade has witnessed tremendous theoretical advances in demystifying their efficacy through the lens of statistical modeling, with the aid of non-asymptotic random matrix theory. This monograph aims to present a systematic, comprehensive, yet accessible introduction to spectral methods from a modern statistical perspective, highlighting their algorithmic implications in diverse large-scale applications. In particular, our exposition gravitates around several central questions that span various applications: how to characterize the sample efficiency of spectral methods in reaching a target level of statistical accuracy, and how to assess their stability in the face of random noise, missing data, and adversarial corruptions? In addition to conventional ℓ_2 perturbation analysis, we present a systematic ℓ_∞ and ℓ_2,∞ perturbation theory for eigenspace and singular subspaces, which has only recently become available owing to a powerful "leave-one-out" analysis framework.

READ FULL TEXT

page 7

page 8

05/30/2022

Leave-one-out Singular Subspace Perturbation Analysis for Spectral Clustering

The singular subspaces perturbation theory is of fundamental importance ...
02/24/2021

Modern Koopman Theory for Dynamical Systems

The field of dynamical systems is being transformed by the mathematical ...
09/11/2019

Unified ℓ_2→∞ Eigenspace Perturbation Theory for Symmetric Random Matrices

Modern applications in statistics, computer science and network science ...
03/19/2022

Perturbation Analysis of Randomized SVD and its Applications to High-dimensional Statistics

Randomized singular value decomposition (RSVD) is a class of computation...
11/16/2020

An exact sinΘ formula for matrix perturbation analysis and its applications

Singular vector perturbation is an important topic in numerical analysis...
09/09/2016

Robust Spectral Detection of Global Structures in the Data by Learning a Regularization

Spectral methods are popular in detecting global structures in the given...
12/01/2020

Spectral Analysis of Word Statistics

Given a random text over a finite alphabet, we study the frequencies at ...