Sample Efficient Toeplitz Covariance Estimation

by   Yonina C. Eldar, et al.
Princeton University
Weizmann Institute of Science

We study the query complexity of estimating the covariance matrix T of a distribution D over d-dimensional vectors, under the assumption that T is Toeplitz. This assumption arises in many signal processing problems, where the covariance between any two measurements only depends on the time or distance between those measurements. We are interested in estimation strategies that may choose to view only a subset of entries in each vector sample x^(ℓ)∼D, which often equates to reducing hardware and communication requirements in applications ranging from wireless signal processing to advanced imaging. Our goal is to minimize both 1) the number of vector samples drawn from D and 2) the number of entries accessed in each sample. We provide some of the first non-asymptotic bounds on these sample complexity measures that exploit T's Toeplitz structure, and by doing so, significantly improve on results for generic covariance matrices. Our bounds follow from a novel analysis of classical and widely used estimation algorithms (along with some new variants), including methods based on selecting entries from each vector sample according to a so-called sparse ruler. In many cases, we pair our upper bounds with matching or nearly matching lower bounds. In addition to results that hold for any Toeplitz T, we further study the important setting when T is close to low-rank, which is often the case in practice. We show that methods based on sparse rulers perform even better in this setting, with sample complexity scaling sublinearly in d. Motivated by this finding, we develop a new covariance estimation strategy that further improves on all existing methods in the low-rank case: when T is rank-k or nearly rank-k, it achieves sample complexity depending polynomially on k and only logarithmically on d.


page 1

page 2

page 3

page 4


Estimation of Shortest Path Covariance Matrices

We study the sample complexity of estimating the covariance matrix Σ∈ℝ^d...

Low-Rank Toeplitz Matrix Estimation via Random Ultra-Sparse Rulers

We study how to estimate a nearly low-rank Toeplitz covariance matrix T ...

Estimation of the sample covariance matrix from compressive measurements

This paper focuses on the estimation of the sample covariance matrix fro...

Tight Sample Complexity of Large-Margin Learning

We obtain a tight distribution-specific characterization of the sample c...

Extreme Compressive Sampling for Covariance Estimation

This paper studies the problem of estimating the covariance of a collect...

Model Agnostic High-Dimensional Error-in-Variable Regression

We consider the problem of high-dimensional error-in-variable regression...

Almost sharp covariance and Wishart-type matrix estimation

Let X_1,..., X_n ∈ℝ^d be independent Gaussian random vectors with indepe...

Please sign up or login with your details

Forgot password? Click here to reset