Optimal Combination of Linear and Spectral Estimators for Generalized Linear Models

by   Marco Mondelli, et al.

We study the problem of recovering an unknown signal x given measurements obtained from a generalized linear model with a Gaussian sensing matrix. Two popular solutions are based on a linear estimator x̂^ L and a spectral estimator x̂^ s. The former is a data-dependent linear combination of the columns of the measurement matrix, and its analysis is quite simple. The latter is the principal eigenvector of a data-dependent matrix, and a recent line of work has studied its performance. In this paper, we show how to optimally combine x̂^ L and x̂^ s. At the heart of our analysis is the exact characterization of the joint empirical distribution of ( x, x̂^ L, x̂^ s) in the high-dimensional limit. This allows us to compute the Bayes-optimal combination of x̂^ L and x̂^ s, given the limiting distribution of the signal x. When the distribution of the signal is Gaussian, then the Bayes-optimal combination has the form θx̂^ L+x̂^ s and we derive the optimal combination coefficient. In order to establish the limiting distribution of ( x, x̂^ L, x̂^ s), we design and analyze an Approximate Message Passing (AMP) algorithm whose iterates give x̂^ L and approach x̂^ s. Numerical simulations demonstrate the improvement of the proposed combination with respect to the two methods considered separately.



There are no comments yet.


page 1

page 2

page 3

page 4


Approximate Message Passing with Spectral Initialization for Generalized Linear Models

We consider the problem of estimating a signal from measurements obtaine...

Estimation in Rotationally Invariant Generalized Linear Models via Approximate Message Passing

We consider the problem of signal estimation in generalized linear model...

Fundamental Limits of Weak Recovery with Applications to Phase Retrieval

In phase retrieval we want to recover an unknown signal x∈ C^d from n q...

PCA Initialization for Approximate Message Passing in Rotationally Invariant Models

We study the problem of estimating a rank-1 signal in the presence of ro...

Generalized Memory Approximate Message Passing

Generalized approximate message passing (GAMP) is a promising technique ...

Optimal Sampling for Generalized Linear Models under Measurement Constraints

Suppose we are using a generalized linear model to predict a scalar outc...

Optimal Spectral Initialization for Signal Recovery with Applications to Phase Retrieval

We present the optimal design of a spectral method widely used to initia...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.