Optimal Combination of Linear and Spectral Estimators for Generalized Linear Models

08/07/2020
by   Marco Mondelli, et al.
0

We study the problem of recovering an unknown signal x given measurements obtained from a generalized linear model with a Gaussian sensing matrix. Two popular solutions are based on a linear estimator x̂^ L and a spectral estimator x̂^ s. The former is a data-dependent linear combination of the columns of the measurement matrix, and its analysis is quite simple. The latter is the principal eigenvector of a data-dependent matrix, and a recent line of work has studied its performance. In this paper, we show how to optimally combine x̂^ L and x̂^ s. At the heart of our analysis is the exact characterization of the joint empirical distribution of ( x, x̂^ L, x̂^ s) in the high-dimensional limit. This allows us to compute the Bayes-optimal combination of x̂^ L and x̂^ s, given the limiting distribution of the signal x. When the distribution of the signal is Gaussian, then the Bayes-optimal combination has the form θx̂^ L+x̂^ s and we derive the optimal combination coefficient. In order to establish the limiting distribution of ( x, x̂^ L, x̂^ s), we design and analyze an Approximate Message Passing (AMP) algorithm whose iterates give x̂^ L and approach x̂^ s. Numerical simulations demonstrate the improvement of the proposed combination with respect to the two methods considered separately.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

10/07/2020

Approximate Message Passing with Spectral Initialization for Generalized Linear Models

We consider the problem of estimating a signal from measurements obtaine...
12/08/2021

Estimation in Rotationally Invariant Generalized Linear Models via Approximate Message Passing

We consider the problem of signal estimation in generalized linear model...
08/20/2017

Fundamental Limits of Weak Recovery with Applications to Phase Retrieval

In phase retrieval we want to recover an unknown signal x∈ C^d from n q...
06/04/2021

PCA Initialization for Approximate Message Passing in Rotationally Invariant Models

We study the problem of estimating a rank-1 signal in the presence of ro...
10/12/2021

Generalized Memory Approximate Message Passing

Generalized approximate message passing (GAMP) is a promising technique ...
07/17/2019

Optimal Sampling for Generalized Linear Models under Measurement Constraints

Suppose we are using a generalized linear model to predict a scalar outc...
11/11/2018

Optimal Spectral Initialization for Signal Recovery with Applications to Phase Retrieval

We present the optimal design of a spectral method widely used to initia...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.