Optimal Combination of Linear and Spectral Estimators for Generalized Linear Models

08/07/2020
by   Marco Mondelli, et al.
0

We study the problem of recovering an unknown signal x given measurements obtained from a generalized linear model with a Gaussian sensing matrix. Two popular solutions are based on a linear estimator x̂^ L and a spectral estimator x̂^ s. The former is a data-dependent linear combination of the columns of the measurement matrix, and its analysis is quite simple. The latter is the principal eigenvector of a data-dependent matrix, and a recent line of work has studied its performance. In this paper, we show how to optimally combine x̂^ L and x̂^ s. At the heart of our analysis is the exact characterization of the joint empirical distribution of ( x, x̂^ L, x̂^ s) in the high-dimensional limit. This allows us to compute the Bayes-optimal combination of x̂^ L and x̂^ s, given the limiting distribution of the signal x. When the distribution of the signal is Gaussian, then the Bayes-optimal combination has the form θx̂^ L+x̂^ s and we derive the optimal combination coefficient. In order to establish the limiting distribution of ( x, x̂^ L, x̂^ s), we design and analyze an Approximate Message Passing (AMP) algorithm whose iterates give x̂^ L and approach x̂^ s. Numerical simulations demonstrate the improvement of the proposed combination with respect to the two methods considered separately.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/07/2020

Approximate Message Passing with Spectral Initialization for Generalized Linear Models

We consider the problem of estimating a signal from measurements obtaine...
research
06/09/2023

Bayes optimal learning in high-dimensional linear regression with network side information

Supervised learning problems with side information in the form of a netw...
research
11/21/2022

Precise Asymptotics for Spectral Methods in Mixed Generalized Linear Models

In a mixed generalized linear model, the objective is to learn multiple ...
research
08/28/2023

Spectral Estimators for Structured Generalized Linear Models via Approximate Message Passing

We consider the problem of parameter estimation from observations given ...
research
10/03/2022

Bayes-optimal limits in structured PCA, and how to reach them

We study the paradigmatic spiked matrix model of principal components an...
research
08/27/2020

Approximate Message Passing algorithms for rotationally invariant matrices

Approximate Message Passing (AMP) algorithms have seen widespread use ac...
research
11/11/2018

Optimal Spectral Initialization for Signal Recovery with Applications to Phase Retrieval

We present the optimal design of a spectral method widely used to initia...

Please sign up or login with your details

Forgot password? Click here to reset