Optimal Combination of Linear and Spectral Estimators for Generalized Linear Models
share
We study the problem of recovering an unknown signal x given measurements obtained from a generalized linear model with a Gaussian sensing matrix. Two popular solutions are based on a linear estimator x̂^ L and a spectral estimator x̂^ s. The former is a data-dependent linear combination of the columns of the measurement matrix, and its analysis is quite simple. The latter is the principal eigenvector of a data-dependent matrix, and a recent line of work has studied its performance. In this paper, we show how to optimally combine x̂^ L and x̂^ s. At the heart of our analysis is the exact characterization of the joint empirical distribution of ( x, x̂^ L, x̂^ s) in the high-dimensional limit. This allows us to compute the Bayes-optimal combination of x̂^ L and x̂^ s, given the limiting distribution of the signal x. When the distribution of the signal is Gaussian, then the Bayes-optimal combination has the form θx̂^ L+x̂^ s and we derive the optimal combination coefficient. In order to establish the limiting distribution of ( x, x̂^ L, x̂^ s), we design and analyze an Approximate Message Passing (AMP) algorithm whose iterates give x̂^ L and approach x̂^ s. Numerical simulations demonstrate the improvement of the proposed combination with respect to the two methods considered separately.
READ FULL TEXT