The fundamental limits of sparse linear regression with sublinear sparsity

01/27/2021
by   Lan V. Truong, et al.
0

We establish exact asymptotic expressions for the normalized mutual information and minimum mean-square-error (MMSE) of sparse linear regression in the sub-linear sparsity regime. Our result is achieved by a simple generalization of the adaptive interpolation method in Bayesian inference for linear regimes to sub-linear ones. A modification of the well-known approximate message passing algorithm to approach the MMSE fundamental limit is also proposed. Our results show that the traditional linear assumption between the signal dimension and number of observations in the replica and adaptive interpolation methods is not necessary for sparse signals. They also show how to modify the existing well-known AMP algorithms for linear regimes to sub-linear ones.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/19/2020

Information theoretic limits of learning a sparse rule

We consider generalized linear models in regimes where the number of non...
research
06/02/2023

Matrix Inference in Growing Rank Regimes

The inference of a large symmetric signal-matrix 𝐒∈ℝ^N× N corrupted by a...
research
02/04/2020

Sparse Polynomial Chaos Expansions: Literature Survey and Benchmark

Sparse polynomial chaos expansions are a popular surrogate modelling met...
research
11/12/2019

0-1 phase transitions in sparse spiked matrix estimation

We consider statistical models of estimation of a rank-one matrix (the s...
research
04/05/2023

Mixed Regression via Approximate Message Passing

We study the problem of regression in a generalized linear model (GLM) w...
research
07/15/2019

Concentration of the matrix-valued minimum mean-square error in optimal Bayesian inference

We consider Bayesian inference of signals with vector-valued entries. Ex...
research
02/21/2013

Nonparametric Basis Pursuit via Sparse Kernel-based Learning

Signal processing tasks as fundamental as sampling, reconstruction, mini...

Please sign up or login with your details

Forgot password? Click here to reset