The fundamental limits of sparse linear regression with sublinear sparsity

01/27/2021
by   Lan V. Truong, et al.
0

We establish exact asymptotic expressions for the normalized mutual information and minimum mean-square-error (MMSE) of sparse linear regression in the sub-linear sparsity regime. Our result is achieved by a simple generalization of the adaptive interpolation method in Bayesian inference for linear regimes to sub-linear ones. A modification of the well-known approximate message passing algorithm to approach the MMSE fundamental limit is also proposed. Our results show that the traditional linear assumption between the signal dimension and number of observations in the replica and adaptive interpolation methods is not necessary for sparse signals. They also show how to modify the existing well-known AMP algorithms for linear regimes to sub-linear ones.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

06/19/2020

Information theoretic limits of learning a sparse rule

We consider generalized linear models in regimes where the number of non...
06/14/2020

All-or-nothing statistical and computational phase transitions in sparse spiked matrix estimation

We determine statistical and computational limits for estimation of a ra...
02/04/2020

Sparse Polynomial Chaos Expansions: Literature Survey and Benchmark

Sparse polynomial chaos expansions are a popular surrogate modelling met...
11/12/2019

0-1 phase transitions in sparse spiked matrix estimation

We consider statistical models of estimation of a rank-one matrix (the s...
02/21/2013

Nonparametric Basis Pursuit via Sparse Kernel-based Learning

Signal processing tasks as fundamental as sampling, reconstruction, mini...
06/13/2018

Adaptive Path Interpolation for Sparse Systems: Application to a Simple Censored Block Model

A new adaptive path interpolation method has been recently developed as ...
09/28/2020

Replica Analysis of the Linear Model with Markov or Hidden Markov Signal Priors

This paper estimates free energy, average mutual information, and minimu...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.