Concentration of the matrix-valued minimum mean-square error in optimal Bayesian inference

07/15/2019
by   Jean Barbier, et al.
0

We consider Bayesian inference of signals with vector-valued entries. Extending concentration techniques from the mathematical physics of spin glasses, we show that the matrix-valued minimum mean-square error concentrates when the size of the problem increases. Such results are often crucial for proving single-letter formulas for the mutual information when they exist. Our proof is valid in the optimal Bayesian inference setting, meaning that it relies on the assumption that the model and all its hyper-parameters are known. Examples of inference and learning problems covered by our results are spiked matrix and tensor models, the committee machine neural network with few hidden neurons in the teacher-student scenario, or multi-layers generalized linear models.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
04/04/2019

Overlap matrix concentration in optimal Bayesian inference

We consider models of Bayesian inference of signals with vectorial compo...
research
05/16/2020

Information-theoretic limits of a multiview low-rank symmetric spiked matrix model

We consider a generalization of an important class of high-dimensional i...
research
01/25/2021

Finding hidden-feature depending laws inside a data set and classifying it using Neural Network

The logcosh loss function for neural networks has been developed to comb...
research
06/19/2020

Information theoretic limits of learning a sparse rule

We consider generalized linear models in regimes where the number of non...
research
01/27/2021

The fundamental limits of sparse linear regression with sublinear sparsity

We establish exact asymptotic expressions for the normalized mutual info...
research
05/06/2020

Strong replica symmetry in high-dimensional optimal Bayesian inference

We consider generic optimal Bayesian inference, namely, models of signal...
research
09/14/2021

Statistical limits of dictionary learning: random matrix theory and the spectral replica method

We consider increasingly complex models of matrix denoising and dictiona...

Please sign up or login with your details

Forgot password? Click here to reset