Lower Bounds for the Minimum Mean-Square Error via Neural Network-based Estimation
share
The minimum mean-square error (MMSE) achievable by optimal estimation of a random variable Y∈ℝ given another random variable X∈ℝ^d is of much interest in a variety of statistical contexts. In this paper we propose two estimators for the MMSE, one based on a two-layer neural network and the other on a special three-layer neural network. We derive lower bounds for the MMSE based on the proposed estimators and the Barron constant of an appropriate function of the conditional expectation of Y given X. Furthermore, we derive a general upper bound for the Barron constant that, when X∈ℝ is post-processed by the additive Gaussian mechanism, produces order optimal estimates in the large noise regime.
READ FULL TEXT