Entropy of the Conditional Expectation under Gaussian Noise

by   Arda Atalik, et al.

This paper considers an additive Gaussian noise channel with arbitrarily distributed finite variance input signals. It studies the differential entropy of the minimum mean-square error (MMSE) estimator and provides a new lower bound which connects the entropy of the input, output, and conditional mean. That is, the sum of entropies of the conditional mean and output is always greater than or equal to twice the input entropy. Various other properties such as upper bounds, asymptotics, Taylor series expansion, and connection to Fisher Information are obtained. An application of the lower bound in the remote-source coding problem is discussed, and extensions of the lower and upper bounds to the vector Gaussian channel are given.



There are no comments yet.


page 1

page 2

page 3

page 4


Remote Source Coding under Gaussian Noise : Dueling Roles of Power and Entropy Power

The distributed remote source coding (so-called CEO) problem is studied ...

On the MMSE Estimation of Norm of a Gaussian Vector under Additive White Gaussian Noise with Randomly Missing Input Entries

This paper considers the task of estimating the l_2 norm of a n-dimensio...

Improved Target Acquisition Rates with Feedback Codes

This paper considers the problem of acquiring an unknown target location...

On the Conditional Smooth Renyi Entropy and its Applications in Guessing and Source Coding

A novel definition of the conditional smooth Renyi entropy, which is dif...

Nonparametric Estimation of the Fisher Information and Its Applications

This paper considers the problem of estimation of the Fisher information...

Tight MMSE Bounds for the AGN Channel Under KL Divergence Constraints on the Input Distribution

Tight bounds on the minimum mean square error for the additive Gaussian ...

An Upper Bound on the Error Induced by Saddlepoint Approximations – Applications to Information Theory

This paper introduces an upper bound on the absolute difference between:...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.