Entropy of the Conditional Expectation under Gaussian Noise

06/08/2021
by   Arda Atalik, et al.
0

This paper considers an additive Gaussian noise channel with arbitrarily distributed finite variance input signals. It studies the differential entropy of the minimum mean-square error (MMSE) estimator and provides a new lower bound which connects the entropy of the input, output, and conditional mean. That is, the sum of entropies of the conditional mean and output is always greater than or equal to twice the input entropy. Various other properties such as upper bounds, asymptotics, Taylor series expansion, and connection to Fisher Information are obtained. An application of the lower bound in the remote-source coding problem is discussed, and extensions of the lower and upper bounds to the vector Gaussian channel are given.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

05/16/2018

Remote Source Coding under Gaussian Noise : Dueling Roles of Power and Entropy Power

The distributed remote source coding (so-called CEO) problem is studied ...
10/16/2020

On the MMSE Estimation of Norm of a Gaussian Vector under Additive White Gaussian Noise with Randomly Missing Input Entries

This paper considers the task of estimating the l_2 norm of a n-dimensio...
12/15/2017

Improved Target Acquisition Rates with Feedback Codes

This paper considers the problem of acquiring an unknown target location...
10/22/2018

On the Conditional Smooth Renyi Entropy and its Applications in Guessing and Source Coding

A novel definition of the conditional smooth Renyi entropy, which is dif...
05/07/2020

Nonparametric Estimation of the Fisher Information and Its Applications

This paper considers the problem of estimation of the Fisher information...
04/26/2018

Tight MMSE Bounds for the AGN Channel Under KL Divergence Constraints on the Input Distribution

Tight bounds on the minimum mean square error for the additive Gaussian ...
07/10/2020

An Upper Bound on the Error Induced by Saddlepoint Approximations – Applications to Information Theory

This paper introduces an upper bound on the absolute difference between:...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.