Tight Bounds on the Rényi Entropy via Majorization with Applications to Guessing and Compression

12/08/2018
by   Igal Sason, et al.
0

This paper provides tight bounds on the Rényi entropy of a function of a discrete random variable with a finite number of possible values, where the considered function is not one-to-one. To that end, a tight lower bound on the Rényi entropy of a discrete random variable with a finite support is derived as a function of the size of the support, and the ratio of the maximal to minimal probability masses. This work was inspired by the recently published paper by Cicalese et al., which is focused on the Shannon entropy, and it strengthens and generalizes the results of that paper to Rényi entropies of arbitrary positive orders. In view of these generalized bounds and the works by Arikan and Campbell, non-asymptotic bounds are derived for guessing moments and lossless data compression of discrete memoryless sources.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
03/31/2021

On Strong Data-Processing and Majorization Inequalities with Applications to Coding Problems

This work provides data-processing and majorization inequalities for f-d...
research
12/21/2017

Bounds on the Entropy of a Function of a Random Variable and their Applications

It is well known that the entropy H(X) of a discrete random variable X i...
research
01/04/2018

Improved Bounds on Lossless Source Coding and Guessing Moments via Rényi Measures

This paper provides upper and lower bounds on the optimal guessing momen...
research
05/29/2018

The deficit in an entropic inequality

In this article, we investigate the entropy of a sum of a discrete and a...
research
05/13/2020

Guessing Cost: Applications to Distributed DataStorage and Repair in Cellular Networks

The notion of guessing cost (also referred to as the cost of guessing) i...
research
05/13/2020

Cost of Guessing: Applications to Distributed Data Storage and Repair

In this paper, we introduce the notion of the cost of guessing and provi...
research
05/19/2019

Optimal Guessing under Nonextensive Framework and associated Moment Bounds

We consider the problem of guessing the realization of a random variable...

Please sign up or login with your details

Forgot password? Click here to reset