Entropy and Compression: A simple proof of an inequality of Khinchin-Ornstein-Shields

07/10/2019

∙

by   Riccardo Aragona, et al.

∙

0

∙

share

We prove that Entropy is a lower bound for the average compression ratio of any lossless compressor by giving a simple proof of an inequality that is a slightly variation of an inequality firstly proved by A. I. Khinchin in 1953. The same idea leads to a simple proof of the analogous Ornstein-Shields pointwise inequality of 1990. Our proof is simpler of the ones (of the same pointwise inequality) given by Shields in 1996.