Information Bottleneck on General Alphabets
share
We prove a source coding theorem that can probably be considered folklore, a generalization to arbitrary alphabets of a problem motivated by the Information Bottleneck method. For general random variables (Y, X), we show essentially that for some n ∈N, a function f with rate limit |f| < nR and I(Y^n; f(X^n)) > nS exists if and only if there is a discrete random variable U such that the Markov chain Y - X - U holds, I(U; X) < R and I(U; Y) < S.
READ FULL TEXT