Generalizing Bottleneck Problems

02/16/2018
by   Hsiang Hsu, et al.
0

Given a pair of random variables (X,Y)∼ P_XY and two convex functions f_1 and f_2, we introduce two bottleneck functionals as the lower and upper boundaries of the two-dimensional convex set that consists of the pairs (I_f_1(W; X), I_f_2(W; Y)), where I_f denotes f-information and W varies over the set of all discrete random variables satisfying the Markov condition W → X → Y. Applying Witsenhausen and Wyner's approach, we provide an algorithm for computing boundaries of this set for f_1, f_2, and discrete P_XY, . In the binary symmetric case, we fully characterize the set when (i) f_1(t)=f_2(t)=t t, (ii) f_1(t)=f_2(t)=t^2-1, and (iii) f_1 and f_2 are both ℓ^β norm function for β > 1. We then argue that upper and lower boundaries in (i) correspond to Mrs. Gerber's Lemma and its inverse (which we call Mr. Gerber's Lemma), in (ii) correspond to estimation-theoretic variants of Information Bottleneck and Privacy Funnel, and in (iii) correspond to Arimoto Information Bottleneck and Privacy Funnel.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
11/12/2020

Bottleneck Problems: Information and Estimation-Theoretic View

Information bottleneck (IB) and privacy funnel (PF) are two closely rela...
research
01/03/2018

Information Bottleneck on General Alphabets

We prove a source coding theorem that can probably be considered folklor...
research
10/03/2021

A Class of Nonbinary Symmetric Information Bottleneck Problems

We study two dual settings of information processing. Let 𝖸→𝖷→𝖶 be a Mar...
research
05/12/2020

Quadratic Privacy-Signaling Games, Payoff Dominant Equilibria and the Information Bottleneck Problem

We introduce a privacy-signaling game problem in which a transmitter wit...
research
04/17/2019

Bottleneck potentials in Markov Random Fields

We consider general discrete Markov Random Fields(MRFs) with additional ...
research
11/17/2020

VIB is Half Bayes

In discriminative settings such as regression and classification there a...
research
10/19/2020

ABC-Di: Approximate Bayesian Computation for Discrete Data

Many real-life problems are represented as a black-box, i.e., the intern...

Please sign up or login with your details

Forgot password? Click here to reset