Potential Conditional Mutual Information: Estimators, Properties and Applications

10/13/2017
by   Arman Rahimzamani, et al.
0

The conditional mutual information I(X;Y|Z) measures the average information that X and Y contain about each other given Z. This is an important primitive in many learning problems including conditional independence testing, graphical model inference, causal strength estimation and time-series problems. In several applications, it is desirable to have a functional purely of the conditional distribution p_Y|X,Z rather than of the joint distribution p_X,Y,Z. We define the potential conditional mutual information as the conditional mutual information calculated with a modified joint distribution p_Y|X,Z q_X,Z, where q_X,Z is a potential distribution, fixed airport. We develop K nearest neighbor based estimators for this functional, employing importance sampling, and a coupling trick, and prove the finite k consistency of such an estimator. We demonstrate that the estimator has excellent practical performance and show an application in dynamical system inference.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/05/2019

CCMI : Classifier based Conditional Mutual Information Estimation

Conditional Mutual Information (CMI) is a measure of conditional depende...
research
10/26/2018

Estimators for Multivariate Information Measures in General Probability Spaces

Information theoretic quantities play an important role in various setti...
research
02/10/2016

Conditional Dependence via Shannon Capacity: Axioms, Estimators and Applications

We conduct an axiomatic study of the problem of estimating the strength ...
research
05/17/2020

C-MI-GAN : Estimation of Conditional Mutual Information using MinMax formulation

Estimation of information theoretic quantities such as mutual informatio...
research
06/12/2020

On Neural Estimators for Conditional Mutual Information Using Nearest Neighbors Sampling

The estimation of mutual information (MI) or conditional mutual informat...
research
11/12/2019

Model-Augmented Nearest-Neighbor Estimation of Conditional Mutual Information for Feature Selection

Markov blanket feature selection, while theoretically optimal, generally...
research
02/18/2020

Conditional Mutual information-based Contrastive Loss for Financial Time Series Forecasting

We present a method for financial time series forecasting using represen...

Please sign up or login with your details

Forgot password? Click here to reset