Entropic Causal Inference

11/12/2016
by   Murat Kocaoglu, et al.
0

We consider the problem of identifying the causal direction between two discrete random variables using observational data. Unlike previous work, we keep the most general functional model but make an assumption on the unobserved exogenous variable: Inspired by Occam's razor, we assume that the exogenous variable is simple in the true causal direction. We quantify simplicity using Rényi entropy. Our main result is that, under natural assumptions, if the exogenous variable has low H_0 entropy (cardinality) in the true direction, it must have high H_0 entropy in the wrong direction. We establish several algorithmic hardness results about estimating the minimum entropy exogenous variable. We show that the problem of finding the exogenous variable with minimum entropy is equivalent to the problem of finding minimum joint entropy given n marginal distributions, also known as minimum entropy coupling problem. We propose an efficient greedy algorithm for the minimum entropy coupling problem, that for n=2 provably finds a local optimum. This gives a greedy algorithm for finding the exogenous variable with minimum H_1 (Shannon Entropy). Our greedy entropy-based causal inference algorithm has similar performance to the state of the art additive noise models in real datasets. One advantage of our approach is that we make no use of the values of random variables but only their distributions. Our method can therefore be used for causal inference for both ordinal and also categorical data, unlike additive noise models.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
01/28/2017

Entropic Causality and Greedy Minimum Entropy Coupling

We study the problem of identifying the causal relationship between two ...
research
01/10/2021

Entropic Causal Inference: Identifiability and Finite Sample Results

Entropic causal inference is a framework for inferring the causal direct...
research
01/19/2019

Minimum--Entropy Couplings and their Applications

Given two discrete random variables X and Y, with probability distributi...
research
05/09/2023

Information Spectrum Converse for Minimum Entropy Couplings and Functional Representations

Given two jointly distributed random variables (X,Y), a functional repre...
research
02/23/2023

Minimum-Entropy Coupling Approximation Guarantees Beyond the Majorization Barrier

Given a set of discrete probability distributions, the minimum entropy c...
research
03/10/2022

A Tighter Approximation Guarantee for Greedy Minimum Entropy Coupling

We examine the minimum entropy coupling problem, where one must find the...
research
07/23/2018

On Enumerating Distributions for Associated Vectors in the Entropy Space

This paper focuses on the problem of finding a distribution for an assoc...

Please sign up or login with your details

Forgot password? Click here to reset