A unified theory of information transfer and causal relation

04/21/2022
by   Yang Tian, et al.
0

Information transfer between coupled stochastic dynamics, measured by transfer entropy and information flow, is suggested as a physical process underlying the causal relation of systems. While information transfer analysis has booming applications in both science and engineering fields, critical mysteries about its foundations remain unsolved. Fundamental yet difficult questions concern how information transfer and causal relation originate, what they depend on, how they differ from each other, and if they are created by a unified and general quantity. These questions essentially determine the validity of causal relation measurement via information transfer. Here we pursue to lay a complete theoretical basis of information transfer and causal relation. Beyond the well-known relations between these concepts that conditionally hold, we demonstrate that information transfer and causal relation universally originate from specific information synergy and redundancy phenomena characterized by high-order mutual information. More importantly, our theory analytically explains the mechanisms for information transfer and causal relation to originate, vanish, and differ from each other. Moreover, our theory naturally defines the effect sizes of information transfer and causal relation based on high-dimensional coupling events. These results may provide a unified view of information, synergy, and causal relation to bridge Pearl's causal inference theory in computer science and information transfer analysis in physics.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
02/06/2020

On Geometry of Information Flow for Causal Inference

Causal inference is perhaps one of the most fundamental concepts in scie...
research
10/09/2012

Quantifying Causal Coupling Strength: A Lag-specific Measure For Multivariate Time Series Related To Transfer Entropy

While it is an important problem to identify the existence of causal ass...
research
10/10/2019

Estimating Transfer Entropy via Copula Entropy

Causal inference is a fundemental problem in statistics and has wide app...
research
09/15/2022

Information Theoretic Measures of Causal Influences during Transient Neural Events

Transient phenomena play a key role in coordinating brain activity at mu...
research
06/18/2019

Transfer Learning for Causal Sentence Detection

We consider the task of detecting sentences that express causality, as a...
research
09/09/2021

Relating Graph Neural Networks to Structural Causal Models

Causality can be described in terms of a structural causal model (SCM) t...
research
02/26/2019

Transfer Learning for Performance Modeling of Configurable Systems: A Causal Analysis

Modern systems (e.g., deep neural networks, big data analytics, and comp...

Please sign up or login with your details

Forgot password? Click here to reset