Higher-order in-and-outeractions reveal synergy and logical dependence beyond Shannon-information

05/09/2022
by   Abel Jansma, et al.
0

Information-theoretic quantities reveal dependencies among variables in the structure of joint, marginal, and conditional entropies, but leave some fundamentally different systems indistinguishable. Furthermore, there is no consensus on how to construct and interpret a higher-order generalisation of mutual information (MI). In this manuscript, we show that a recently proposed model-free definition of higher-order interactions amongst binary variables (MFIs), like mutual information, is a Möbius inversion on a Boolean algebra, but of surprisal instead of entropy. This gives an information-theoretic interpretation to the MFIs, and by extension to Ising interactions. We study the dual objects to MI and MFIs on the order-reversed lattice, and find that dual MI is related to the previously studied differential mutual information, while dual interactions (outeractions) are interactions with respect to a different background state. Unlike mutual information, in- and outeractions uniquely identify all six 2-input logic gates, the dy- and triadic distributions, and different causal dynamics that are identical in terms of their Shannon-information content.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/11/2019

Generalized Mutual Information

Mutual information is one of the essential building blocks of informatio...
research
09/14/2023

Generalized Decomposition of Multivariate Information

Since its introduction, the partial information decomposition (PID) has ...
research
05/03/2023

Inferential Moments of Uncertain Multivariable Systems

This article offers a new paradigm for analyzing the behavior of uncerta...
research
03/21/2023

Higher-order Organization in the Human Brain from Matrix-Based Rényi's Entropy

Pairwise metrics are often employed to estimate statistical dependencies...
research
02/24/2017

The Stochastic complexity of spin models: How simple are simple spin models?

Simple models, in information theoretic terms, are those with a small st...
research
11/27/2018

Maximizing Multivariate Information with Error-Correcting Codes

Multivariate mutual information provides a conceptual framework for char...
research
05/22/2023

Fundamental connections between utility theories of wealth and information theory

We establish fundamental connections between utility theories of wealth ...

Please sign up or login with your details

Forgot password? Click here to reset