Information Decomposition based on Cooperative Game Theory

10/14/2019
by   Nihat Ay, et al.
0

We offer a new approach to the information decomposition problem in information theory: given a 'target' random variable co-distributed with multiple 'source' variables, how can we decompose the mutual information into a sum of non-negative terms that quantify the contributions of each random variable, not only individually but also in combination? We derive our composition from cooperative game theory. It can be seen as assigning a "fair share" of the mutual information to each combination of the source variables. Our decomposition is based on a different lattice from the usual 'partial information decomposition' (PID) approach, and as a consequence our decomposition has a smaller number of terms: it has analogs of the synergy and unique information terms, but lacks terms corresponding to redundancy. Because of this, it is able to obey equivalents of the axioms known as 'local positivity' and 'identity', which cannot be simultaneously satisfied by a PID measure.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/23/2021

On shared and multiple information

We address three outstanding problems in information theory. Problem one...
research
07/12/2022

Values of Games for Information Decomposition

The information decomposition problem requires an additive decomposition...
research
01/27/2018

Pointwise Information Decomposition Using the Specificity and Ambiguity Lattices

What are the distinct ways in which a set of predictor variables can pro...
research
11/13/2017

The identity of information: how deterministic dependencies constrain information synergy and redundancy

Understanding how different information sources together transmit inform...
research
06/01/2023

From Babel to Boole: The Logical Organization of Information Decompositions

The conventional approach to the general Partial Information Decompositi...
research
08/21/2020

Bits and Pieces: Understanding Information Decomposition from Part-whole Relationships and Formal Logic

Partial information decomposition (PID) seeks to decompose the multivari...
research
11/07/2022

Information Properties of a Random Variable Decomposition through Lattices

A full-rank lattice in the Euclidean space is a discrete set formed by a...

Please sign up or login with your details

Forgot password? Click here to reset