
A differentiable measure of pointwise shared information
Partial information decomposition (PID) of the multivariate mutual infor...
read it

Pointwise Information Decomposition Using the Specificity and Ambiguity Lattices
What are the distinct ways in which a set of predictor variables can pro...
read it

Bits and Pieces: Understanding Information Decomposition from Partwhole Relationships and Formal Logic
Partial information decomposition (PID) seeks to decompose the multivari...
read it

Invariant components of synergy, redundancy, and unique information among three variables
In a system of three stochastic variables, the Partial Information Decom...
read it

Information Decomposition based on Cooperative Game Theory
We offer a new approach to the information decomposition problem in info...
read it

The identity of information: how deterministic dependencies constrain information synergy and redundancy
Understanding how different information sources together transmit inform...
read it

MultiSource Neural Variational Inference
Learning from multiple sources of information is an important problem in...
read it
A novel approach to multivariate redundancy and synergy
Consider a situation in which a set of n "source" random variables X_1,...,X_n have information about some "target" random variable Y. For example, in neuroscience Y might represent the state of an external stimulus and X_1,...,X_n the activity of n different brain regions. Recent work in information theory has considered how to decompose the information that the sources X_1,...,X_n provide about the target Y into separate terms such as (1) the "redundant information" that is shared among all of sources, (2) the "unique information" that is provided only by a single source, (3) the "synergistic information" that is provided by all sources only when considered jointly, and (4) the "union information" that is provided by at least one source. We propose a novel framework deriving such a decomposition that can be applied to any number of sources. Our measures are motivated in three distinct ways: via a formal analogy to intersection and union operators in set theory, via a decisiontheoretic operationalization based on Blackwell's theorem, and via an axiomatic derivation. A key aspect of our approach is that we relax the assumption that measures of redundancy and union information should be related by the inclusionexclusion principle. We discuss relations to previous proposals as well as possible generalizations.
READ FULL TEXT
Comments
There are no comments yet.