Log In Sign Up

Generalised Measures of Multivariate Information Content

by   Conor Finn, et al.

The entropy of a pair of random variables is commonly depicted using a Venn diagram. This representation is potentially misleading, however, since the multivariate mutual information can be negative. This paper presents new measures of multivariate information content that can be accurately depicted using Venn diagrams for any number of random variables. These measures complement the existing measures of multivariate mutual information and are constructed by considering the algebraic structure of information sharing. It is shown that the distinct ways in which a set of marginal observers can share their information with a non-observing third party corresponds to the elements of a free distributive lattice. The redundancy lattice from partial information decomposition is then subsequently and independently derived by combining the algebraic structures of joint and shared information content.


page 1

page 2

page 3

page 4


A differentiable measure of pointwise shared information

Partial information decomposition (PID) of the multivariate mutual infor...

Pointwise Information Decomposition Using the Specificity and Ambiguity Lattices

What are the distinct ways in which a set of predictor variables can pro...

Exact partial information decompositions for Gaussian systems based on dependency constraints

The Partial Information Decomposition (PID) [arXiv:1004.2515] provides a...

Information Properties of a Random Variable Decomposition through Lattices

A full-rank lattice in the Euclidean space is a discrete set formed by a...

Gaussianizing the Earth: Multidimensional Information Measures for Earth Data Analysis

Information theory is an excellent framework for analyzing Earth system ...

Partial and semi-partial measures of spatial associations for multivariate lattice data

This paper concerns the development of partial and semi-partial measures...

Decomposing information into copying versus transformation

In many real-world systems, information can be transmitted in two qualit...