DeepAI
Log In Sign Up

Generalised Measures of Multivariate Information Content

09/26/2019
by   Conor Finn, et al.
0

The entropy of a pair of random variables is commonly depicted using a Venn diagram. This representation is potentially misleading, however, since the multivariate mutual information can be negative. This paper presents new measures of multivariate information content that can be accurately depicted using Venn diagrams for any number of random variables. These measures complement the existing measures of multivariate mutual information and are constructed by considering the algebraic structure of information sharing. It is shown that the distinct ways in which a set of marginal observers can share their information with a non-observing third party corresponds to the elements of a free distributive lattice. The redundancy lattice from partial information decomposition is then subsequently and independently derived by combining the algebraic structures of joint and shared information content.

READ FULL TEXT

page 1

page 2

page 3

page 4

02/09/2020

A differentiable measure of pointwise shared information

Partial information decomposition (PID) of the multivariate mutual infor...
01/27/2018

Pointwise Information Decomposition Using the Specificity and Ambiguity Lattices

What are the distinct ways in which a set of predictor variables can pro...
03/06/2018

Exact partial information decompositions for Gaussian systems based on dependency constraints

The Partial Information Decomposition (PID) [arXiv:1004.2515] provides a...
11/07/2022

Information Properties of a Random Variable Decomposition through Lattices

A full-rank lattice in the Euclidean space is a discrete set formed by a...
10/13/2020

Gaussianizing the Earth: Multidimensional Information Measures for Earth Data Analysis

Information theory is an excellent framework for analyzing Earth system ...
06/04/2019

Partial and semi-partial measures of spatial associations for multivariate lattice data

This paper concerns the development of partial and semi-partial measures...
03/21/2019

Decomposing information into copying versus transformation

In many real-world systems, information can be transmitted in two qualit...