Estimating the Unique Information of Continuous Variables

01/30/2021
by   Ari Pakman, et al.
0

Partial information decompositions (PIDs) identify different modes in which information from multiple sources may affect a target, by isolating synergistic, redundant, and unique contributions to the mutual information. While many works have studied aspects of PIDs for Gaussian and discrete distributions, the case of general continuous distributions is still uncharted territory. In this work we present a method for estimating the unique information in continuous distributions, for the case of two sources and one target. Our method solves the associated optimization problem over the space of distributions with constrained marginals by combining copula decompositions and techniques developed to optimize variational autoencoders. We illustrate our approach by showing excellent agreement with known analytic results for Gaussians and by analyzing model systems of three coupled random variables.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/23/2021

A partial information decomposition for discrete and continuous variables

Conceptually, partial information decomposition (PID) is concerned with ...
research
12/23/2021

Signed and Unsigned Partial Information Decompositions of Continuous Network Interactions

We investigate the partial information decomposition (PID) framework as ...
research
05/11/2023

Computing Unique Information for Poisson and Multinomial Systems

Bivariate Partial Information Decomposition (PID) describes how the mutu...
research
03/03/2022

Joint Probability Estimation Using Tensor Decomposition and Dictionaries

In this work, we study non-parametric estimation of joint probabilities ...
research
06/13/2022

A comparison of partial information decompositions using data from real and simulated layer 5b pyramidal cells

Partial information decomposition allows the joint mutual information be...
research
07/20/2023

Gaussian Partial Information Decomposition: Bias Correction and Application to High-dimensional Data

Recent advances in neuroscientific experimental techniques have enabled ...
research
03/06/2018

Exact partial information decompositions for Gaussian systems based on dependency constraints

The Partial Information Decomposition (PID) [arXiv:1004.2515] provides a...

Please sign up or login with your details

Forgot password? Click here to reset