Estimating the Unique Information of Continuous Variables
share
Partial information decompositions (PIDs) identify different modes in which information from multiple sources may affect a target, by isolating synergistic, redundant, and unique contributions to the mutual information. While many works have studied aspects of PIDs for Gaussian and discrete distributions, the case of general continuous distributions is still uncharted territory. In this work we present a method for estimating the unique information in continuous distributions, for the case of two sources and one target. Our method solves the associated optimization problem over the space of distributions with constrained marginals by combining copula decompositions and techniques developed to optimize variational autoencoders. We illustrate our approach by showing excellent agreement with known analytic results for Gaussians and by analyzing model systems of three coupled random variables.
READ FULL TEXT