Log In Sign Up

Information Theory in Density Destructors

by   J. Emmanuel Johnson, et al.

Density destructors are differentiable and invertible transforms that map multivariate PDFs of arbitrary structure (low entropy) into non-structured PDFs (maximum entropy). Multivariate Gaussianization and multivariate equalization are specific examples of this family, which break down the complexity of the original PDF through a set of elementary transforms that progressively remove the structure of the data. We demonstrate how this property of density destructive flows is connected to classical information theory, and how density destructors can be used to get more accurate estimates of information theoretic quantities. Experiments with total correlation and mutual information inmultivariate sets illustrate the ability of density destructors compared to competing methods. These results suggest that information theoretic measures may be an alternative optimization criteria when learning density destructive flows.


page 1

page 2

page 3

page 4


infotheory: A C++/Python package for multivariate information theoretic analysis

This paper introduces infotheory: a package that implements multivariate...

Optimal Partitions for Nonparametric Multivariate Entropy Estimation

Efficient and accurate estimation of multivariate empirical probability ...

The Channel Multivariate Entropy Triangle and Balance Equation

In this paper we use information-theoretic measures to provide a theory ...

Estimators for Multivariate Information Measures in General Probability Spaces

Information theoretic quantities play an important role in various setti...

An Elementary Proof of a Classical Information-Theoretic Formula

A renowned information-theoretic formula by Shannon expresses the mutual...

Gaussianizing the Earth: Multidimensional Information Measures for Earth Data Analysis

Information theory is an excellent framework for analyzing Earth system ...

Information Theory Measures via Multidimensional Gaussianization

Information theory is an outstanding framework to measure uncertainty, d...