DeepAI
Log In Sign Up

Information Theory in Density Destructors

12/02/2020
by   J. Emmanuel Johnson, et al.
8

Density destructors are differentiable and invertible transforms that map multivariate PDFs of arbitrary structure (low entropy) into non-structured PDFs (maximum entropy). Multivariate Gaussianization and multivariate equalization are specific examples of this family, which break down the complexity of the original PDF through a set of elementary transforms that progressively remove the structure of the data. We demonstrate how this property of density destructive flows is connected to classical information theory, and how density destructors can be used to get more accurate estimates of information theoretic quantities. Experiments with total correlation and mutual information inmultivariate sets illustrate the ability of density destructors compared to competing methods. These results suggest that information theoretic measures may be an alternative optimization criteria when learning density destructive flows.

READ FULL TEXT

page 1

page 2

page 3

page 4

07/04/2019

infotheory: A C++/Python package for multivariate information theoretic analysis

This paper introduces infotheory: a package that implements multivariate...
12/12/2021

Optimal Partitions for Nonparametric Multivariate Entropy Estimation

Efficient and accurate estimation of multivariate empirical probability ...
11/30/2017

The Channel Multivariate Entropy Triangle and Balance Equation

In this paper we use information-theoretic measures to provide a theory ...
10/26/2018

Estimators for Multivariate Information Measures in General Probability Spaces

Information theoretic quantities play an important role in various setti...
09/05/2018

An Elementary Proof of a Classical Information-Theoretic Formula

A renowned information-theoretic formula by Shannon expresses the mutual...
10/13/2020

Gaussianizing the Earth: Multidimensional Information Measures for Earth Data Analysis

Information theory is an excellent framework for analyzing Earth system ...
10/08/2020

Information Theory Measures via Multidimensional Gaussianization

Information theory is an outstanding framework to measure uncertainty, d...