Multi-variate correlation and mixtures of product measures
share
Total correlation (`TC') and dual total correlation (`DTC') are two classical way to quantify the correlation among an n-tuple of random variables. They both reduce to mutual information when n=2. The first part of this paper sets up the theory of TC and DTC for general random variables, not necessarily finite-valued. This generality has not been exposed in the literature before. The second part considers the structural implications when a joint distribution μ has small TC or DTC. If TC(μ) = o(n), then μ is close to a product measure according to a suitable transportation metric: this follows directly from Marton's classical transportation-entropy inequality. If DTC(μ) = o(n), then the structural consequence is more complicated: μ is a mixture of a controlled number of terms, most of them close to product measures in the transportation metric. This is the main new result of the paper.
READ FULL TEXT