Equivalence Principle of the P-value and Mutual Information
share
In this paper, we propose a novel equivalence between probability theory and information theory. For a single random variable, Shannon's self-information, I=-logp, is an alternative expression of a probability p. However, for two random variables, no information equivalent to the p-value has been identified. Here, we prove theorems that demonstrate that mutual information (MI) is equivalent to the p-value irrespective of prior information about the distribution of the variables. If the maximum entropy principle can be applied, our equivalence theorems allow us to readily compute the p-value from multidimensional MI. By contrast, in a contingency table of any size with known marginal frequencies, our theorem states that MI asymptotically coincides with the logarithm of the p-value of Fisher's exact test, divided by the sample size. Accordingly, the theorems enable us to perform a meta-analysis to accurately estimate MI with a low p-value, thereby calculating informational interdependence that is robust against sample size variation. Thus, our theorems demonstrate the equivalence of the p-value and MI at every dimension, use the merits of both, and provide fundamental information for integrating probability theory and information theory.
READ FULL TEXT
