Measuring Dependencies of Order Statistics: An Information Theoretic Perspective
share
Consider a random sample X_1 , X_2 , ..., X_n drawn independently and identically distributed from some known sampling distribution P_X. Let X_(1)≤ X_(2)≤ ... ≤ X_(n) represent the order statistics of the sample. The first part of the paper focuses on distributions with an invertible cumulative distribution function. Under this assumption, a distribution-free property is established, which shows that the f-divergence between the joint distribution of order statistics and the product distribution of order statistics does not depend on the original sampling distribution P_X. Moreover, it is shown that the mutual information between two subsets of order statistics also satisfies a distribution-free property; that is, it does not depend on P_X. Furthermore, the decoupling rates between X_(r) and X_(m) (i.e., rates at which the mutual information approaches zero) are characterized for various choices of (r,m). The second part of the paper considers a family of discrete distributions, which does not satisfy the assumptions in the first part of the paper. In comparison to the results of the first part, it is shown that in the discrete setting, the mutual information between order statistics does depend on the sampling distribution P_X. Nonetheless, it is shown that the results of the first part can still be used as upper bounds on the decoupling rates.
READ FULL TEXT