Mutual Dependence: A Novel Method for Computing Dependencies Between Random Variables

by   Rahul Agarwal, et al.
Johns Hopkins University

In data science, it is often required to estimate dependencies between different data sources. These dependencies are typically calculated using Pearson's correlation, distance correlation, and/or mutual information. However, none of these measures satisfy all the Granger's axioms for an "ideal measure". One such ideal measure, proposed by Granger himself, calculates the Bhattacharyya distance between the joint probability density function (pdf) and the product of marginal pdfs. We call this measure the mutual dependence. However, to date this measure has not been directly computable from data. In this paper, we use our recently introduced maximum likelihood non-parametric estimator for band-limited pdfs, to compute the mutual dependence directly from the data. We construct the estimator of mutual dependence and compare its performance to standard measures (Pearson's and distance correlation) for different known pdfs by computing convergence rates, computational complexity, and the ability to capture nonlinear dependencies. Our mutual dependence estimator requires fewer samples to converge to theoretical values, is faster to compute, and captures more complex dependencies than standard measures.


TCMI: a non-parametric mutual-dependence estimator for multivariate continuous distributions

The identification of relevant features, i.e., the driving variables tha...

Neural Fields for Interactive Visualization of Statistical Dependencies in 3D Simulation Ensembles

We present the first neural network that has learned to compactly repres...

A fast algorithm for computing distance correlation

Classical dependence measures such as Pearson correlation, Spearman's ρ,...

Scalable Mutual Information Estimation using Dependence Graphs

We propose a unified method for empirical non-parametric estimation of g...

Survey On The Estimation Of Mutual Information Methods as a Measure of Dependency Versus Correlation Analysis

In this survey, we present and compare different approaches to estimate ...

Learning Independent Features with Adversarial Nets for Non-linear ICA

Reliable measures of statistical dependence could be useful tools for le...

Please sign up or login with your details

Forgot password? Click here to reset