DeepAI AI Chat
Log In Sign Up

fastMI: a fast and consistent copula-based estimator of mutual information

12/20/2022
by   Soumik Purkayastha, et al.
0

As a fundamental concept in information theory, mutual information (MI) has been commonly applied to quantify association between random variables. Most existing estimators of MI have unstable statistical performance since they involve parameter tuning. We develop a consistent and powerful estimator, called fastMI, that does not incur any parameter tuning. Based on a copula formulation, fastMI estimates MI by leveraging Fast Fourier transform-based estimation of the underlying density. Extensive simulation studies reveal that fastMI outperforms state-of-the-art estimators with improved estimation accuracy and reduced run time for large data sets. fastMI provides a powerful test for independence that exhibits satisfactory type I error control. Anticipating that it will be a powerful tool in estimating mutual information in a broad range of data, we develop an R package fastMI for broader dissemination.

READ FULL TEXT

page 22

page 23

05/18/2021

A note on the unbiased estimation of mutual information

Estimators for mutual information are typically biased. However, in the ...
05/17/2020

C-MI-GAN : Estimation of Conditional Mutual Information using MinMax formulation

Estimation of information theoretic quantities such as mutual informatio...
11/02/2017

Geometric k-nearest neighbor estimation of entropy and mutual information

Like most nonparametric estimators of information functionals involving ...
06/16/2016

Estimating mutual information in high dimensions via classification error

Multivariate pattern analyses approaches in neuroimaging are fundamental...
04/30/2018

On the Effect of Suboptimal Estimation of Mutual Information in Feature Selection and Classification

This paper introduces a new property of estimators of the strength of st...
10/14/2019

Understanding the Limitations of Variational Mutual Information Estimators

Variational approaches based on neural networks are showing promise for ...
10/26/2022

Asymmetric predictability in causal discovery: an information theoretic approach

Causal investigations in observational studies pose a great challenge in...