DeepAI AI Chat
Log In Sign Up

Estimating Feature-Label Dependence Using Gini Distance Statistics

by   Silu Zhang, et al.

Identifying statistical dependence between the features and the label is a fundamental problem in supervised learning. This paper presents a framework for estimating dependence between numerical features and a categorical label using generalized Gini distance, an energy distance in reproducing kernel Hilbert spaces (RKHS). Two Gini distance based dependence measures are explored: Gini distance covariance and Gini distance correlation. Unlike Pearson covariance and correlation, which do not characterize independence, the above Gini distance based measures define dependence as well as independence of random variables. The test statistics are simple to calculate and do not require probability density estimation. Uniform convergence bounds and asymptotic bounds are derived for the test statistics. Comparisons with distance covariance statistics are provided. It is shown that Gini distance statistics converge faster than distance covariance statistics in the uniform convergence bounds, hence tighter upper bounds on both Type I and Type II errors. Moreover, the probability of Gini distance covariance statistic under-performing the distance covariance statistic in Type II error decreases to 0 exponentially with the increase of the sample size. Extensive experimental results are presented to demonstrate the performance of the proposed method.


Detecting independence of random vectors II. Distance multivariance and Gaussian multivariance

We introduce two new measures for the dependence of n > 2 random variabl...

A fast algorithm for computing distance correlation

Classical dependence measures such as Pearson correlation, Spearman's ρ,...

Nonasymptotic one-and two-sample tests in high dimension with unknown covariance structure

Let 𝐗 = (X_i)_1≤ i ≤ n be an i.i.d. sample of square-integrable variable...

Testing Serial Independence of Object-Valued Time Series

We propose a novel method for testing serial independence of object-valu...

Distance covariance for discretized stochastic processes

Given an iid sequence of pairs of stochastic processes on the unit inter...

A Basic Treatment of the Distance Covariance

The distance covariance of Székely, et al. [23] and Székely and Rizzo [2...