Influence Functions for Machine Learning: Nonparametric Estimators for Entropies, Divergences and Mutual Informations

11/17/2014
by   Kirthevasan Kandasamy, et al.
0

We propose and analyze estimators for statistical functionals of one or more distributions under nonparametric assumptions. Our estimators are based on the theory of influence functions, which appear in the semiparametric statistics literature. We show that estimators based either on data-splitting or a leave-one-out technique enjoy fast rates of convergence and other favorable theoretical properties. We apply this framework to derive estimators for several popular information theoretic quantities, and via empirical evaluation, show the advantage of this approach over existing estimators.

READ FULL TEXT
research
05/19/2016

Efficient Nonparametric Smoothness Estimation

Sobolev quantities (norms, inner products, and distances) of probability...
research
05/20/2020

Nonparametric Score Estimators

Estimating the score, i.e., the gradient of log density function, from a...
research
05/12/2022

Estimation of Matusita Overlapping Coefficient for Pair Normal Distributions

The Matusita overlapping coefficient is defined as agreement or similari...
research
02/12/2014

Nonparametric Estimation of Renyi Divergence and Friends

We consider nonparametric estimation of L_2, Renyi-α and Tsallis-α diver...
research
12/31/2020

On Gaussian Approximation for M-Estimator

This study develops a non-asymptotic Gaussian approximation theory for d...
research
10/08/2018

Visually Communicating and Teaching Intuition for Influence Functions

Estimators based on influence functions (IFs) have been shown effective ...
research
06/03/2022

Debiased Machine Learning without Sample-Splitting for Stable Estimators

Estimation and inference on causal parameters is typically reduced to a ...

Please sign up or login with your details

Forgot password? Click here to reset