An Optimal Witness Function for Two-Sample Testing

02/10/2021
by   Jonas M. Kübler, et al.
4

We propose data-dependent test statistics based on a one-dimensional witness function, which we call witness two-sample tests (WiTS tests). We first optimize the witness function by maximizing an asymptotic test-power objective and then use as the test statistic the difference in means of the witness evaluated on two held-out test samples. When the witness function belongs to a reproducing kernel Hilbert space, we show that the optimal witness is given via kernel Fisher discriminant analysis, whose solution we compute in closed form. We show that the WiTS test based on a characteristic kernel is consistent against any fixed alternative. Our experiments demonstrate that the WiTS test can achieve higher test power than existing two-sample tests with optimized kernels, suggesting that learning a high- or infinite-dimensional representation of the data may not be necessary for two-sample testing. The proposed procedure works beyond kernel methods, allowing practitioners to apply it within their preferred machine learning framework.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
11/12/2020

Generalized Kernel Two-Sample Tests

Kernel two-sample tests have been widely used for multivariate data in t...
research
06/03/2020

Learning Kernel Tests Without Data Splitting

Modern large-scale kernel-based tests such as maximum mean discrepancy (...
research
10/14/2019

Two-sample Testing Using Deep Learning

We propose a two-sample testing procedure based on learned deep neural n...
research
06/17/2022

AutoML Two-Sample Test

Two-sample tests are important in statistics and machine learning, both ...
research
09/25/2019

Classification Logit Two-sample Testing by Neural Networks

The recent success of generative adversarial networks and variational le...
research
03/02/2018

Robust Multivariate Nonparametric Tests via Projection-Pursuit

In this work, we generalize the Cramér-von Mises statistic via projectio...
research
06/14/2021

Meta Two-Sample Testing: Learning Kernels for Testing with Limited Data

Modern kernel-based two-sample tests have shown great success in disting...

Please sign up or login with your details

Forgot password? Click here to reset