DeepAI AI Chat
Log In Sign Up

Active Slices for Sliced Stein Discrepancy

by   Wenbo Gong, et al.

Sliced Stein discrepancy (SSD) and its kernelized variants have demonstrated promising successes in goodness-of-fit tests and model learning in high dimensions. Despite their theoretical elegance, their empirical performance depends crucially on the search of optimal slicing directions to discriminate between two distributions. Unfortunately, previous gradient-based optimisation approaches for this task return sub-optimal results: they are computationally expensive, sensitive to initialization, and they lack theoretical guarantees for convergence. We address these issues in two steps. First, we provide theoretical results stating that the requirement of using optimal slicing directions in the kernelized version of SSD can be relaxed, validating the resulting discrepancy with finite random slicing directions. Second, given that good slicing directions are crucial for practical performance, we propose a fast algorithm for finding such slicing directions based on ideas of active sub-space construction and spectral decomposition. Experiments on goodness-of-fit tests and model learning show that our approach achieves both improved performance and faster convergence. Especially, we demonstrate a 14-80x speed-up in goodness-of-fit tests when comparing with gradient-based alternatives.


page 1

page 2

page 3

page 4


Sliced Kernelized Stein Discrepancy

Kernelized Stein discrepancy (KSD), though being extensively used in goo...

A Kernelized Stein Discrepancy for Goodness-of-fit Tests and Model Evaluation

We derive a new discrepancy statistic for measuring differences between ...

Kernel Selection for Stein Variational Gradient Descent

Stein variational gradient descent (SVGD) and its variants have shown pr...

Towards Gradient-based Bilevel Optimization with Non-convex Followers and Beyond

In recent years, Bi-Level Optimization (BLO) techniques have received ex...

On a projection-based class of uniformity tests on the hypersphere

We propose a projection-based class of uniformity tests on the hypersphe...

Gradient-Free Kernel Stein Discrepancy

Stein discrepancies have emerged as a powerful statistical tool, being a...

Towards Shockingly Easy Structured Classification: A Search-based Probabilistic Online Learning Framework

There are two major approaches for structured classification. One is the...