Generalized Kernel Thinning

10/04/2021
by   Raaz Dwivedi, et al.
0

The kernel thinning (KT) algorithm of Dwivedi and Mackey (2021) compresses a probability distribution more effectively than independent sampling by targeting a reproducing kernel Hilbert space (RKHS) and leveraging a less smooth square-root kernel. Here we provide four improvements. First, we show that KT applied directly to the target RKHS yields tighter, dimension-free guarantees for any kernel, any distribution, and any fixed function in the RKHS. Second, we show that, for analytic kernels like Gaussian, inverse multiquadric, and sinc, target KT admits maximum mean discrepancy (MMD) guarantees comparable to or better than those of square-root KT without making explicit use of a square-root kernel. Third, we prove that KT with a fractional power kernel yields better-than-Monte-Carlo MMD guarantees for non-smooth kernels, like Laplace and Matérn, that do not have square-roots. Fourth, we establish that KT applied to a sum of the target and power kernels (a procedure we call KT+) simultaneously inherits the improved MMD guarantees of power KT and the tighter individual function guarantees of target KT. In our experiments with target KT and KT+, we witness significant improvements in integration error even in 100 dimensions and when compressing challenging differential equation posteriors.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

05/12/2021

Kernel Thinning

We introduce kernel thinning, a new procedure for compressing a distribu...
09/22/2020

Deep Neural Tangent Kernel and Laplace Kernel Have the Same RKHS

We prove that the reproducing kernel Hilbert spaces (RKHS) of a deep neu...
10/11/2020

A kernel-independent sum-of-Gaussians method by de la Vallée-Poussin sums

Approximation of interacting kernels by sum of Gaussians (SOG) is freque...
11/15/2021

Distribution Compression in Near-linear Time

In distribution compression, one aims to accurately summarize a probabil...
03/02/2020

A Fractional Hawkes process

We modify ETAS models by replacing the Pareto-like kernel proposed by Og...
02/22/2019

The Generalized Complex Kernel Least-Mean-Square Algorithm

We propose a novel adaptive kernel based regression method for complex-v...
07/22/2020

A Brief Note on the Convergence of Langevin Monte Carlo in Chi-Square Divergence

We study sampling from a target distribution ν_* ∝ e^-f using the unadju...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.