Generalized Kernel Thinning

by   Raaz Dwivedi, et al.

The kernel thinning (KT) algorithm of Dwivedi and Mackey (2021) compresses a probability distribution more effectively than independent sampling by targeting a reproducing kernel Hilbert space (RKHS) and leveraging a less smooth square-root kernel. Here we provide four improvements. First, we show that KT applied directly to the target RKHS yields tighter, dimension-free guarantees for any kernel, any distribution, and any fixed function in the RKHS. Second, we show that, for analytic kernels like Gaussian, inverse multiquadric, and sinc, target KT admits maximum mean discrepancy (MMD) guarantees comparable to or better than those of square-root KT without making explicit use of a square-root kernel. Third, we prove that KT with a fractional power kernel yields better-than-Monte-Carlo MMD guarantees for non-smooth kernels, like Laplace and Matérn, that do not have square-roots. Fourth, we establish that KT applied to a sum of the target and power kernels (a procedure we call KT+) simultaneously inherits the improved MMD guarantees of power KT and the tighter individual function guarantees of target KT. In our experiments with target KT and KT+, we witness significant improvements in integration error even in 100 dimensions and when compressing challenging differential equation posteriors.



There are no comments yet.


page 1

page 2

page 3

page 4


Kernel Thinning

We introduce kernel thinning, a new procedure for compressing a distribu...

Deep Neural Tangent Kernel and Laplace Kernel Have the Same RKHS

We prove that the reproducing kernel Hilbert spaces (RKHS) of a deep neu...

A kernel-independent sum-of-Gaussians method by de la Vallée-Poussin sums

Approximation of interacting kernels by sum of Gaussians (SOG) is freque...

Distribution Compression in Near-linear Time

In distribution compression, one aims to accurately summarize a probabil...

A Fractional Hawkes process

We modify ETAS models by replacing the Pareto-like kernel proposed by Og...

The Generalized Complex Kernel Least-Mean-Square Algorithm

We propose a novel adaptive kernel based regression method for complex-v...

A Brief Note on the Convergence of Langevin Monte Carlo in Chi-Square Divergence

We study sampling from a target distribution ν_* ∝ e^-f using the unadju...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.