Generalized Kernel Thinning

10/04/2021
by   Raaz Dwivedi, et al.
0

The kernel thinning (KT) algorithm of Dwivedi and Mackey (2021) compresses a probability distribution more effectively than independent sampling by targeting a reproducing kernel Hilbert space (RKHS) and leveraging a less smooth square-root kernel. Here we provide four improvements. First, we show that KT applied directly to the target RKHS yields tighter, dimension-free guarantees for any kernel, any distribution, and any fixed function in the RKHS. Second, we show that, for analytic kernels like Gaussian, inverse multiquadric, and sinc, target KT admits maximum mean discrepancy (MMD) guarantees comparable to or better than those of square-root KT without making explicit use of a square-root kernel. Third, we prove that KT with a fractional power kernel yields better-than-Monte-Carlo MMD guarantees for non-smooth kernels, like Laplace and Matérn, that do not have square-roots. Fourth, we establish that KT applied to a sum of the target and power kernels (a procedure we call KT+) simultaneously inherits the improved MMD guarantees of power KT and the tighter individual function guarantees of target KT. In our experiments with target KT and KT+, we witness significant improvements in integration error even in 100 dimensions and when compressing challenging differential equation posteriors.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
05/12/2021

Kernel Thinning

We introduce kernel thinning, a new procedure for compressing a distribu...
research
09/22/2020

Deep Neural Tangent Kernel and Laplace Kernel Have the Same RKHS

We prove that the reproducing kernel Hilbert spaces (RKHS) of a deep neu...
research
10/11/2020

A kernel-independent sum-of-Gaussians method by de la Vallée-Poussin sums

Approximation of interacting kernels by sum of Gaussians (SOG) is freque...
research
11/15/2021

Distribution Compression in Near-linear Time

In distribution compression, one aims to accurately summarize a probabil...
research
03/02/2020

A Fractional Hawkes process

We modify ETAS models by replacing the Pareto-like kernel proposed by Og...
research
02/22/2019

The Generalized Complex Kernel Least-Mean-Square Algorithm

We propose a novel adaptive kernel based regression method for complex-v...
research
07/22/2020

A Brief Note on the Convergence of Langevin Monte Carlo in Chi-Square Divergence

We study sampling from a target distribution ν_* ∝ e^-f using the unadju...

Please sign up or login with your details

Forgot password? Click here to reset