Deep Neural Tangent Kernel and Laplace Kernel Have the Same RKHS

09/22/2020
by   Lin Chen, et al.
13

We prove that the reproducing kernel Hilbert spaces (RKHS) of a deep neural tangent kernel and the Laplace kernel include the same set of functions, when both kernels are restricted to the sphere 𝕊^d-1. Additionally, we prove that the exponential power kernel with a smaller power (making the kernel more non-smooth) leads to a larger RKHS, when it is restricted to the sphere 𝕊^d-1 and when it is defined on the entire ℝ^d.

READ FULL TEXT

page 1

page 2

page 3

page 4

07/03/2020

On the Similarity between the Laplace and Neural Tangent Kernels

Recent theoretical work has shown that massively overparameterized neura...
03/02/2020

A Fractional Hawkes process

We modify ETAS models by replacing the Pareto-like kernel proposed by Og...
10/04/2021

Generalized Kernel Thinning

The kernel thinning (KT) algorithm of Dwivedi and Mackey (2021) compress...
06/01/2020

Analysis of Least Squares Regularized Regression in Reproducing Kernel Krein Spaces

In this paper, we study the asymptotical properties of least squares reg...
04/07/2021

Spectral Analysis of the Neural Tangent Kernel for Deep Residual Networks

Deep residual network architectures have been shown to achieve superior ...
06/02/2021

Transformers are Deep Infinite-Dimensional Non-Mercer Binary Kernel Machines

Despite their ubiquity in core AI fields like natural language processin...
05/19/2022

Consistent Interpolating Ensembles via the Manifold-Hilbert Kernel

Recent research in the theory of overparametrized learning has sought to...