An Empirical Approach For Probing the Definiteness of Kernels

07/10/2018
by   Martin Zaefferer, et al.
0

Models like support vector machines or Gaussian process regression often require positive semi-definite kernels. These kernels may be based on distance functions. While definiteness is proven for common distances and kernels, a proof for a new kernel may require too much time and effort for users who simply aim at practical usage. Furthermore, designing definite distances or kernels may be equally intricate. Finally, models can be enabled to use indefinite kernels. This may deteriorate the accuracy or computational cost of the model. Hence, an efficient method to determine definiteness is required. We propose an empirical approach. We show that sampling as well as optimization with an evolutionary algorithm may be employed to determine definiteness. We provide a proof-of-concept with 16 different distance measures for permutations. Our approach allows to disprove definiteness if a respective counter-example is found. It can also provide an estimate of how likely it is to obtain indefinite kernel matrices. This provides a simple, efficient tool to decide whether additional effort should be spent on designing/selecting a more suitable kernel or algorithm.

READ FULL TEXT
research
10/09/2015

On the Definiteness of Earth Mover's Distance Yields and Its Relation to Set Intersection

Positive definite kernels are an important tool in machine learning that...
research
09/06/2012

Solving Support Vector Machines in Reproducing Kernel Banach Spaces with Positive Definite Functions

In this paper we solve support vector machines in reproducing kernel Ban...
research
02/13/2015

The Responsibility Weighted Mahalanobis Kernel for Semi-Supervised Training of Support Vector Machines for Classification

Kernel functions in support vector machines (SVM) are needed to assess t...
research
06/07/2015

Generalized Spectral Kernels

In this paper we propose a family of tractable kernels that is dense in ...
research
05/30/2020

Generalizing Random Fourier Features via Generalized Measures

We generalize random Fourier features, that usually require kernel funct...
research
06/12/2015

Optimal γ and C for ε-Support Vector Regression with RBF Kernels

The objective of this study is to investigate the efficient determinatio...
research
09/27/2021

Learning from Small Samples: Transformation-Invariant SVMs with Composition and Locality at Multiple Scales

Motivated by the problem of learning when the number of training samples...

Please sign up or login with your details

Forgot password? Click here to reset