Several Tunable GMM Kernels

05/08/2018
by   Ping Li, et al.
0

While tree methods have been popular in practice, researchers and practitioners are also looking for simple algorithms which can reach similar accuracy of trees. In 2010, (Ping Li UAI'10) developed the method of "abc-robust-logitboost" and compared it with other supervised learning methods on datasets used by the deep learning literature. In this study, we propose a series of "tunable GMM kernels" which are simple and perform largely comparably to tree methods on the same datasets. Note that "abc-robust-logitboost" substantially improved the original "GDBT" in that (a) it developed a tree-split formula based on second-order information of the derivatives of the loss function; (b) it developed a new set of derivatives for multi-class classification formulation. In the prior study in 2017, the "generalized min-max" (GMM) kernel was shown to have good performance compared to the "radial-basis function" (RBF) kernel. However, as demonstrated in this paper, the original GMM kernel is often not as competitive as tree methods on the datasets used in the deep learning literature. Since the original GMM kernel has no parameters, we propose tunable GMM kernels by adding tuning parameters in various ways. Three basic (i.e., with only one parameter) GMM kernels are the "eGMM kernel", "pGMM kernel", and "γGMM kernel", respectively. Extensive experiments show that they are able to produce good results for a large number of classification tasks. Furthermore, the basic kernels can be combined to boost the performance.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
01/09/2017

Tunable GMM Kernels

The recently proposed "generalized min-max" (GMM) kernel can be efficien...
research
07/12/2016

Nystrom Method for Approximating the GMM Kernel

The GMM (generalized min-max) kernel was recently proposed (Li, 2016) as...
research
03/21/2016

A Comparison Study of Nonlinear Kernels

In this paper, we compare 5 different nonlinear kernels: min-max, RBF, f...
research
07/18/2022

Package for Fast ABC-Boost

This report presents the open-source package which implements the series...
research
07/18/2022

pGMM Kernel Regression and Comparisons with Boosted Trees

In this work, we demonstrate the advantage of the pGMM (“powered general...
research
05/01/2022

Generalized Reference Kernel for One-class Classification

In this paper, we formulate a new generalized reference kernel hoping to...
research
04/10/2019

The Weight Function in the Subtree Kernel is Decisive

Tree data are ubiquitous because they model a large variety of situation...

Please sign up or login with your details

Forgot password? Click here to reset