Heuristical choice of SVM parameters

11/03/2021
by   Michał Cholewa, et al.
7

Support Vector Machine (SVM) is one of the most popular classification methods, and a de-facto reference for many Machine Learning approaches. Its performance is determined by parameter selection, which is usually achieved by a time-consuming grid search cross-validation procedure. There exist, however, several unsupervised heuristics that take advantage of the characteristics of the dataset for selecting parameters instead of using class label information. Unsupervised heuristics, while an order of magnitude faster, are scarcely used under the assumption that their results are significantly worse than those of grid search. To challenge that assumption we have conducted a wide study of various heuristics for SVM parameter selection on over thirty datasets, in both supervised and semi-supervised scenarios. In most cases, the cross-validation grid search did not achieve a significant advantage over the heuristics. In particular, heuristical parameter selection may be preferable for high dimensional and unbalanced datasets or when a small number of examples is available. Our results also show that using a heuristic to determine the starting point of further cross-validation does not yield significantly better results than the default start.

READ FULL TEXT

page 6

page 12

page 15

page 16

page 17

page 18

page 20

research
02/10/2016

Fast model selection by limiting SVM training times

Kernelized Support Vector Machines (SVMs) are among the best performing ...
research
05/17/2023

Separability and Scatteredness (S S) Ratio-Based Efficient SVM Regularization Parameter, Kernel, and Kernel Parameter Selection

Support Vector Machine (SVM) is a robust machine learning algorithm with...
research
04/03/2017

Geometric Insights into Support Vector Machine Behavior using the KKT Conditions

The Support Vector Machine (SVM) is a powerful and widely used classific...
research
07/14/2023

MaxMin-L2-SVC-NCH: A New Method to Train Support Vector Classifier with the Selection of Model's Parameters

The selection of model's parameters plays an important role in the appli...
research
02/13/2019

Efficient Cross-Validation for Semi-Supervised Learning

Manifold regularization, such as laplacian regularized least squares (La...
research
02/04/2021

Optimised one-class classification performance

We provide a thorough treatment of hyperparameter optimisation for three...
research
07/16/2021

In-Bed Person Monitoring Using Thermal Infrared Sensors

The world is expecting an aging population and shortage of healthcare pr...

Please sign up or login with your details

Forgot password? Click here to reset