Importance of Tuning Hyperparameters of Machine Learning Algorithms

07/15/2020
by   Hilde J. P. Weerts, et al.
0

The performance of many machine learning algorithms depends on their hyperparameter settings. The goal of this study is to determine whether it is important to tune a hyperparameter or whether it can be safely set to a default value. We present a methodology to determine the importance of tuning a hyperparameter based on a non-inferiority test and tuning risk: the performance loss that is incurred when a hyperparameter is not tuned, but set to a default value. Because our methods require the notion of a default parameter, we present a simple procedure that can be used to determine reasonable default parameters. We apply our methods in a benchmark study using 59 datasets from OpenML. Our results show that leaving particular hyperparameters at their default value is non-inferior to tuning these hyperparameters. In some cases, leaving the hyperparameter at its default value even outperforms tuning it using a search procedure with a limited number of iterations.

READ FULL TEXT

page 7

page 13

page 16

page 17

02/26/2018

Tunability: Importance of Hyperparameters of Machine Learning Algorithms

Modern machine learning algorithms for classification or regression such...
07/13/2022

High Per Parameter: A Large-Scale Study of Hyperparameter Tuning for Machine Learning Algorithms

Hyperparameters in machine learning (ML) have received a fair amount of ...
07/19/2021

Experimental Investigation and Evaluation of Model-based Hyperparameter Optimization

Machine learning algorithms such as random forests or xgboost are gainin...
01/13/2022

Hyperparameter Importance for Machine Learning Algorithms

Hyperparameter plays an essential role in the fitting of supervised mach...
06/04/2019

A meta-learning recommender system for hyperparameter tuning: predicting when tuning improves SVM classifiers

For many machine learning algorithms, predictive performance is critical...
01/16/2021

JITuNE: Just-In-Time Hyperparameter Tuning for Network Embedding Algorithms

Network embedding (NE) can generate succinct node representations for ma...
04/20/2018

Autotune: A Derivative-free Optimization Framework for Hyperparameter Tuning

Machine learning applications often require hyperparameter tuning. The h...