Efficient Hyperparameter Tuning with Dynamic Accuracy Derivative-Free Optimization

11/06/2020
by   Matthias J. Ehrhardt, et al.
0

Many machine learning solutions are framed as optimization problems which rely on good hyperparameters. Algorithms for tuning these hyperparameters usually assume access to exact solutions to the underlying learning problem, which is typically not practical. Here, we apply a recent dynamic accuracy derivative-free optimization method to hyperparameter tuning, which allows inexact evaluations of the learning problem while retaining convergence guarantees. We test the method on the problem of learning elastic net weights for a logistic classifier, and demonstrate its robustness and efficiency compared to a fixed accuracy approach. This demonstrates a promising approach for hyperparameter tuning, with both convergence guarantees and practical performance.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
04/20/2018

Autotune: A Derivative-free Optimization Framework for Hyperparameter Tuning

Machine learning applications often require hyperparameter tuning. The h...
research
09/20/2023

A Region-Shrinking-Based Acceleration for Classification-Based Derivative-Free Optimization

Derivative-free optimization algorithms play an important role in scient...
research
08/21/2023

Relax and penalize: a new bilevel approach to mixed-binary hyperparameter optimization

In recent years, bilevel approaches have become very popular to efficien...
research
11/29/2022

Fast Hyperparameter Tuning for Ising Machines

In this paper, we propose a novel technique to accelerate Ising machines...
research
06/14/2020

The Statistical Cost of Robust Kernel Hyperparameter Tuning

This paper studies the statistical complexity of kernel hyperparameter t...
research
07/16/2023

MindOpt Tuner: Boost the Performance of Numerical Software by Automatic Parameter Tuning

Numerical software is usually shipped with built-in hyperparameters. By ...
research
12/03/2021

Enhanced Self-Organizing Map Solution for the Traveling Salesman Problem

Using an enhanced Self-Organizing Map method, we provided suboptimal sol...

Please sign up or login with your details

Forgot password? Click here to reset