Model Agnostic Conformal Hyperparameter Optimization

07/06/2022
by   Riccardo Doyle, et al.
0

Several novel frameworks for hyperparameter search have emerged in the last decade, but most rely on strict, often normal, distributional assumptions, limiting search model flexibility. This paper proposes a novel optimization framework based on Conformal prediction, assuming only exchangeability, and allowing for a larger choice of search model architectures and variance estimators. Several such models are explored and benchmarked against random hyperparameter search on both dense and convolutional neural networks with consistent overperformance both in final loss achieved and time to achievement.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
03/30/2020

Weighted Random Search for CNN Hyperparameter Optimization

Nearly all model algorithms used in machine learning use two different s...
research
06/06/2017

Random Search for Hyperparameters using Determinantal Point Processes

We propose the use of k-determinantal point processes in hyperparameter ...
research
06/18/2020

Multi-Source Unsupervised Hyperparameter Optimization

How can we conduct efficient hyperparameter optimization for a completel...
research
02/24/2022

DC and SA: Robust and Efficient Hyperparameter Optimization of Multi-subnetwork Deep Learning Models

We present two novel hyperparameter optimization strategies for optimiza...
research
11/22/2020

A Population-based Hybrid Approach to Hyperparameter Optimization for Neural Networks

In recent years, large amounts of data have been generated, and computer...
research
05/30/2017

Accelerating Neural Architecture Search using Performance Prediction

Methods for neural network hyperparameter optimization and meta-modeling...
research
02/27/2020

Using a thousand optimization tasks to learn hyperparameter search strategies

We present TaskSet, a dataset of tasks for use in training and evaluatin...

Please sign up or login with your details

Forgot password? Click here to reset