Squirrel: A Switching Hyperparameter Optimizer

12/15/2020
by   Noor Awad, et al.
26

In this short note, we describe our submission to the NeurIPS 2020 BBO challenge. Motivated by the fact that different optimizers work well on different problems, our approach switches between different optimizers. Since the team names on the competition's leaderboard were randomly generated "alliteration nicknames", consisting of an adjective and an animal with the same initial letter, we called our approach the Switching Squirrel, or here, short, Squirrel.

READ FULL TEXT

page 1

page 2

page 3

11/17/2014

Feedback Solution to Optimal Switching Problems with Switching Cost

The problem of optimal switching between nonlinear autonomous subsystems...
10/02/2021

Multi-Feasibility Variable Selection

This paper is the report of the problem proposed for the !Optimizer 2021...
02/15/2022

A short note on an inequality between KL and TV

The goal of this short note is to discuss the relation between Kullback–...
10/25/2019

On the Tunability of Optimizers in Deep Learning

There is no consensus yet on the question whether adaptive gradient meth...
06/03/2020

A quest for a fair schedule: The Young Physicists' Tournament

The Young Physicists Tournament is an established team-oriented scientif...
11/16/2020

The Person Index Challenge: Extraction of Persons from Messy, Short Texts

When persons are mentioned in texts with their first name, last name and...