GPU Accelerated Exhaustive Search for Optimal Ensemble of Black-Box Optimization Algorithms

12/08/2020
by   Jiwei Liu, et al.
44

Black-box optimization is essential for tuning complex machine learning algorithms which are easier to experiment with than to understand. In this paper, we show that a simple ensemble of black-box optimization algorithms can outperform any single one of them. However, searching for such an optimal ensemble requires a large number of experiments. We propose a Multi-GPU-optimized framework to accelerate a brute force search for the optimal ensemble of black-box optimization algorithms by running many experiments in parallel. The lightweight optimizations are performed by CPU while expensive model training and evaluations are assigned to GPUs. The multi-GPU solution achieves 10x speedup of the CPU implementation. With the optimal ensemble found by GPU-accelerated exhaustive search, we won the 2nd place of NeurIPS 2020 black-box optimization challenge.

READ FULL TEXT
research
10/11/2018

Multi-Strategy Coevolving Aging Particle Optimization

We propose Multi-Strategy Coevolving Aging Particles (MS-CAP), a novel p...
research
06/17/2021

Optimum-statistical collaboration towards efficient black-box optimization

With increasingly more hyperparameters involved in their training, machi...
research
06/26/2021

Quantum Dynamics Interpretation of Black-box Optimization

In recent decades, with the emergence of numerous novel intelligent opti...
research
03/10/2017

Evolution Strategies as a Scalable Alternative to Reinforcement Learning

We explore the use of Evolution Strategies (ES), a class of black box op...
research
05/03/2023

Black-box Optimizers vs Taste Shocks

We evaluate and extend the solution methods for models with binary and m...
research
10/24/2018

Noisy Blackbox Optimization with Multi-Fidelity Queries: A Tree Search Approach

We study the problem of black-box optimization of a noisy function in th...

Please sign up or login with your details

Forgot password? Click here to reset