Bayesian Optimization for auto-tuning GPU kernels

11/26/2021
by   Floris-Jan Willemsen, et al.
0

Finding optimal parameter configurations for tunable GPU kernels is a non-trivial exercise for large search spaces, even when automated. This poses an optimization task on a non-convex search space, using an expensive to evaluate function with unknown derivative. These characteristics make a good candidate for Bayesian Optimization, which has not been applied to this problem before. However, the application of Bayesian Optimization to this problem is challenging. We demonstrate how to deal with the rough, discrete, constrained search spaces, containing invalid configurations. We introduce a novel contextual variance exploration factor, as well as new acquisition functions with improved scalability, combined with an informed acquisition function selection mechanism. By comparing the performance of our Bayesian Optimization implementation on various test cases to the existing search strategies in Kernel Tuner, as well as other Bayesian Optimization implementations, we demonstrate that our search strategies generalize well and consistently outperform other search strategies by a wide margin.

READ FULL TEXT
research
06/22/2020

MUMBO: MUlti-task Max-value Bayesian Optimization

We propose MUMBO, the first high-performing yet computationally efficien...
research
12/16/2021

BoGraph: Structured Bayesian Optimization From Logs for Expensive Systems with Many Parameters

Current auto-tuning frameworks struggle with tuning computer systems con...
research
10/15/2020

Autotuning PolyBench Benchmarks with LLVM Clang/Polly Loop Optimization Pragmas Using Bayesian Optimization

An autotuning is an approach that explores a search space of possible im...
research
05/29/2023

Identification of stormwater control strategies and their associated uncertainties using Bayesian Optimization

Dynamic control is emerging as an effective methodology for operating st...
research
10/29/2019

Bayesian Optimization with Unknown Search Space

Applying Bayesian optimization in problems wherein the search space is u...
research
11/02/2021

Geometry-aware Bayesian Optimization in Robotics using Riemannian Matérn Kernels

Bayesian optimization is a data-efficient technique which can be used fo...
research
12/14/2021

Triangulation candidates for Bayesian optimization

Bayesian optimization is a form of sequential design: idealize input-out...

Please sign up or login with your details

Forgot password? Click here to reset