DBA bandits: Self-driving index tuning under ad-hoc, analytical workloads with safety guarantees

10/19/2020
by   R. Malinga Perera, et al.
0

Automating physical database design has remained a long-term interest in database research due to substantial performance gains afforded by optimised structures. Despite significant progress, a majority of today's commercial solutions are highly manual, requiring offline invocation by database administrators (DBAs) who are expected to identify and supply representative training workloads. Unfortunately, the latest advancements like query stores provide only limited support for dynamic environments. This status quo is untenable: identifying representative static workloads is no longer realistic; and physical design tools remain susceptible to the query optimiser's cost misestimates (stemming from unrealistic assumptions such as attribute value independence and uniformity of data distribution). We propose a self-driving approach to online index selection that eschews the DBA and query optimiser, and instead learns the benefits of viable structures through strategic exploration and direct performance observation. We view the problem as one of sequential decision making under uncertainty, specifically within the bandit learning setting. Multi-armed bandits balance exploration and exploitation to provably guarantee average performance that converges to a fixed policy that is optimal with perfect hindsight. Our comprehensive empirical results demonstrate up to 75 workloads compared against a state-of-the-art commercial tuning tool.

READ FULL TEXT
research
08/23/2021

No DBA? No regret! Multi-armed bandits for index tuning of analytical and HTAP workloads with provable guarantees

Automating physical database design has remained a long-term interest in...
research
04/07/2018

IDEBench: A Benchmark for Interactive Data Exploration

Existing benchmarks for analytical database systems such as TPC-DS and T...
research
04/12/2017

Value Directed Exploration in Multi-Armed Bandits with Structured Priors

Multi-armed bandits are a quintessential machine learning problem requir...
research
07/20/2016

On the Identification and Mitigation of Weaknesses in the Knowledge Gradient Policy for Multi-Armed Bandits

The Knowledge Gradient (KG) policy was originally proposed for online ra...
research
03/28/2023

Dias: Dynamic Rewriting of Pandas Code

In recent years, dataframe libraries, such as pandas have exploded in po...
research
04/08/2020

Bao: Learning to Steer Query Optimizers

Query optimization remains one of the most challenging problems in data ...
research
01/21/2019

Predictive Indexing

There has been considerable research on automated index tuning in databa...

Please sign up or login with your details

Forgot password? Click here to reset