LIFT: Reinforcement Learning in Computer Systems by Learning From Demonstrations

by   Michael Schaarschmidt, et al.
University of Cambridge
Baqend GmbH

Reinforcement learning approaches have long appealed to the data management community due to their ability to learn to control dynamic behavior from raw system performance. Recent successes in combining deep neural networks with reinforcement learning have sparked significant new interest in this domain. However, practical solutions remain elusive due to large training data requirements, algorithmic instability, and lack of standard tools. In this work, we introduce LIFT, an end-to-end software stack for applying deep reinforcement learning to data management tasks. While prior work has frequently explored applications in simulations, LIFT centers on utilizing human expertise to learn from demonstrations, thus lowering online training times. We further introduce TensorForce, a TensorFlow library for applied deep reinforcement learning exposing a unified declarative interface to common RL algorithms, thus providing a backend to LIFT. We demonstrate the utility of LIFT in two case studies in database compound indexing and resource management in stream processing. Results show LIFT controllers initialized from demonstrations can outperform human baselines and heuristics across latency metrics and space usage by up to 70


Benchmarking Robustness of Deep Reinforcement Learning approaches to Online Portfolio Management

Deep Reinforcement Learning approaches to Online Portfolio Selection hav...

Deep Reinforcement Learning for Robotic Manipulation with Asynchronous Off-Policy Updates

Reinforcement learning holds the promise of enabling autonomous robots t...

A Survey on Deep Reinforcement Learning for Data Processing and Analytics

Data processing and analytics are fundamental and pervasive. Algorithms ...

Intelligent Replication Management for HDFS Using Reinforcement Learning

Storage systems for cloud computing merge a large number of commodity co...

Checklist Models for Improved Output Fluency in Piano Fingering Prediction

In this work we present a new approach for the task of predicting finger...

Experience-Enhanced Learning: One Size Still does not Fit All in Automatic Database

Recent years, the database committee has attempted to develop automatic ...

Deep Inventory Management

We present a Deep Reinforcement Learning approach to solving a periodic ...

Please sign up or login with your details

Forgot password? Click here to reset