DeepAI AI Chat
Log In Sign Up

A Comparative Tutorial of Bayesian Sequential Design and Reinforcement Learning

by   Mauricio Tec, et al.

Reinforcement Learning (RL) is a computational approach to reward-driven learning in sequential decision problems. It implements the discovery of optimal actions by learning from an agent interacting with an environment rather than from supervised data. We contrast and compare RL with traditional sequential design, focusing on simulation-based Bayesian sequential design (BSD). Recently, there has been an increasing interest in RL techniques for healthcare applications. We introduce two related applications as motivating examples. In both applications, the sequential nature of the decisions is restricted to sequential stopping. Rather than a comprehensive survey, the focus of the discussion is on solutions using standard tools for these two relatively simple sequential stopping problems. Both problems are inspired by adaptive clinical trial design. We use examples to explain the terminology and mathematical background that underlie each framework and map one to the other. The implementations and results illustrate the many similarities between RL and BSD. The results motivate the discussion of the potential strengths and limitations of each approach.


page 11

page 13


A Survey on Reinforcement Learning in Aviation Applications

Compared with model-based control and optimization methods, reinforcemen...

Reinforcement Learning in Healthcare: A Survey

As a subfield of machine learning, reinforcement learning (RL) aims at e...

Datasets for Online Controlled Experiments

Online Controlled Experiments (OCE) are the gold standard to measure imp...

Deep Reinforcement Learning for Optimal Stopping with Application in Financial Engineering

Optimal stopping is the problem of deciding the right time at which to t...

Sequential Bayesian experimental designs via reinforcement learning

Bayesian experimental design (BED) has been used as a method for conduct...

Deep Adaptive Design: Amortizing Sequential Bayesian Experimental Design

We introduce Deep Adaptive Design (DAD), a method for amortizing the cos...