Sequential Bayesian experimental designs via reinforcement learning

by   Hikaru Asano, et al.

Bayesian experimental design (BED) has been used as a method for conducting efficient experiments based on Bayesian inference. The existing methods, however, mostly focus on maximizing the expected information gain (EIG); the cost of experiments and sample efficiency are often not taken into account. In order to address this issue and enhance practical applicability of BED, we provide a new approach Sequential Experimental Design via Reinforcement Learning to construct BED in a sequential manner by applying reinforcement learning in this paper. Here, reinforcement learning is a branch of machine learning in which an agent learns a policy to maximize its reward by interacting with the environment. The characteristics of interacting with the environment are similar to the sequential experiment, and reinforcement learning is indeed a method that excels at sequential decision making. By proposing a new real-world-oriented experimental environment, our approach aims to maximize the EIG while keeping the cost of experiments and sample efficiency in mind simultaneously. We conduct numerical experiments for three different examples. It is confirmed that our method outperforms the existing methods in various indices such as the EIG and sampling efficiency, indicating that our proposed method and experimental environment can make a significant contribution to application of BED to the real world.


A Mixture of Surprises for Unsupervised Reinforcement Learning

Unsupervised reinforcement learning aims at learning a generalist policy...

Reinforcement Learning with Efficient Active Feature Acquisition

Solving real-life sequential decision making problems under partial obse...

Experimentation Platforms Meet Reinforcement Learning: Bayesian Sequential Decision-Making for Continuous Monitoring

With the growing needs of online A/B testing to support the innovation i...

Improving Cost Learning for JPEG Steganography by Exploiting JPEG Domain Knowledge

Although significant progress in automatic learning of steganographic co...

Data-Efficient Reinforcement Learning for Malaria Control

Sequential decision-making under cost-sensitive tasks is prohibitively d...

A Comparative Tutorial of Bayesian Sequential Design and Reinforcement Learning

Reinforcement Learning (RL) is a computational approach to reward-driven...

Environment Reconstruction with Hidden Confounders for Reinforcement Learning based Recommendation

Reinforcement learning aims at searching the best policy model for decis...

Please sign up or login with your details

Forgot password? Click here to reset