Bayesian Reinforcement Learning in Factored POMDPs

11/14/2018
by   Sammie Katt, et al.
0

Bayesian approaches provide a principled solution to the exploration-exploitation trade-off in Reinforcement Learning. Typical approaches, however, either assume a fully observable environment or scale poorly. This work introduces the Factored Bayes-Adaptive POMDP model, a framework that is able to exploit the underlying structure while learning the dynamics in partially observable systems. We also present a belief tracking method to approximate the joint posterior over state and model variables, and an adaptation of the Monte-Carlo Tree Search solution method, which together are capable of solving the underlying problem near-optimally. Our method is able to learn efficiently given a known factorization or also learn the factorization and the model parameters at the same time. We demonstrate that this approach is able to outperform current methods and tackle problems that were previously infeasible.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/14/2018

Learning in POMDPs with Monte Carlo Tree Search

The POMDP is a powerful framework for reasoning under outcome and inform...
research
02/17/2022

BADDr: Bayes-Adaptive Deep Dropout RL for POMDPs

While reinforcement learning (RL) has made great advances in scalability...
research
07/22/2023

On-Robot Bayesian Reinforcement Learning for POMDPs

Robot learning is often difficult due to the expense of gathering data. ...
research
06/27/2012

Monte Carlo Bayesian Reinforcement Learning

Bayesian reinforcement learning (BRL) encodes prior knowledge of the wor...
research
06/13/2012

Model-Based Bayesian Reinforcement Learning in Large Structured Domains

Model-based Bayesian reinforcement learning has generated significant in...
research
05/25/2023

Bayesian Reinforcement Learning for Automatic Voltage Control under Cyber-Induced Uncertainty

Voltage control is crucial to large-scale power system reliable operatio...
research
10/13/2015

Dual Control for Approximate Bayesian Reinforcement Learning

Control of non-episodic, finite-horizon dynamical systems with uncertain...

Please sign up or login with your details

Forgot password? Click here to reset