Variational Inference for Data-Efficient Model Learning in POMDPs

05/23/2018
by   Sebastian Tschiatschek, et al.
0

Partially observable Markov decision processes (POMDPs) are a powerful abstraction for tasks that require decision making under uncertainty, and capture a wide range of real world tasks. Today, effective planning approaches exist that generate effective strategies given black-box models of a POMDP task. Yet, an open question is how to acquire accurate models for complex domains. In this paper we propose DELIP, an approach to model learning for POMDPs that utilizes amortized structured variational inference. We empirically show that our model leads to effective control strategies when coupled with state-of-the-art planners. Intuitively, model-based approaches should be particularly beneficial in environments with changing reward structures, or where rewards are initially unknown. Our experiments confirm that DELIP is particularly effective in this setting.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
03/09/2020

Learning discrete state abstractions with deep variational inference

Abstraction is crucial for effective sequential decision making in domai...
research
12/17/2021

Visual Learning-based Planning for Continuous High-Dimensional POMDPs

The Partially Observable Markov Decision Process (POMDP) is a powerful f...
research
02/02/2019

Variational Bayesian Decision-making for Continuous Utilities

Bayesian decision theory outlines a rigorous framework for making optima...
research
06/25/2021

Predictive Control Using Learned State Space Models via Rolling Horizon Evolution

A large part of the interest in model-based reinforcement learning deriv...
research
10/22/2021

Automatic Guide Generation for Stan via NumPyro

Stan is a very popular probabilistic language with a state-of-the-art HM...
research
01/13/2020

POPCORN: Partially Observed Prediction COnstrained ReiNforcement Learning

Many medical decision-making settings can be framed as partially observe...
research
05/07/2019

Optimal Control of Complex Systems through Variational Inference with a Discrete Event Decision Process

Complex social systems are composed of interconnected individuals whose ...

Please sign up or login with your details

Forgot password? Click here to reset