Reinforcement Learning of POMDPs using Spectral Methods

02/25/2016
by   Kamyar Azizzadenesheli, et al.
0

We propose a new reinforcement learning algorithm for partially observable Markov decision processes (POMDP) based on spectral decomposition methods. While spectral methods have been previously employed for consistent learning of (passive) latent variable models such as hidden Markov models, POMDPs are more challenging since the learner interacts with the environment and possibly changes the future observations in the process. We devise a learning algorithm running through episodes, in each episode we employ spectral techniques to learn the POMDP parameters from a trajectory generated by a fixed policy. At the end of the episode, an optimization oracle returns the optimal memoryless planning policy which maximizes the expected reward based on the estimated POMDP model. We prove an order-optimal regret bound with respect to the optimal memoryless policy and efficient scaling with respect to the dimensionality of observation and action spaces.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
05/07/2017

Experimental results : Reinforcement Learning of POMDPs using Spectral Methods

We propose a new reinforcement learning algorithm for partially observab...
research
07/08/2021

Sublinear Regret for Learning POMDPs

We study the model-based undiscounted reinforcement learning for partial...
research
05/18/2006

Cross-Entropic Learning of a Machine for the Decision in a Partially Observable Universe

Revision of the paper previously entitled "Learning a Machine for the De...
research
06/07/2022

Learning in Observable POMDPs, without Computationally Intractable Oracles

Much of reinforcement learning theory is built on top of oracles that ar...
research
09/22/2021

A Spectral Approach to Off-Policy Evaluation for POMDPs

We consider off-policy evaluation (OPE) in Partially Observable Markov D...
research
02/08/2016

Data-Efficient Reinforcement Learning in Continuous-State POMDPs

We present a data-efficient reinforcement learning algorithm resistant t...
research
11/12/2019

Efficient Planning under Partial Observability with Unnormalized Q Functions and Spectral Learning

Learning and planning in partially-observable domains is one of the most...

Please sign up or login with your details

Forgot password? Click here to reset