Bounded Optimal Exploration in MDP

04/05/2016 ∙ by Kenji Kawaguchi, et al. ∙ 0 ∙ share

Within the framework of probably approximately correct Markov decision processes (PAC-MDP), much theoretical work has focused on methods to attain near optimality after a relatively long period of learning and exploration. However, practical concerns require the attainment of satisfactory behavior within a short period of time. In this paper, we relax the PAC-MDP conditions to reconcile theoretically driven exploration methods and practical needs. We propose simple algorithms for discrete and continuous state spaces, and illustrate the benefits of our proposed relaxation via theoretical analyses and numerical examples. Our algorithms also maintain anytime error bounds and average loss bounds. Our approach accommodates both Bayesian and non-Bayesian methods.

READ FULL TEXT

Exploration Bonus for Regret Minimization in Undiscounted Discrete and Continuous Markov Decision Processes

We introduce and analyse two algorithms for exploration-exploitation in ...
12/11/2018 ∙ by Jian Qian, et al. ∙ 0 ∙ share

read it
Of Cores: A Partial-Exploration Framework for Markov Decision Processes

We introduce a framework for approximate analysis of Markov decision pro...
06/17/2019 ∙ by Jan Křetínský, et al. ∙ 0 ∙ share

read it
Bayes-CPACE: PAC Optimal Exploration in Continuous Space Bayes-Adaptive Markov Decision Processes

We present the first PAC optimal algorithm for Bayes-Adaptive Markov Dec...
10/06/2018 ∙ by Gilwoo Lee, et al. ∙ 0 ∙ share

read it
Markov Decision Processes with Continuous Side Information

We consider a reinforcement learning (RL) setting in which the agent int...
11/15/2017 ∙ by Aditya Modi, et al. ∙ 0 ∙ share

read it
Refined Analysis of FPL for Adversarial Markov Decision Processes

We consider the adversarial Markov Decision Process (MDP) problem, where...
08/21/2020 ∙ by Yuanhao Wang, et al. ∙ 5 ∙ share

read it
Directed Exploration in PAC Model-Free Reinforcement Learning

We study an exploration method for model-free RL that generalizes the co...
08/31/2018 ∙ by Min-hwan Oh, et al. ∙ 0 ∙ share

read it
Bayesian Residual Policy Optimization: Scalable Bayesian Reinforcement Learning with Clairvoyant Experts

Informed and robust decision making in the face of uncertainty is critic...
02/07/2020 ∙ by Gilwoo Lee, et al. ∙ 0 ∙ share

read it

Bounded Optimal Exploration in MDP

Authors

Exploration Bonus for Regret Minimization in Undiscounted Discrete and Continuous Markov Decision Processes

Of Cores: A Partial-Exploration Framework for Markov Decision Processes

Bayes-CPACE: PAC Optimal Exploration in Continuous Space Bayes-Adaptive Markov Decision Processes

Markov Decision Processes with Continuous Side Information

Refined Analysis of FPL for Adversarial Markov Decision Processes

Directed Exploration in PAC Model-Free Reinforcement Learning

Bayesian Residual Policy Optimization: Scalable Bayesian Reinforcement Learning with Clairvoyant Experts

This week in AI