A Kernel-Based Approach to Non-Stationary Reinforcement Learning in Metric Spaces

07/09/2020 ∙ by Omar Darwiche Domingues, et al. ∙ 51 ∙ share

In this work, we propose KeRNS: an algorithm for episodic reinforcement learning in non-stationary Markov Decision Processes (MDPs) whose state-action set is endowed with a metric. Using a non-parametric model of the MDP built with time-dependent kernels, we prove a regret bound that scales with the covering dimension of the state-action space and the total variation of the MDP with time, which quantifies its level of non-stationarity. Our method generalizes previous approaches based on sliding windows and exponential discounting used to handle changing environments. We further propose a practical implementation of KeRNS, we analyze its regret and validate it experimentally.

READ FULL TEXT

Variational Regret Bounds for Reinforcement Learning

We consider undiscounted reinforcement learning in Markov decision proce...
05/14/2019 ∙ by Pratik Gajane, et al. ∙ 0 ∙ share

read it
Regret Bounds for Kernel-Based Reinforcement Learning

We consider the exploration-exploitation dilemma in finite-horizon reinf...
04/12/2020 ∙ by Omar Darwiche Domingues, et al. ∙ 0 ∙ share

read it
Reinforcement Learning for Non-Stationary Markov Decision Processes: The Blessing of (More) Optimism

We consider un-discounted reinforcement learning (RL) in Markov decision...
06/24/2020 ∙ by Wang Chi Cheung, et al. ∙ 0 ∙ share

read it
A Sliding-Window Algorithm for Markov Decision Processes with Arbitrarily Changing Rewards and Transitions

We consider reinforcement learning in changing Markov Decision Processes...
05/25/2018 ∙ by Pratik Gajane, et al. ∙ 0 ∙ share

read it
Optimizing for the Future in Non-Stationary MDPs

Most reinforcement learning methods are based upon the key assumption th...
05/17/2020 ∙ by Yash Chandak, et al. ∙ 2 ∙ share

read it
Non-Stationary Markov Decision Processes a Worst-Case Approach using Model-Based Reinforcement Learning

This work tackles the problem of robust zero-shot planning in non-statio...
04/22/2019 ∙ by Erwan Lecarpentier, et al. ∙ 0 ∙ share

read it
Provably adaptive reinforcement learning in metric spaces

We study reinforcement learning in continuous state and action spaces en...
06/18/2020 ∙ by Tongyi Cao, et al. ∙ 0 ∙ share

read it

A Kernel-Based Approach to Non-Stationary Reinforcement Learning in Metric Spaces

Authors

Variational Regret Bounds for Reinforcement Learning

Regret Bounds for Kernel-Based Reinforcement Learning

Reinforcement Learning for Non-Stationary Markov Decision Processes: The Blessing of (More) Optimism

A Sliding-Window Algorithm for Markov Decision Processes with Arbitrarily Changing Rewards and Transitions

Optimizing for the Future in Non-Stationary MDPs

Non-Stationary Markov Decision Processes a Worst-Case Approach using Model-Based Reinforcement Learning

Provably adaptive reinforcement learning in metric spaces

This week in AI