Non-Markovian Reinforcement Learning using Fractional Dynamics

07/29/2021
by   Gaurav Gupta, et al.
0

Reinforcement learning (RL) is a technique to learn the control policy for an agent that interacts with a stochastic environment. In any given state, the agent takes some action, and the environment determines the probability distribution over the next state as well as gives the agent some reward. Most RL algorithms typically assume that the environment satisfies Markov assumptions (i.e. the probability distribution over the next state depends only on the current state). In this paper, we propose a model-based RL technique for a system that has non-Markovian dynamics. Such environments are common in many real-world applications such as in human physiology, biological systems, material science, and population dynamics. Model-based RL (MBRL) techniques typically try to simultaneously learn a model of the environment from the data, as well as try to identify an optimal policy for the learned model. We propose a technique where the non-Markovianity of the system is modeled through a fractional dynamical system. We show that we can quantify the difference in the performance of an MBRL algorithm that uses bounded horizon model predictive control from the optimal policy. Finally, we demonstrate our proposed framework on a pharmacokinetic model of human blood glucose dynamics and show that our fractional models can capture distant correlations on real-world datasets.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/21/2022

Finding Optimal Policy for Queueing Models: New Parameterization

Queueing systems appear in many important real-life applications includi...
research
11/03/2022

Contrastive Value Learning: Implicit Models for Simple Offline RL

Model-based reinforcement learning (RL) methods are appealing in the off...
research
03/16/2021

Few Shot System Identification for Reinforcement Learning

Learning by interaction is the key to skill acquisition for most living ...
research
11/28/2022

Inapplicable Actions Learning for Knowledge Transfer in Reinforcement Learning

Reinforcement Learning (RL) algorithms are known to scale poorly to envi...
research
12/18/2017

Multi-Fidelity Reinforcement Learning with Gaussian Processes

This paper studies the problem of Reinforcement Learning (RL) using as f...
research
12/14/2021

Quantifying Multimodality in World Models

Model-based Deep Reinforcement Learning (RL) assumes the availability of...
research
05/24/2018

Intelligent Trainer for Model-Based Reinforcement Learning

Model-based deep reinforcement learning (DRL) algorithm uses the sampled...

Please sign up or login with your details

Forgot password? Click here to reset