Model Embedding Model-Based Reinforcement Learning

06/16/2020
by   Xiaoyu Tan, et al.
0

Model-based reinforcement learning (MBRL) has shown its advantages in sample-efficiency over model-free reinforcement learning (MFRL). Despite the impressive results it achieves, it still faces a trade-off between the ease of data generation and model bias. In this paper, we propose a simple and elegant model-embedding model-based reinforcement learning (MEMB) algorithm in the framework of the probabilistic reinforcement learning. To balance the sample-efficiency and model bias, we exploit both real and imaginary data in the training. In particular, we embed the model in the policy update and learn Q and V functions from the real data set. We provide the theoretical analysis of MEMB with the Lipschitz continuity assumption on the model and policy. At last, we evaluate MEMB on several benchmarks and demonstrate our algorithm can achieve state-of-the-art performance.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/19/2019

When to Trust Your Model: Model-Based Policy Optimization

Designing effective model-based reinforcement learning algorithms is dif...
research
11/16/2021

On Effective Scheduling of Model-based Reinforcement Learning

Model-based reinforcement learning has attracted wide attention due to i...
research
08/15/2019

Model-based Lookahead Reinforcement Learning

Model-based Reinforcement Learning (MBRL) allows data-efficient learning...
research
01/04/2019

Accelerating Goal-Directed Reinforcement Learning by Model Characterization

We propose a hybrid approach aimed at improving the sample efficiency in...
research
10/19/2020

Model-based Policy Optimization with Unsupervised Model Adaptation

Model-based reinforcement learning methods learn a dynamics model with r...
research
03/02/2021

Minimax Model Learning

We present a novel off-policy loss function for learning a transition mo...
research
06/01/2023

What model does MuZero learn?

Model-based reinforcement learning has drawn considerable interest in re...

Please sign up or login with your details

Forgot password? Click here to reset