Evolutionary Planning in Latent Space

11/23/2020
by   Thor V. A. N. Olesen, et al.
0

Planning is a powerful approach to reinforcement learning with several desirable properties. However, it requires a model of the world, which is not readily available in many real-life problems. In this paper, we propose to learn a world model that enables Evolutionary Planning in Latent Space (EPLS). We use a Variational Auto Encoder (VAE) to learn a compressed latent representation of individual observations and extend a Mixture Density Recurrent Neural Network (MDRNN) to learn a stochastic, multi-modal forward model of the world that can be used for planning. We use the Random Mutation Hill Climbing (RMHC) to find a sequence of actions that maximize expected reward in this learned model of the world. We demonstrate how to build a model of the world by bootstrapping it with rollouts from a random policy and iteratively refining it with rollouts from an increasingly accurate planning policy using the learned world model. After a few iterations of this refinement, our planning agents are better than standard model-free reinforcement learning approaches demonstrating the viability of our approach.

READ FULL TEXT
research
12/09/2019

Learning Latent State Spaces for Planning through Reward Prediction

Model-based reinforcement learning methods typically learn models for hi...
research
09/12/2018

Coordinated Heterogeneous Distributed Perception based on Latent Space Representation

We investigate a reinforcement approach for distributed sensing based on...
research
11/16/2020

Distilling a Hierarchical Policy for Planning and Control via Representation and Reinforcement Learning

We present a hierarchical planning and control framework that enables an...
research
04/26/2019

Self Training Autonomous Driving Agent

Intrinsically, driving is a Markov Decision Process which suits well the...
research
06/07/2023

Dual policy as self-model for planning

Planning is a data efficient decision-making strategy where an agent sel...
research
03/27/2023

Ensemble Latent Space Roadmap for Improved Robustness in Visual Action Planning

Planning in learned latent spaces helps to decrease the dimensionality o...
research
02/04/2019

A Forest from the Trees: Generation through Neighborhoods

In this work, we propose to learn a generative model using both learned ...

Please sign up or login with your details

Forgot password? Click here to reset