Dream and Search to Control: Latent Space Planning for Continuous Control

10/19/2020
by   Anurag Koul, et al.
0

Learning and planning with latent space dynamics has been shown to be useful for sample efficiency in model-based reinforcement learning (MBRL) for discrete and continuous control tasks. In particular, recent work, for discrete action spaces, demonstrated the effectiveness of latent-space planning via Monte-Carlo Tree Search (MCTS) for bootstrapping MBRL during learning and at test time. However, the potential gains from latent-space tree search have not yet been demonstrated for environments with continuous action spaces. In this work, we propose and explore an MBRL approach for continuous action spaces based on tree-based planning over learned latent dynamics. We show that it is possible to demonstrate the types of bootstrapping benefits as previously shown for discrete spaces. In particular, the approach achieves improved sample efficiency and performance on a majority of challenging continuous-control benchmarks compared to the state-of-the-art.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
03/27/2023

Ensemble Latent Space Roadmap for Improved Robustness in Visual Action Planning

Planning in learned latent spaces helps to decrease the dimensionality o...
research
04/04/2014

Scalable Planning and Learning for Multiagent POMDPs: Extended Version

Online, sample-based planning algorithms for POMDPs have shown great pro...
research
06/20/2019

Exploring Model-based Planning with Policy Networks

Model-based reinforcement learning (MBRL) with model-predictive control ...
research
05/24/2018

A0C: Alpha Zero in Continuous Action Space

A core novelty of Alpha Zero is the interleaving of tree search and deep...
research
09/07/2018

Monte Carlo Tree Search with Scalable Simulation Periods for Continuously Running Tasks

Monte Carlo Tree Search (MCTS) is particularly adapted to domains where ...
research
06/10/2020

Marginal Utility for Planning in Continuous or Large Discrete Action Spaces

Sample-based planning is a powerful family of algorithms for generating ...
research
09/29/2022

Learning Parsimonious Dynamics for Generalization in Reinforcement Learning

Humans are skillful navigators: We aptly maneuver through new places, re...

Please sign up or login with your details

Forgot password? Click here to reset