SAGE: Generating Symbolic Goals for Myopic Models in Deep Reinforcement Learning

03/09/2022
by   Andrew Chester, et al.
0

Model-based reinforcement learning algorithms are typically more sample efficient than their model-free counterparts, especially in sparse reward problems. Unfortunately, many interesting domains are too complex to specify the complete models required by traditional model-based approaches. Learning a model takes a large number of environment samples, and may not capture critical information if the environment is hard to explore. If we could specify an incomplete model and allow the agent to learn how best to use it, we could take advantage of our partial understanding of many domains. Existing hybrid planning and learning systems which address this problem often impose highly restrictive assumptions on the sorts of models which can be used, limiting their applicability to a wide range of domains. In this work we propose SAGE, an algorithm combining learning and planning to exploit a previously unusable class of incomplete models. This combines the strengths of symbolic planning and neural learning approaches in a novel way that outperforms competing methods on variations of taxi world and Minecraft.

READ FULL TEXT

page 1

page 8

research
08/08/2017

Neural Network Dynamics for Model-Based Deep Reinforcement Learning with Model-Free Fine-Tuning

Model-free deep reinforcement learning algorithms have been shown to be ...
research
07/17/2021

High-Accuracy Model-Based Reinforcement Learning, a Survey

Deep reinforcement learning has shown remarkable success in the past few...
research
08/11/2020

Model-Based Deep Reinforcement Learning for High-Dimensional Problems, a Survey

Deep reinforcement learning has shown remarkable success in the past few...
research
06/16/2021

Contrastive Reinforcement Learning of Symbolic Reasoning Domains

Abstract symbolic reasoning, as required in domains such as mathematics ...
research
05/28/2021

Learning Neuro-Symbolic Relational Transition Models for Bilevel Planning

Despite recent, independent progress in model-based reinforcement learni...
research
09/30/2018

Few-Shot Goal Inference for Visuomotor Learning and Planning

Reinforcement learning and planning methods require an objective or rewa...
research
02/09/2023

Equivariant MuZero

Deep reinforcement learning repeatedly succeeds in closed, well-defined ...

Please sign up or login with your details

Forgot password? Click here to reset