Graph Convolutional Memory for Deep Reinforcement Learning

06/27/2021
by   Steven D. Morad, et al.
20

Solving partially-observable Markov decision processes (POMDPs) is critical when applying deep reinforcement learning (DRL) to real-world robotics problems, where agents have an incomplete view of the world. We present graph convolutional memory (GCM) for solving POMDPs using deep reinforcement learning. Unlike recurrent neural networks (RNNs) or transformers, GCM embeds domain-specific priors into the memory recall process via a knowledge graph. By encapsulating priors in the graph, GCM adapts to specific tasks but remains applicable to any DRL task. Using graph convolutions, GCM extracts hierarchical graph features, analogous to image features in a convolutional neural network (CNN). We show GCM outperforms long short-term memory (LSTM), gated transformers for reinforcement learning (GTrXL), and differentiable neural computers (DNCs) on control, long-term non-sequential recall, and 3D navigation tasks while using significantly fewer parameters.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

02/24/2021

Memory-based Deep Reinforcement Learning for POMDP

A promising characteristic of Deep Reinforcement Learning (DRL) is its c...
09/10/2015

Recurrent Reinforcement Learning: A Hybrid Approach

Successful applications of reinforcement learning in real-world problems...
12/28/2017

Multi-timescale memory dynamics in a reinforcement learning network with attention-gated memory

Learning and memory are intertwined in our brain and their relationship ...
12/17/2015

An Empirical Comparison of Neural Architectures for Reinforcement Learning in Partially Observable Environments

This paper explores the performance of fitted neural Q iteration for rei...
10/15/2018

Visual Semantic Navigation using Scene Priors

How do humans navigate to target objects in novel scenes? Do we use the ...
06/01/2018

Integrating Episodic Memory into a Reinforcement Learning Agent using Reservoir Sampling

Episodic memory is a psychology term which refers to the ability to reca...
11/09/2020

Application and Comparison of Deep Learning Methods in the Prediction of RNA Sequence Degradation and Stability

mRNA vaccines are receiving increased interest as potential alternatives...

Code Repositories

graph-conv-memory

Graph convolutional memory


view repo
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.