Grid-to-Graph: Flexible Spatial Relational Inductive Biases for Reinforcement Learning

02/08/2021
by   Zhengyao Jiang, et al.
0

Although reinforcement learning has been successfully applied in many domains in recent years, we still lack agents that can systematically generalize. While relational inductive biases that fit a task can improve generalization of RL agents, these biases are commonly hard-coded directly in the agent's neural architecture. In this work, we show that we can incorporate relational inductive biases, encoded in the form of relational graphs, into agents. Based on this insight, we propose Grid-to-Graph (GTG), a mapping from grid structures to relational graphs that carry useful spatial relational inductive biases when processed through a Relational Graph Convolution Network (R-GCN). We show that, with GTG, R-GCNs generalize better both in terms of in-distribution and out-of-distribution compared to baselines based on Convolutional Neural Networks and Neural Logic Machines on challenging procedurally generated environments and MinAtar. Furthermore, we show that GTG produces agents that can jointly reason over observations and environment dynamics encoded in knowledge bases.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

03/23/2020

Incorporating Relational Background Knowledge into Reinforcement Learning via Differentiable Inductive Logic Programming

Relational Reinforcement Learning (RRL) can offers various desirable fea...
09/09/2019

Learning Visual Dynamics Models of Rigid Objects using Relational Inductive Biases

Endowing robots with human-like physical reasoning abilities remains cha...
12/11/2019

Biases for Emergent Communication in Multi-agent Reinforcement Learning

We study the problem of emergent communication, in which language arises...
06/05/2018

Relational Deep Reinforcement Learning

We introduce an approach for deep reinforcement learning (RL) that impro...
07/12/2020

Relational-Grid-World: A Novel Relational Reasoning Environment and An Agent Model for Relational Information Extraction

Reinforcement learning (RL) agents are often designed specifically for a...
06/04/2018

Relational inductive biases, deep learning, and graph networks

Artificial intelligence (AI) has undergone a renaissance recently, makin...
09/15/2021

Target Languages (vs. Inductive Biases) for Learning to Act and Plan

Recent breakthroughs in AI have shown the remarkable power of deep learn...

Code Repositories

GTG

Source code of "Grid-to-Graph: Flexible Spatial Relational Inductive Biases for Reinforcement Learning" (AAMAS 2021).


view repo
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.