Compositional Multi-Object Reinforcement Learning with Linear Relation Networks

01/31/2022
by   Davide Mambelli, et al.
21

Although reinforcement learning has seen remarkable progress over the last years, solving robust dexterous object-manipulation tasks in multi-object settings remains a challenge. In this paper, we focus on models that can learn manipulation tasks in fixed multi-object settings and extrapolate this skill zero-shot without any drop in performance when the number of objects changes. We consider the generic task of bringing a specific cube out of a set to a goal position. We find that previous approaches, which primarily leverage attention and graph neural network-based architectures, do not generalize their skills when the number of input objects changes while scaling as K^2. We propose an alternative plug-and-play module based on relational inductive biases to overcome these limitations. Besides exceeding performances in their training environment, we show that our approach, which scales linearly in K, allows agents to extrapolate and generalize zero-shot to any new object number.

READ FULL TEXT

page 2

page 17

research
06/22/2022

Curious Exploration via Structured World Models Yields Zero-Shot Object Manipulation

It has been a long-standing dream to design artificial agents that explo...
research
02/10/2023

Improving Zero-Shot Coordination Performance Based on Policy Similarity

Over these years, multi-agent reinforcement learning has achieved remark...
research
12/23/2019

Towards Practical Multi-Object Manipulation using Relational Reinforcement Learning

Learning robotic manipulation tasks using reinforcement learning with sp...
research
05/04/2023

Learning Generalizable Pivoting Skills

The skill of pivoting an object with a robotic system is challenging for...
research
06/01/2020

Encoding formulas as deep networks: Reinforcement learning for zero-shot execution of LTL formulas

We demonstrate a reinforcement learning agent which uses a compositional...
research
06/14/2017

Schema Networks: Zero-shot Transfer with a Generative Causal Model of Intuitive Physics

The recent adaptation of deep neural network-based methods to reinforcem...
research
03/26/2020

CAZSL: Zero-Shot Regression for Pushing Models by Generalizing Through Context

Learning accurate models of the physical world is required for a lot of ...

Please sign up or login with your details

Forgot password? Click here to reset