DeepAI
Log In Sign Up

Spatial Intention Maps for Multi-Agent Mobile Manipulation

03/23/2021
by   Jimmy Wu, et al.
5

The ability to communicate intention enables decentralized multi-agent robots to collaborate while performing physical tasks. In this work, we present spatial intention maps, a new intention representation for multi-agent vision-based deep reinforcement learning that improves coordination between decentralized mobile manipulators. In this representation, each agent's intention is provided to other agents, and rendered into an overhead 2D map aligned with visual observations. This synergizes with the recently proposed spatial action maps framework, in which state and action representations are spatially aligned, providing inductive biases that encourage emergent cooperative behaviors requiring spatial coordination, such as passing objects to each other or avoiding collisions. Experiments across a variety of multi-agent environments, including heterogeneous robot teams with different abilities (lifting, pushing, or throwing), show that incorporating spatial intention maps improves performance for different mobile manipulation tasks while significantly enhancing cooperative behaviors.

READ FULL TEXT

page 1

page 2

page 3

page 4

page 5

page 9

08/26/2022

CH-MARL: A Multimodal Benchmark for Cooperative, Heterogeneous Multi-Agent Reinforcement Learning

We propose a multimodal (vision-and-language) benchmark for cooperative ...
04/15/2021

Joint Attention for Multi-Agent Coordination and Social Learning

Joint attention - the ability to purposefully coordinate attention with ...
01/18/2021

HAMMER: Multi-Level Coordination of Reinforcement Learning Agents via Learned Messaging

Cooperative multi-agent reinforcement learning (MARL) has achieved signi...
12/05/2021

Intention Recognition for Multiple Agents

Intention recognition is an important step to facilitate collaboration i...
04/20/2020

Spatial Action Maps for Mobile Manipulation

This paper proposes a new action representation for learning to perform ...
07/11/2012

Genetic agent approach for improving on-the-fly web map generalization

The utilization of web mapping becomes increasingly important in the dom...

Code Repositories

spatial-intention-maps

Learning multi-agent robotic mobile manipulation with deep reinforcement learning


view repo