Two Body Problem: Collaborative Visual Task Completion

04/11/2019
by   Unnat Jain, et al.
12

Collaboration is a necessary skill to perform tasks that are beyond one agent's capabilities. Addressed extensively in both conventional and modern AI, multi-agent collaboration has often been studied in the context of simple grid worlds. We argue that there are inherently visual aspects to collaboration which should be studied in visually rich environments. A key element in collaboration is communication that can be either explicit, through messages, or implicit, through perception of the other agents and the visual world. Learning to collaborate in a visual environment entails learning (1) to perform the task, (2) when and what to communicate, and (3) how to act based on these communications and the perception of the visual world. In this paper we study the problem of learning to collaborate directly from pixels in AI2-THOR and demonstrate the benefits of explicit and implicit modes of communication to perform visual tasks. Refer to our project page for more details: https://prior.allenai.org/projects/two-body-problem

READ FULL TEXT

page 17

page 18

page 20

page 21

page 22

page 23

page 25

page 26

research
05/30/2020

When2com: Multi-Agent Perception via Communication Graph Grouping

While significant advances have been made for single-agent perception, m...
research
07/26/2023

Heterogeneous Embodied Multi-Agent Collaboration

Multi-agent embodied tasks have recently been studied in complex indoor ...
research
07/02/2021

Collaborative Visual Navigation

As a fundamental problem for Artificial Intelligence, multi-agent system...
research
10/02/2018

Learning-Based Physical Layer Communications for Multi-agent Collaboration

Consider a collaborative task carried out by two autonomous agents that ...
research
10/02/2018

Learning-Based Physical Layer Communications for Multiagent Collaboration

Consider a collaborative task carried out by two autonomous agents that ...
research
10/12/2021

Interpretation of Emergent Communication in Heterogeneous Collaborative Embodied Agents

Communication between embodied AI agents has received increasing attenti...
research
03/22/2023

UMC: A Unified Bandwidth-efficient and Multi-resolution based Collaborative Perception Framework

Multi-agent collaborative perception (MCP) has recently attracted much a...

Please sign up or login with your details

Forgot password? Click here to reset