Scalable Perception-Action-Communication Loops with Convolutional and Graph Neural Networks

06/24/2021
by   Ting-Kuei Hu, et al.
0

In this paper, we present a perception-action-communication loop design using Vision-based Graph Aggregation and Inference (VGAI). This multi-agent decentralized learning-to-control framework maps raw visual observations to agent actions, aided by local communication among neighboring agents. Our framework is implemented by a cascade of a convolutional and a graph neural network (CNN / GNN), addressing agent-level visual perception and feature learning, as well as swarm-level communication, local information aggregation and agent action inference, respectively. By jointly training the CNN and GNN, image features and communication messages are learned in conjunction to better address the specific task. We use imitation learning to train the VGAI controller in an offline phase, relying on a centralized expert controller. This results in a learned VGAI controller that can be deployed in a distributed manner for online execution. Additionally, the controller exhibits good scaling properties, with training in smaller teams and application in larger teams. Through a multi-agent flocking application, we demonstrate that VGAI yields performance comparable to or better than other decentralized controllers, using only the visual input modality and without accessing precise location or motion state information.

READ FULL TEXT

page 17

page 20

research
02/06/2020

VGAI: A Vision-Based Decentralized Controller Learning Framework for Robot Swarms

Despite the popularity of decentralized controller learning, very few su...
research
01/23/2023

Graph Neural Networks for Decentralized Multi-Agent Perimeter Defense

In this work, we study the problem of decentralized multi-agent perimete...
research
09/24/2022

Learning Decentralized Strategies for a Perimeter Defense Game with Graph Neural Networks

We consider the problem of finding decentralized strategies for multi-ag...
research
03/08/2021

Learning Connectivity for Data Distribution in Robot Teams

Many algorithms for control of multi-robot teams operate under the assum...
research
11/02/2020

Multi-Robot Coverage and Exploration using Spatial Graph Neural Networks

The multi-robot coverage problem is an essential building block for syst...
research
04/28/2021

Communication Topology Co-Design in Graph Recurrent Neural Network Based Distributed Control

When designing large-scale distributed controllers, the information-shar...
research
02/25/2023

Simulation of robot swarms for learning communication-aware coordination

Robotics research has been focusing on cooperative multi-agent problems,...

Please sign up or login with your details

Forgot password? Click here to reset