DeepAI
Log In Sign Up

VMAS: A Vectorized Multi-Agent Simulator for Collective Robot Learning

07/07/2022
by   Matteo Bettini, et al.
3

While many multi-robot coordination problems can be solved optimally by exact algorithms, solutions are often not scalable in the number of robots. Multi-Agent Reinforcement Learning (MARL) is gaining increasing attention in the robotics community as a promising solution to tackle such problems. Nevertheless, we still lack the tools that allow us to quickly and efficiently find solutions to large-scale collective learning tasks. In this work, we introduce the Vectorized Multi-Agent Simulator (VMAS). VMAS is an open-source framework designed for efficient MARL benchmarking. It is comprised of a vectorized 2D physics engine written in PyTorch and a set of twelve challenging multi-robot scenarios. Additional scenarios can be implemented through a simple and modular interface. We demonstrate how vectorization enables parallel simulation on accelerated hardware without added complexity. When comparing VMAS to OpenAI MPE, we show how MPE's execution time increases linearly in the number of simulations while VMAS is able to execute 30,000 parallel simulations in under 10s, proving more than 100x faster. Using VMAS's RLlib interface, we benchmark our multi-robot scenarios using various Proximal Policy Optimization (PPO)-based MARL algorithms. VMAS's scenarios prove challenging in orthogonal ways for state-of-the-art MARL algorithms. The VMAS framework is available at https://github.com/proroklab/VectorizedMultiAgentSimulator. A video of VMAS scenarios and experiments is available at https://youtu.be/aaDRYfiesAY.

READ FULL TEXT

page 1

page 2

page 3

page 4

02/06/2021

RIIT: Rethinking the Importance of Implementation Tricks in Multi-Agent Reinforcement Learning

In recent years, Multi-Agent Reinforcement Learning (MARL) has revolutio...
09/26/2021

MetaDrive: Composing Diverse Driving Scenarios for Generalizable Reinforcement Learning

Driving safely requires multiple capabilities from human and intelligent...
01/25/2021

ROS-NetSim: A Framework for the Integration of Robotic and Network Simulators

Multi-agent systems play an important role in modern robotics. Due to th...
10/26/2020

ChoiRbot: A ROS 2 Toolbox for Cooperative Robotics

In this paper, we introduce ChoiRbot, a toolbox for distributed cooperat...
11/25/2020

TLeague: A Framework for Competitive Self-Play based Distributed Multi-Agent Reinforcement Learning

Competitive Self-Play (CSP) based Multi-Agent Reinforcement Learning (MA...

Code Repositories

VectorizedMultiAgentSimulator

VMAS is a vectorized framework designed for efficient Multi-Agent Reinforcement Learning benchmarking. It is comprised of a vectorized 2D physics engine written in PyTorch and a set of challenging multi-robot scenarios. Additional scenarios can be implemented through a simple and modular interface.


view repo