Semantic Tracklets: An Object-Centric Representation for Visual Multi-Agent Reinforcement Learning

08/06/2021
by   Iou-Jen Liu, et al.
0

Solving complex real-world tasks, e.g., autonomous fleet control, often involves a coordinated team of multiple agents which learn strategies from visual inputs via reinforcement learning. Many existing multi-agent reinforcement learning (MARL) algorithms however don't scale to environments where agents operate on visual inputs. To address this issue, algorithmically, recent works have focused on non-stationarity and exploration. In contrast, we study whether scalability can also be achieved via a disentangled representation. For this, we explicitly construct an object-centric intermediate representation to characterize the states of an environment, which we refer to as `semantic tracklets.' We evaluate `semantic tracklets' on the visual multi-agent particle environment (VMPE) and on the challenging visual multi-agent GFootball environment. `Semantic tracklets' consistently outperform baselines on VMPE, and achieve a +2.4 higher score difference than baselines on GFootball. Notably, this method is the first to successfully learn a strategy for five players in the GFootball environment using only visual data.

READ FULL TEXT

page 1

page 6

page 7

research
07/23/2021

Cooperative Exploration for Multi-Agent Deep Reinforcement Learning

Exploration is critical for good results in deep reinforcement learning ...
research
04/19/2021

Agent-Centric Representations for Multi-Agent Reinforcement Learning

Object-centric representations have recently enabled significant progres...
research
03/06/2023

MAESTRO: Open-Ended Environment Design for Multi-Agent Reinforcement Learning

Open-ended learning methods that automatically generate a curriculum of ...
research
02/10/2023

Scalability Bottlenecks in Multi-Agent Reinforcement Learning Systems

Multi-Agent Reinforcement Learning (MARL) is a promising area of researc...
research
07/03/2023

Environmental effects on emergent strategy in micro-scale multi-agent reinforcement learning

Multi-Agent Reinforcement Learning (MARL) is a promising candidate for r...
research
09/17/2019

Emergent Tool Use From Multi-Agent Autocurricula

Through multi-agent competition, the simple objective of hide-and-seek, ...
research
01/27/2023

SNeRL: Semantic-aware Neural Radiance Fields for Reinforcement Learning

As previous representations for reinforcement learning cannot effectivel...

Please sign up or login with your details

Forgot password? Click here to reset