Attention-Based Recurrence for Multi-Agent Reinforcement Learning under State Uncertainty

01/04/2023
by   Thomy Phan, et al.
0

State uncertainty poses a major challenge for decentralized coordination but is largely neglected in state-of-the-art research due to a strong focus on state-based centralized training for decentralized execution (CTDE) and benchmarks that lack sufficient stochasticity like StarCraft Multi-Agent Challenge (SMAC). In this paper, we propose Attention-based Embeddings of Recurrence In multi-Agent Learning (AERIAL) to approximate value functions under agent-wise state uncertainty. AERIAL replaces the true state with a learned representation of multi-agent recurrence, considering more accurate information about decentralized agent decisions than state-based CTDE. We then introduce MessySMAC, a modified version of SMAC with stochastic observations and higher variance in initial states, to provide a more general and configurable benchmark regarding state uncertainty. We evaluate AERIAL in Dec-Tiger as well as in a variety of SMAC and MessySMAC maps, and compare the results with state-based CTDE. Furthermore, we evaluate the robustness of AERIAL and state-based CTDE against various state uncertainty configurations in MessySMAC.

READ FULL TEXT
research
02/15/2022

Disentangling Successor Features for Coordination in Multi-agent Reinforcement Learning

Multi-agent reinforcement learning (MARL) is a promising framework for s...
research
09/17/2022

MA2QL: A Minimalist Approach to Fully Decentralized Multi-Agent Reinforcement Learning

Decentralized learning has shown great promise for cooperative multi-age...
research
10/19/2021

State-based Episodic Memory for Multi-Agent Reinforcement Learning

Multi-agent reinforcement learning (MARL) algorithms have made promising...
research
01/03/2022

A Deeper Understanding of State-Based Critics in Multi-Agent Reinforcement Learning

Centralized Training for Decentralized Execution, where training is done...
research
12/09/2021

Value Function Factorisation with Hypergraph Convolution for Cooperative Multi-agent Reinforcement Learning

Cooperation between agents in a multi-agent system (MAS) has become a ho...
research
02/14/2023

Graph Attention Multi-Agent Fleet Autonomy for Advanced Air Mobility

Autonomous mobility is emerging as a new mode of urban transportation fo...
research
07/12/2023

Diffusion Based Multi-Agent Adversarial Tracking

Target tracking plays a crucial role in real-world scenarios, particular...

Please sign up or login with your details

Forgot password? Click here to reset