Log In Sign Up

baller2vec++: A Look-Ahead Multi-Entity Transformer For Modeling Coordinated Agents

by   Michael A. Alcorn, et al.

In many multi-agent spatiotemporal systems, the agents are under the influence of shared, unobserved variables (e.g., the play a team is executing in a game of basketball). As a result, the trajectories of the agents are often statistically dependent at any given time step; however, almost universally, multi-agent models implicitly assume the agents' trajectories are statistically independent at each time step. In this paper, we introduce baller2vec++, a multi-entity Transformer that can effectively model coordinated agents. Specifically, baller2vec++ applies a specially designed self-attention mask to a mixture of location and "look-ahead" trajectory sequences to learn the distributions of statistically dependent agent trajectories. We show that, unlike baller2vec (baller2vec++'s predecessor), baller2vec++ can learn to emulate the behavior of perfectly coordinated agents in a simulated toy dataset. Additionally, when modeling the trajectories of professional basketball players, baller2vec++ outperforms baller2vec by a wide margin.


baller2vec: A Multi-Entity Transformer For Multi-Agent Spatiotemporal Modeling

Multi-agent spatiotemporal modeling is a challenging task from both an a...

Agent-Temporal Attention for Reward Redistribution in Episodic Multi-Agent Reinforcement Learning

This paper considers multi-agent reinforcement learning (MARL) tasks whe...

Fine-Grained Retrieval of Sports Plays using Tree-Based Alignment of Trajectories

We propose a novel method for effective retrieval of multi-agent spatiot...

Multi-Agent Imitation Learning with Copulas

Multi-agent imitation learning aims to train multiple agents to perform ...

Latent Variable Nested Set Transformers AutoBots

Humans have the innate ability to attend to the most relevant actors in ...

Generative Multi-Agent Behavioral Cloning

We propose and study the problem of generative multi-agent behavioral cl...

Imagining The Road Ahead: Multi-Agent Trajectory Prediction via Differentiable Simulation

We develop a deep generative model built on a fully differentiable simul...

Code Repositories


A look-ahead multi-entity Transformer for modeling coordinated agents.

view repo