The MABe22 Benchmarks for Representation Learning of Multi-Agent Behavior

07/21/2022
by   Jennifer J. Sun, et al.
5

Real-world behavior is often shaped by complex interactions between multiple agents. To scalably study multi-agent behavior, advances in unsupervised and self-supervised learning have enabled a variety of different behavioral representations to be learned from trajectory data. To date, there does not exist a unified set of benchmarks that can enable comparing methods quantitatively and systematically across a broad set of behavior analysis settings. We aim to address this by introducing a large-scale, multi-agent trajectory dataset from real-world behavioral neuroscience experiments that covers a range of behavior analysis tasks. Our dataset consists of trajectory data from common model organisms, with 9.6 million frames of mouse data and 4.4 million frames of fly data, in a variety of experimental settings, such as different strains, lengths of interaction, and optogenetic stimulation. A subset of the frames also consist of expert-annotated behavior labels. Improvements on our dataset corresponds to behavioral representations that work across multiple organisms and is able to capture differences for common behavior analysis tasks.

READ FULL TEXT

page 4

page 40

04/06/2021

The Multi-Agent Behavior Dataset: Mouse Dyadic Social Interactions

Multi-agent behavior modeling aims to understand the interactions that o...
06/14/2022

Learning Behavior Representations Through Multi-Timescale Bootstrapping

Natural behavior consists of dynamics that are both unpredictable, can s...
04/01/2021

Unsupervised Speech Representation Learning for Behavior Modeling using Triplet Enhanced Contextualized Networks

Speech encodes a wealth of information related to human behavior and has...
06/11/2021

Interpreting Expert Annotation Differences in Animal Behavior

Hand-annotated data can vary due to factors such as subjective differenc...
09/10/2020

Learning Behavioral Representations of Human Mobility

In this paper, we investigate the suitability of state-of-the-art repres...
11/29/2021

Overcoming the Domain Gap in Contrastive Learning of Neural Action Representations

A fundamental goal in neuroscience is to understand the relationship bet...
02/23/2017

A Goal-Based Movement Model for Continuous Multi-Agent Tasks

Despite increasing attention paid to the need for fast, scalable methods...