The Multi-Agent Behavior Dataset: Mouse Dyadic Social Interactions

by   Jennifer J. Sun, et al.

Multi-agent behavior modeling aims to understand the interactions that occur between agents. We present a multi-agent dataset from behavioral neuroscience, the Caltech Mouse Social Interactions (CalMS21) Dataset. Our dataset consists of trajectory data of social interactions, recorded from videos of freely behaving mice in a standard resident-intruder assay. The CalMS21 dataset is part of the Multi-Agent Behavior Challenge 2021 and for our next step, our goal is to incorporate datasets from other domains studying multi-agent behavior. To help accelerate behavioral studies, the CalMS21 dataset provides a benchmark to evaluate the performance of automated behavior classification methods in three settings: (1) for training on large behavioral datasets all annotated by a single annotator, (2) for style transfer to learn inter-annotator differences in behavior definitions, and (3) for learning of new behaviors of interest given limited training data. The dataset consists of 6 million frames of unlabelled tracked poses of interacting mice, as well as over 1 million frames with tracked poses and corresponding frame-level behavior annotations. The challenge of our dataset is to be able to classify behaviors accurately using both labelled and unlabelled tracking data, as well as being able to generalize to new annotators and behaviors.


page 1

page 2

page 7

page 9

page 12

page 13

page 14

page 15


The MABe22 Benchmarks for Representation Learning of Multi-Agent Behavior

Real-world behavior is often shaped by complex interactions between mult...

Tracking all members of a honey bee colony over their lifetime

Computational approaches to the analysis of collective behavior in socia...

Robust Stochastic Bayesian Games for Behavior Space Coverage

A key challenge in multi-agent systems is the design of intelligent agen...

Interpreting Expert Annotation Differences in Animal Behavior

Hand-annotated data can vary due to factors such as subjective differenc...

Emergent Behaviors from Folksonomy Driven Interactions

To reflect the evolving knowledge on the Web this paper considers ontolo...

Massive Multi-Agent Data-Driven Simulations of the GitHub Ecosystem

Simulating and predicting planetary-scale techno-social systems poses he...

Unsupervised Speech Representation Learning for Behavior Modeling using Triplet Enhanced Contextualized Networks

Speech encodes a wealth of information related to human behavior and has...