Learning Behavior Representations Through Multi-Timescale Bootstrapping

06/14/2022
by   Mehdi Azabou, et al.
0

Natural behavior consists of dynamics that are both unpredictable, can switch suddenly, and unfold over many different timescales. While some success has been found in building representations of behavior under constrained or simplified task-based conditions, many of these models cannot be applied to free and naturalistic settings due to the fact that they assume a single scale of temporal dynamics. In this work, we introduce Bootstrap Across Multiple Scales (BAMS), a multi-scale representation learning model for behavior: we combine a pooling module that aggregates features extracted over encoders with different temporal receptive fields, and design a set of latent objectives to bootstrap the representations in each respective space to encourage disentanglement across different timescales. We first apply our method on a dataset of quadrupeds navigating in different terrain types, and show that our model captures the temporal complexity of behavior. We then apply our method to the MABe 2022 Multi-agent behavior challenge, where our model ranks 3rd overall and 1st on two subtasks, and show the importance of incorporating multi-timescales when analyzing behavior.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
03/15/2023

Relax, it doesn't matter how you get there: A new self-supervised approach for multi-timescale behavior analysis

Natural behavior consists of dynamics that are complex and unpredictable...
research
07/21/2022

The MABe22 Benchmarks for Representation Learning of Multi-Agent Behavior

Real-world behavior is often shaped by complex interactions between mult...
research
08/02/2021

Dynamic Multi-scale Convolution for Dialect Identification

Time Delay Neural Networks (TDNN)-based methods are widely used in diale...
research
02/28/2019

Bi-Directional Cascade Network for Perceptual Edge Detection

Exploiting multi-scale representations is critical to improve edge detec...
research
02/21/2023

Learning signatures of decision making from many individuals playing the same game

Human behavior is incredibly complex and the factors that drive decision...
research
02/02/2019

Belief dynamics extraction

Animal behavior is not driven simply by its current observations, but is...

Please sign up or login with your details

Forgot password? Click here to reset