Latent Variable Nested Set Transformers AutoBots

02/19/2021
by   Roger Girgis, et al.
0

Humans have the innate ability to attend to the most relevant actors in their vicinity and can forecast how they may behave in the future. This ability will be crucial for the deployment of safety-critical agents such as robots or vehicles which interact with humans. We propose a theoretical framework for this problem setting based on autoregressively modelling sequences of nested sets, using latent variables to better capture multimodal distributions over future sets of sets. We present a new model architecture which we call a Nested Set Transformer which employs multi-head self-attention blocks over sets of sets that serve as a form of social attention between the elements of the sets at every timestep. Our approach can produce a distribution over future trajectories for all agents under consideration, or focus upon the trajectory of an ego-agent. We validate the Nested Set Transformer for autonomous driving settings which we refer to as ("AutoBot"), where we model the trajectory of an ego-agent based on the sequential observations of key attributes of multiple agents in a scene. AutoBot produces results better than state-of-the-art published prior work on the challenging nuScenes vehicle trajectory modeling benchmark. We also examine the multi-agent prediction version of our model and jointly forecast an ego-agent's future trajectory along with the other agents in the scene. We validate the behavior of our proposed Nested Set Transformer for scene level forecasting with a pedestrian trajectory dataset.

READ FULL TEXT
research
03/03/2022

LatentFormer: Multi-Agent Transformer-Based Interaction Modeling and Trajectory Prediction

Multi-agent trajectory prediction is a fundamental problem in autonomous...
research
03/31/2020

TITAN: Future Forecast using Action Priors

We consider the problem of predicting the future trajectory of scene age...
research
06/15/2023

Motion Perceiver: Real-Time Occupancy Forecasting for Embedded Systems

This work introduces a flexible architecture for real-time occupancy for...
research
10/07/2021

SVG-Net: An SVG-based Trajectory Prediction Model

Anticipating motions of vehicles in a scene is an essential problem for ...
research
11/26/2021

StarNet: Joint Action-Space Prediction with Star Graphs and Implicit Global Frame Self-Attention

In this work, we present a novel multi-modal multi-agent trajectory pred...
research
04/24/2021

baller2vec++: A Look-Ahead Multi-Entity Transformer For Modeling Coordinated Agents

In many multi-agent spatiotemporal systems, the agents are under the inf...
research
03/18/2020

Transformer Networks for Trajectory Forecasting

Most recent successes on forecasting the people motion are based on LSTM...

Please sign up or login with your details

Forgot password? Click here to reset