Joint Metrics Matter: A Better Standard for Trajectory Forecasting

by   Erica Weng, et al.

Multi-modal trajectory forecasting methods commonly evaluate using single-agent metrics (marginal metrics), such as minimum Average Displacement Error (ADE) and Final Displacement Error (FDE), which fail to capture joint performance of multiple interacting agents. Only focusing on marginal metrics can lead to unnatural predictions, such as colliding trajectories or diverging trajectories for people who are clearly walking together as a group. Consequently, methods optimized for marginal metrics lead to overly-optimistic estimations of performance, which is detrimental to progress in trajectory forecasting research. In response to the limitations of marginal metrics, we present the first comprehensive evaluation of state-of-the-art (SOTA) trajectory forecasting methods with respect to multi-agent metrics (joint metrics): JADE, JFDE, and collision rate. We demonstrate the importance of joint metrics as opposed to marginal metrics with quantitative evidence and qualitative examples drawn from the ETH / UCY and Stanford Drone datasets. We introduce a new loss function incorporating joint metrics that, when applied to a SOTA trajectory forecasting method, achieves a 7 on the ETH / UCY datasets with respect to the previous SOTA. Our results also indicate that optimizing for joint metrics naturally leads to an improvement in interaction modeling, as evidenced by a 16 the ETH / UCY datasets with respect to the previous SOTA.


page 1

page 6

page 8

page 13

page 16


QCNeXt: A Next-Generation Framework For Joint Multi-Agent Trajectory Prediction

Estimating the joint distribution of on-road agents' future trajectories...

Rethinking Trajectory Forecasting Evaluation

Forecasting the behavior of other agents is an integral part of the mode...

Collaborative Uncertainty Benefits Multi-Agent Multi-Modal Trajectory Forecasting

In multi-modal multi-agent trajectory forecasting, two major challenges ...

Joint 3D Tracking and Forecasting with Graph Neural Network and Diversity Sampling

3D multi-object tracking (MOT) and trajectory forecasting are two critic...

Diverse and Admissible Trajectory Forecasting through Multimodal Context Understanding

Multi-agent trajectory forecasting in autonomous driving requires an age...

StarNet: Joint Action-Space Prediction with Star Graphs and Implicit Global Frame Self-Attention

In this work, we present a novel multi-modal multi-agent trajectory pred...

Back to square one: probabilistic trajectory forecasting without bells and whistles

We introduce a spatio-temporal convolutional neural network model for tr...

Please sign up or login with your details

Forgot password? Click here to reset