Three Steps to Multimodal Trajectory Prediction: Modality Clustering, Classification and Synthesis

03/14/2021
by   Jianhua Sun, et al.
0

Multimodal prediction results are essential for trajectory forecasting task as there is no single correct answer for the future. Previous frameworks can be divided into three categories: regression, generation and classification frameworks. However, these frameworks have weaknesses in different aspects so that they cannot model the multimodal prediction task comprehensively. In this paper, we present a novel insight along with a brand-new prediction framework by formulating multimodal prediction into three steps: modality clustering, classification and synthesis, and address the shortcomings of earlier frameworks. Exhaustive experiments on popular benchmarks have demonstrated that our proposed method surpasses state-of-the-art works even without introducing social and map information. Specifically, we achieve 19.2 improvement on ADE and FDE respectively on ETH/UCY dataset. Our code will be made publicly available.

READ FULL TEXT

page 2

page 4

page 7

page 8

research
08/17/2022

Leukocyte Classification using Multimodal Architecture Enhanced by Knowledge Distillation

Recently, a lot of automated white blood cells (WBC) or leukocyte classi...
research
08/09/2021

Disentangling Hate in Online Memes

Hateful and offensive content detection has been extensively explored in...
research
05/10/2023

MMoT: Mixture-of-Modality-Tokens Transformer for Composed Multimodal Conditional Image Synthesis

Existing multimodal conditional image synthesis (MCIS) methods generate ...
research
05/17/2023

Rethinking Multimodal Content Moderation from an Asymmetric Angle with Mixed-modality

There is a rapidly growing need for multimodal content moderation (CM) a...
research
03/03/2022

Graph Neural Networks for Multimodal Single-Cell Data Integration

Recent advances in multimodal single-cell technologies have enabled simu...
research
09/08/2021

TrAISformer-A generative transformer for AIS trajectory prediction

Modelling trajectory in general, and vessel trajectory in particular, is...
research
11/11/2021

Trustworthy Multimodal Regression with Mixture of Normal-inverse Gamma Distributions

Multimodal regression is a fundamental task, which integrates the inform...

Please sign up or login with your details

Forgot password? Click here to reset