Shared Cross-Modal Trajectory Prediction for Autonomous Driving

04/01/2020
by   Chiho Choi, et al.
0

We propose a framework for predicting future trajectories of traffic agents in highly interactive environments. On the basis of the fact that autonomous driving vehicles are equipped with various types of sensors (e.g., LiDAR scanner, RGB camera, etc.), our work aims to get benefit from the use of multiple input modalities that are complementary to each other. The proposed approach is composed of two stages. (i) feature encoding where we discover motion behavior of the target agent with respect to other directly and indirectly observable influences. We extract such behaviors from multiple perspectives such as in top-down and frontal view. (ii) cross-modal embedding where we embed a set of learned behavior representations into a single cross-modal latent space. We construct a generative model and formulate the objective functions with an additional regularizer specifically designed for future prediction. An extensive evaluation is conducted to show the efficacy of the proposed framework using two benchmark driving datasets.

READ FULL TEXT

page 14

page 15

page 16

page 17

research
09/18/2019

Deep Latent Space Learning for Cross-modal Mapping of Audio and Visual Signals

We propose a novel deep training algorithm for joint representation of a...
research
07/02/2023

Deep Cross-Modal Steganography Using Neural Representations

Steganography is the process of embedding secret data into another messa...
research
10/13/2022

X-Align: Cross-Modal Cross-View Alignment for Bird's-Eye-View Segmentation

Bird's-eye-view (BEV) grid is a common representation for the perception...
research
08/15/2022

Multi-modal Transformer Path Prediction for Autonomous Vehicle

Reasoning about vehicle path prediction is an essential and challenging ...
research
08/29/2022

ProspectNet: Weighted Conditional Attention for Future Interaction Modeling in Behavior Prediction

Behavior prediction plays an important role in integrated autonomous dri...
research
07/07/2019

A methodology for multisensory product experience design using cross-modal effect: A case of SLR camera

Throughout the course of product experience, a user employs multiple sen...
research
03/28/2022

Domain Knowledge Driven Pseudo Labels for Interpretable Goal-Conditioned Interactive Trajectory Prediction

Motion forecasting in highly interactive scenarios is a challenging prob...

Please sign up or login with your details

Forgot password? Click here to reset