Multimodal semantic forecasting based on conditional generation of future features

10/18/2020
by   Kristijan Fugošić, et al.
0

This paper considers semantic forecasting in road-driving scenes. Most existing approaches address this problem as deterministic regression of future features or future predictions given observed frames. However, such approaches ignore the fact that future can not always be guessed with certainty. For example, when a car is about to turn around a corner, the road which is currently occluded by buildings may turn out to be either free to drive, or occupied by people, other vehicles or roadworks. When a deterministic model confronts such situation, its best guess is to forecast the most likely outcome. However, this is not acceptable since it defeats the purpose of forecasting to improve security. It also throws away valuable training data, since a deterministic model is unable to learn any deviation from the norm. We address this problem by providing more freedom to the model through allowing it to forecast different futures. We propose to formulate multimodal forecasting as sampling of a multimodal generative model conditioned on the observed frames. Experiments on the Cityscapes dataset reveal that our multimodal model outperforms its deterministic counterpart in short-term forecasting while performing slightly worse in the mid-term case.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 7

page 8

page 11

page 17

page 18

page 19

page 20

05/03/2019

PRECOG: PREdiction Conditioned On Goals in Visual Multi-Agent Settings

For autonomous vehicles (AVs) to behave appropriately on roads populated...
04/08/2021

Panoptic Segmentation Forecasting

Our goal is to forecast the near future given a set of recent observatio...
12/11/2019

Towards Better Forecasting by Fusing Near and Distant Future Visions

Multivariate time series forecasting is an important yet challenging pro...
06/09/2020

Physically constrained short-term vehicle trajectory forecasting with naive semantic maps

Urban environments manifest a high level of complexity, and therefore it...
09/06/2016

Comparison of several short-term traffic speed forecasting models

The widespread adoption of smartphones in recent years has made it possi...
05/07/2021

A Multivariate Density Forecast Approach for Online Power System Security Assessment

A multivariate density forecast model based on deep learning is designed...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.