FusionMotion: Multi-Sensor Asynchronous Fusion for Continuous Occupancy Prediction via Neural-ODE

02/19/2023
by   Yining Shi, et al.
0

Occupancy maps are widely recognized as an efficient method for facilitating robot motion planning in static environments. However, for intelligent vehicles, occupancy of both the present and future moments is required to ensure safe driving. In the automotive industry, the accurate and continuous prediction of future occupancy maps in traffic scenarios remains a formidable challenge. This paper investigates multi-sensor spatio-temporal fusion strategies for continuous occupancy prediction in a systematic manner. This paper presents FusionMotion, a novel bird's eye view (BEV) occupancy predictor which is capable of achieving the fusion of asynchronous multi-sensor data and predicting the future occupancy map with variable time intervals and temporal horizons. Remarkably, FusionMotion features the adoption of neural ordinary differential equations on recurrent neural networks for occupancy prediction. FusionMotion learns derivatives of BEV features over temporal horizons, updates the implicit sensor's BEV feature measurements and propagates future states for each ODE step. Extensive experiments on large-scale nuScenes and Lyft L5 datasets demonstrate that FusionMotion significantly outperforms previous methods. In addition, it outperforms the BEVFusion-style fusion strategy on the Lyft L5 dataset while reducing synchronization requirements. Codes and models will be made available.

READ FULL TEXT

page 1

page 3

page 8

page 10

research
04/21/2021

FIERY: Future Instance Prediction in Bird's-Eye View from Surround Monocular Cameras

Driving requires interacting with road agents and predicting their futur...
research
08/31/2023

Irregular Traffic Time Series Forecasting Based on Asynchronous Spatio-Temporal Graph Convolutional Network

Accurate traffic forecasting at intersections governed by intelligent tr...
research
05/26/2022

Deep Sensor Fusion with Pyramid Fusion Networks for 3D Semantic Segmentation

Robust environment perception for autonomous vehicles is a tremendous ch...
research
08/13/2020

DSDNet: Deep Structured self-Driving Network

In this paper, we propose the Deep Structured self-Driving Network (DSDN...
research
10/02/2020

LiRaNet: End-to-End Trajectory Prediction using Spatio-Temporal Radar Fusion

In this paper, we present LiRaNet, a novel end-to-end trajectory predict...
research
11/01/2018

Convolutional Recurrent Predictor: Implicit Representation for Multi-target Filtering and Tracking

Defining a multi-target motion model, which is an important step of trac...
research
12/21/2018

Multi-Step Prediction of Occupancy Grid Maps with Recurrent Neural Networks

We investigate the multi-step prediction of the drivable space, represen...

Please sign up or login with your details

Forgot password? Click here to reset