nuPlan: A closed-loop ML-based planning benchmark for autonomous vehicles

by   Holger Caesar, et al.

In this work, we propose the world's first closed-loop ML-based planning benchmark for autonomous driving. While there is a growing body of ML-based motion planners, the lack of established datasets and metrics has limited the progress in this area. Existing benchmarks for autonomous vehicle motion prediction have focused on short-term motion forecasting, rather than long-term planning. This has led previous works to use open-loop evaluation with L2-based metrics, which are not suitable for fairly evaluating long-term planning. Our benchmark overcomes these limitations by introducing a large-scale driving dataset, lightweight closed-loop simulator, and motion-planning-specific metrics. We provide a high-quality dataset with 1500h of human driving data from 4 cities across the US and Asia with widely varying traffic patterns (Boston, Pittsburgh, Las Vegas and Singapore). We will provide a closed-loop simulation framework with reactive agents and provide a large set of both general and scenario-specific planning metrics. We plan to release the dataset at NeurIPS 2021 and organize benchmark challenges starting in early 2022.


page 1

page 2

page 3

page 4


Rethinking Closed-loop Training for Autonomous Driving

Recent advances in high-fidelity simulators have enabled closed-loop tra...

Parting with Misconceptions about Learning-based Vehicle Motion Planning

The release of nuPlan marks a new era in vehicle motion planning researc...

Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Dataset

As autonomous driving systems mature, motion forecasting has received in...

Injecting Planning-Awareness into Prediction and Detection Evaluation

Detecting other agents and forecasting their behavior is an integral par...

What-If Motion Prediction for Autonomous Driving

Forecasting the long-term future motion of road actors is a core challen...

CausalAgents: A Robustness Benchmark for Motion Forecasting using Causal Relationships

As machine learning models become increasingly prevalent in motion forec...

Closing the Planning-Learning Loop with Application to Autonomous Driving in a Crowd

Imagine an autonomous robot vehicle driving in dense, possibly unregulat...

Please sign up or login with your details

Forgot password? Click here to reset