DeepAI AI Chat
Log In Sign Up

SimNet: Learning Reactive Self-driving Simulations from Real-world Observations

by   Luca Bergamini, et al.

In this work, we present a simple end-to-end trainable machine learning system capable of realistically simulating driving experiences. This can be used for the verification of self-driving system performance without relying on expensive and time-consuming road testing. In particular, we frame the simulation problem as a Markov Process, leveraging deep neural networks to model both state distribution and transition function. These are trainable directly from the existing raw observations without the need for any handcrafting in the form of plant or kinematic models. All that is needed is a dataset of historical traffic episodes. Our formulation allows the system to construct never seen scenes that unfold realistically reacting to the self-driving car's behaviour. We train our system directly from 1,000 hours of driving logs and measure both realism, reactivity of the simulation as the two key properties of the simulation. At the same time, we apply the method to evaluate the performance of a recently proposed state-of-the-art ML planning system trained from human driving logs. We discover this planning system is prone to previously unreported causal confusion issues that are difficult to test by non-reactive simulation. To the best of our knowledge, this is the first work that directly merges highly realistic data-driven simulations with a closed-loop evaluation for self-driving vehicles. We make the data, code, and pre-trained models publicly available to further stimulate simulation development.


page 1

page 2

page 3

page 4

page 5

page 6

page 7


Analyzing and Enhancing Closed-loop Stability in Reactive Simulation

Simulation has played an important role in efficiently evaluating self-d...

Autonomy 2.0: Why is self-driving always 5 years away?

Despite the numerous successes of machine learning over the past decade ...

Urban Driver: Learning to Drive from Real-world Demonstrations Using Policy Gradients

In this work we are the first to present an offline policy gradient meth...

Deep Structured Reactive Planning

An intelligent agent operating in the real-world must balance achieving ...

On the Choice of Data for Efficient Training and Validation of End-to-End Driving Models

The emergence of data-driven machine learning (ML) has facilitated signi...

Towards Safer Self-Driving Through Great PAIN (Physically Adversarial Intelligent Networks)

Automated vehicles' neural networks suffer from overfit, poor generaliza...

Interpretable Learning for Self-Driving Cars by Visualizing Causal Attention

Deep neural perception and control networks are likely to be a key compo...