DriveGAN: Towards a Controllable High-Quality Neural Simulation

04/30/2021
by   Seung Wook Kim, et al.
56

Realistic simulators are critical for training and verifying robotics systems. While most of the contemporary simulators are hand-crafted, a scaleable way to build simulators is to use machine learning to learn how the environment behaves in response to an action, directly from data. In this work, we aim to learn to simulate a dynamic environment directly in pixel-space, by watching unannotated sequences of frames and their associated action pairs. We introduce a novel high-quality neural simulator referred to as DriveGAN that achieves controllability by disentangling different components without supervision. In addition to steering controls, it also includes controls for sampling features of a scene, such as the weather as well as the location of non-player objects. Since DriveGAN is a fully differentiable simulator, it further allows for re-simulation of a given video sequence, offering an agent to drive through a recorded scene again, possibly taking different actions. We train DriveGAN on multiple datasets, including 160 hours of real-world driving data. We showcase that our approach greatly surpasses the performance of previous data-driven simulators, and allows for new features not explored before.

READ FULL TEXT

page 1

page 3

page 5

page 6

page 7

page 8

page 16

research
10/26/2022

InterSim: Interactive Traffic Simulation via Explicit Relation Modeling

Interactive traffic simulation is crucial to autonomous driving systems ...
research
05/24/2023

Multi-Abstractive Neural Controller: An Efficient Hierarchical Control Architecture for Interactive Driving

As learning-based methods make their way from perception systems to plan...
research
08/03/2016

Learning a Driving Simulator

Comma.ai's approach to Artificial Intelligence for self-driving cars is ...
research
05/25/2020

Learning to Simulate Dynamic Environments with GameGAN

Simulation is a crucial component of any robotic system. In order to sim...
research
06/16/2020

LiDARsim: Realistic LiDAR Simulation by Leveraging the Real World

We tackle the problem of producing realistic simulations of LiDAR point ...
research
10/05/2018

Learning To Simulate

Simulation is a useful tool in situations where training data for machin...
research
05/30/2019

Does computer vision matter for action?

Computer vision produces representations of scene content. Much computer...

Please sign up or login with your details

Forgot password? Click here to reset