Recovering and Simulating Pedestrians in the Wild

11/16/2020
by   Ze Yang, et al.
0

Sensor simulation is a key component for testing the performance of self-driving vehicles and for data augmentation to better train perception systems. Typical approaches rely on artists to create both 3D assets and their animations to generate a new scenario. This, however, does not scale. In contrast, we propose to recover the shape and motion of pedestrians from sensor readings captured in the wild by a self-driving car driving around. Towards this goal, we formulate the problem as energy minimization in a deep structured model that exploits human shape priors, reprojection consistency with 2D poses extracted from images, and a ray-caster that encourages the reconstructed mesh to agree with the LiDAR readings. Importantly, we do not require any ground-truth 3D scans or 3D pose annotations. We then incorporate the reconstructed pedestrian assets bank in a realistic LiDAR simulation system by performing motion retargeting, and show that the simulated LiDAR data can be used to significantly reduce the amount of annotated real-world data required for visual perception tasks.

READ FULL TEXT

page 2

page 5

page 7

page 8

08/13/2020

Testing the Safety of Self-driving Vehicles by Simulating Perception and Prediction

We present a novel method for testing the safety of self-driving vehicle...
11/17/2018

Simulating LIDAR Point Cloud for Autonomous Driving using Real-world Scenes and Traffic Flows

We present a LIDAR simulation framework that can automatically generate ...
01/16/2021

AdvSim: Generating Safety-Critical Scenarios for Self-Driving Vehicles

As self-driving systems become better, simulating scenarios where the au...
06/12/2019

Lidar based Detection and Classification of Pedestrians and Vehicles Using Machine Learning Methods

The goal of this paper is to classify objects mapped by LiDAR sensor int...
09/20/2021

Sequential Joint Shape and Pose Estimation of Vehicles with Application to Automatic Amodal Segmentation Labeling

Shape and pose estimation is a critical perception problem for a self-dr...
04/06/2021

Lidar-Monocular Surface Reconstruction Using Line Segments

Structure from Motion (SfM) often fails to estimate accurate poses in en...
01/16/2021

Diverse Complexity Measures for Dataset Curation in Self-driving

Modern self-driving autonomy systems heavily rely on deep learning. As a...