DeepAI AI Chat
Log In Sign Up

PSI: A Pedestrian Behavior Dataset for Socially Intelligent Autonomous Car

by   Tina Chen, et al.

Prediction of pedestrian behavior is critical for fully autonomous vehicles to drive in busy city streets safely and efficiently. The future autonomous cars need to fit into mixed conditions with not only technical but also social capabilities. As more algorithms and datasets have been developed to predict pedestrian behaviors, these efforts lack the benchmark labels and the capability to estimate the temporal-dynamic intent changes of the pedestrians, provide explanations of the interaction scenes, and support algorithms with social intelligence. This paper proposes and shares another benchmark dataset called the IUPUI-CSRC Pedestrian Situated Intent (PSI) data with two innovative labels besides comprehensive computer vision labels. The first novel label is the dynamic intent changes for the pedestrians to cross in front of the ego-vehicle, achieved from 24 drivers with diverse backgrounds. The second one is the text-based explanations of the driver reasoning process when estimating pedestrian intents and predicting their behaviors during the interaction period. These innovative labels can enable several computer vision tasks, including pedestrian intent/behavior prediction, vehicle-pedestrian interaction segmentation, and video-to-language mapping for explainable algorithms. The released dataset can fundamentally improve the development of pedestrian behavior prediction models and develop socially intelligent autonomous cars to interact with pedestrians efficiently. The dataset has been evaluated with different tasks and is released to the public to access.


page 1

page 4

page 5

page 8

page 9


Intent Communication between Autonomous Vehicles and Pedestrians

When pedestrians encounter vehicles, they typically stop and wait for a ...

nuReality: A VR environment for research of pedestrian and autonomous vehicle interactions

We present nuReality, a virtual reality 'VR' environment designed to tes...

A Hybrid Control Design for Autonomous Vehicles at Uncontrolled Intersections

As autonomous vehicles (AVs) inch closer to reality, a central requireme...

The Adaptability and Challenges of Autonomous Vehicles to Pedestrians in Urban China

China is the world's largest automotive market and is ambitious for auto...

Learning the Pedestrian-Vehicle Interaction for Pedestrian Trajectory Prediction

In this paper, we study the interaction between pedestrians and vehicles...

A new approach for pedestrian density estimation using moving sensors and computer vision

An understanding of pedestrians dynamics is indispensable for numerous u...

Learning a Pedestrian Social Behavior Dictionary

Understanding pedestrian behavior patterns is a key component to buildin...