DeepAI AI Chat
Log In Sign Up

Recognition and 3D Localization of Pedestrian Actions from Monocular Video

by   Jun Hayakawa, et al.
Honda Research Institute USA

Understanding and predicting pedestrian behavior is an important and challenging area of research for realizing safe and effective navigation strategies in automated and advanced driver assistance technologies in urban scenes. This paper focuses on monocular pedestrian action recognition and 3D localization from an egocentric view for the purpose of predicting intention and forecasting future trajectory. A challenge in addressing this problem in urban traffic scenes is attributed to the unpredictable behavior of pedestrians, whereby actions and intentions are constantly in flux and depend on the pedestrians pose, their 3D spatial relations, and their interaction with other agents as well as with the environment. To partially address these challenges, we consider the importance of pose toward recognition and 3D localization of pedestrian actions. In particular, we propose an action recognition framework using a two-stream temporal relation network with inputs corresponding to the raw RGB image sequence of the tracked pedestrian as well as the pedestrian pose. The proposed method outperforms methods using a single-stream temporal relation network based on evaluations using the JAAD public dataset. The estimated pose and associated body key-points are also used as input to a network that estimates the 3D location of the pedestrian using a unique loss function. The evaluation of our 3D localization method on the KITTI dataset indicates the improvement of the average localization error as compared to existing state-of-the-art methods. Finally, we conduct qualitative tests of action recognition and 3D localization on HRI's H3D driving dataset.


page 1

page 2

page 3

page 6


Action and intention recognition of pedestrians in urban traffic

Action and intention recognition of pedestrians in urban settings are ch...

Pedestrian Environment Model for Automated Driving

Besides interacting correctly with other vehicles, automated vehicles sh...

Cross-Enhancement Transform Two-Stream 3D ConvNets for Pedestrian Action Recognition of Autonomous Vehicles

Action recognition is an important research topic in machine vision. It ...

TITAN: Future Forecast using Action Priors

We consider the problem of predicting the future trajectory of scene age...

MonoLoco: Monocular 3D Pedestrian Localization and Uncertainty Estimation

We tackle the fundamentally ill-posed problem of 3D human localization f...

Bypass Enhancement RGB Stream Model for Pedestrian Action Recognition of Autonomous Vehicles

Pedestrian action recognition and intention prediction are one of the co...

Bio-LSTM: A Biomechanically Inspired Recurrent Neural Network for 3D Pedestrian Pose and Gait Prediction

In applications such as autonomous driving, it is important to understan...