Log In Sign Up

Fusion of Multispectral Data Through Illumination-aware Deep Neural Networks for Pedestrian Detection

by   Dayan Guan, et al.

Multispectral pedestrian detection has received extensive attention in recent years as a promising solution to facilitate robust human target detection for around-the-clock applications (e.g. security surveillance and autonomous driving). In this paper, we demonstrate illumination information encoded in multispectral images can be utilized to significantly boost performance of pedestrian detection. A novel illumination-aware weighting mechanism is present to accurately depict illumination condition of a scene. Such illumination information is incorporated into two-stream deep convolutional neural networks to learn multispectral human-related features under different illumination conditions (daytime and nighttime). Moreover, we utilized illumination information together with multispectral data to generate more accurate semantic segmentation which are used to boost pedestrian detection accuracy. Putting all of the pieces together, we present a powerful framework for multispectral pedestrian detection based on multi-task learning of illumination-aware pedestrian detection and semantic segmentation. Our proposed method is trained end-to-end using a well-designed multi-task loss function and outperforms state-of-the-art approaches on KAIST multispectral pedestrian dataset.


page 1

page 2

page 3

page 8

page 11

page 12


SSA-CNN: Semantic Self-Attention CNN for Pedestrian Detection

Pedestrian detection plays an important role in many applications such a...

Multispectral Pedestrian Detection via Simultaneous Detection and Segmentation

Multispectral pedestrian detection has attracted increasing attention fr...

Booster-SHOT: Boosting Stacked Homography Transformations for Multiview Pedestrian Detection with Attention

Improving multi-view aggregation is integral for multi-view pedestrian d...

Pedestrian Detection aided by Deep Learning Semantic Tasks

Deep learning methods have achieved great success in pedestrian detectio...

Illuminating Pedestrians via Simultaneous Detection & Segmentation

Pedestrian detection is a critical problem in computer vision with signi...

End-to-End Pedestrian Collision Warning System based on a Convolutional Neural Network with Semantic Segmentation

Traditional pedestrian collision warning systems sometimes raise alarms ...

Illumination and Temperature-Aware Multispectral Networks for Edge-Computing-Enabled Pedestrian Detection

Accurate and efficient pedestrian detection is crucial for the intellige...