Fail-Safe Human Detection for Drones Using a Multi-Modal Curriculum Learning Approach

09/28/2021
by   Ali Safa, et al.
0

Drones are currently being explored for safety-critical applications where human agents are expected to evolve in their vicinity. In such applications, robust people avoidance must be provided by fusing a number of sensing modalities in order to avoid collisions. Currently however, people detection systems used on drones are solely based on standard cameras besides an emerging number of works discussing the fusion of imaging and event-based cameras. On the other hand, radar-based systems provide up-most robustness towards environmental conditions but do not provide complete information on their own and have mainly been investigated in automotive contexts, not for drones. In order to enable the fusion of radars with both event-based and standard cameras, we present KUL-UAVSAFE, a first-of-its-kind dataset for the study of safety-critical people detection by drones. In addition, we propose a baseline CNN architecture with cross-fusion highways and introduce a curriculum learning strategy for multi-modal data termed SAUL, which greatly enhances the robustness of the system towards hard RGB failures and provides a significant gain of 15 proposed for cross-fusion networks. We demonstrate the real-time performance and feasibility of the approach by implementing the system in an edge-computing unit. We release our dataset and additional material in the project home page.

READ FULL TEXT

page 1

page 3

page 4

page 5

research
09/26/2022

DeepFusion: A Robust and Modular 3D Object Detector for Lidars, Cameras and Radars

We propose DeepFusion, a modular multi-modal architecture to fuse lidars...
research
08/20/2023

ThermRad: A Multi-modal Dataset for Robust 3D Object Detection under Challenging Conditions

Robust 3D object detection in extreme weather and illumination condition...
research
03/23/2022

On Adversarial Robustness of Large-scale Audio Visual Learning

As audio-visual systems are being deployed for safety-critical tasks suc...
research
07/17/2018

Robust Deep Multi-modal Learning Based on Gated Information Fusion Network

The goal of multi-modal learning is to use complimentary information on ...
research
07/07/2023

Robust Human Detection under Visual Degradation via Thermal and mmWave Radar Fusion

The majority of human detection methods rely on the sensor using visible...
research
02/18/2022

Multi-view and Multi-modal Event Detection Utilizing Transformer-based Multi-sensor fusion

We tackle a challenging task: multi-view and multi-modal event detection...
research
06/21/2017

Multi-Modal Trip Hazard Affordance Detection On Construction Sites

Trip hazards are a significant contributor to accidents on construction ...

Please sign up or login with your details

Forgot password? Click here to reset