Robots Autonomously Detecting People: A Multimodal Deep Contrastive Learning Method Robust to Intraclass Variations

03/01/2022
by   Angus Fung, et al.
0

Robotic detection of people in crowded and/or cluttered human-centered environments including hospitals, long-term care, stores and airports is challenging as people can become occluded by other people or objects, and deform due to variations in clothing or pose. There can also be loss of discriminative visual features due to poor lighting. In this paper, we present a novel multimodal person detection architecture to address the mobile robot problem of person detection under intraclass variations. We present a two-stage training approach using 1) a unique pretraining method we define as Temporal Invariant Multimodal Contrastive Learning (TimCLR), and 2) a Multimodal Faster R-CNN (MFRCNN) detector. TimCLR learns person representations that are invariant under intraclass variations through unsupervised learning. Our approach is unique in that it generates image pairs from natural variations within multimodal image sequences, in addition to synthetic data augmentation, and contrasts crossmodal features to transfer invariances between different modalities. These pretrained features are used by the MFRCNN detector for finetuning and person detection from RGB-D images. Extensive experiments validate the performance of our DL architecture in both human-centered crowded and cluttered environments. Results show that our method outperforms existing unimodal and multimodal person detection approaches in terms of detection accuracy in detecting people with body occlusions and pose deformations in different lighting conditions.

READ FULL TEXT

page 1

page 4

page 7

research
03/17/2022

Cascade Transformers for End-to-End Person Search

The goal of person search is to localize a target person from a gallery ...
research
04/06/2018

Deep Person Detection in 2D Range Data

Detecting humans is a key skill for mobile robots and intelligent vehicl...
research
01/03/2013

A Self-Organizing Neural Scheme for Door Detection in Different Environments

Doors are important landmarks for indoor mobile robot navigation and als...
research
07/14/2020

Towards Dense People Detection with Deep Learning and Depth images

This paper proposes a DNN-based system that detects multiple people from...
research
08/02/2017

Deep Detection of People and their Mobility Aids for a Hospital Robot

Robots operating in populated environments encounter many different type...
research
01/16/2013

Deep Learning for Detecting Robotic Grasps

We consider the problem of detecting robotic grasps in an RGB-D view of ...

Please sign up or login with your details

Forgot password? Click here to reset