Choosing Smartly: Adaptive Multimodal Fusion for Object Detection in Changing Environments

07/18/2017
by   Oier Mees, et al.
0

Object detection is an essential task for autonomous robots operating in dynamic and changing environments. A robot should be able to detect objects in the presence of sensor noise that can be induced by changing lighting conditions for cameras and false depth readings for range sensors, especially RGB-D cameras. To tackle these challenges, we propose a novel adaptive fusion approach for object detection that learns weighting the predictions of different sensor modalities in an online manner. Our approach is based on a mixture of convolutional neural network (CNN) experts and incorporates multiple modalities including appearance, depth and motion. We test our method in extensive robot experiments, in which we detect people in a combined indoor and outdoor scenario from RGB-D data, and we demonstrate that our method can adapt to harsh lighting changes and severe camera motion blur. Furthermore, we present a new RGB-D dataset for people detection in mixed in- and outdoor environments, recorded with a mobile robot.

READ FULL TEXT

page 1

page 5

page 6

research
11/07/2019

Model Adaption Object Detection System for Robot

How to detect the object and guide the robot to get close to the object ...
research
10/22/2021

CNN-based Omnidirectional Object Detection for HermesBot Autonomous Delivery Robot with Preliminary Frame Classification

Mobile autonomous robots include numerous sensors for environment percep...
research
01/18/2014

Modelling Observation Correlations for Active Exploration and Robust Object Detection

Today, mobile robots are expected to carry out increasingly complex task...
research
11/08/2022

Detection and depth estimation for domestic waste in outdoor environments by sensors fusion

In this work, we estimate the depth in which domestic waste are located ...
research
11/11/2020

FINO-Net: A Deep Multimodal Sensor Fusion Framework for Manipulation Failure Detection

Safe manipulation in unstructured environments for service robots is a c...
research
04/21/2022

Weakly Aligned Feature Fusion for Multimodal Object Detection

To achieve accurate and robust object detection in the real-world scenar...
research
04/07/2021

Multimodal Object Detection via Bayesian Fusion

Object detection with multimodal inputs can improve many safety-critical...

Please sign up or login with your details

Forgot password? Click here to reset