DeepAI AI Chat
Log In Sign Up

Choosing Smartly: Adaptive Multimodal Fusion for Object Detection in Changing Environments

by   Oier Mees, et al.

Object detection is an essential task for autonomous robots operating in dynamic and changing environments. A robot should be able to detect objects in the presence of sensor noise that can be induced by changing lighting conditions for cameras and false depth readings for range sensors, especially RGB-D cameras. To tackle these challenges, we propose a novel adaptive fusion approach for object detection that learns weighting the predictions of different sensor modalities in an online manner. Our approach is based on a mixture of convolutional neural network (CNN) experts and incorporates multiple modalities including appearance, depth and motion. We test our method in extensive robot experiments, in which we detect people in a combined indoor and outdoor scenario from RGB-D data, and we demonstrate that our method can adapt to harsh lighting changes and severe camera motion blur. Furthermore, we present a new RGB-D dataset for people detection in mixed in- and outdoor environments, recorded with a mobile robot.


page 1

page 5

page 6


Model Adaption Object Detection System for Robot

How to detect the object and guide the robot to get close to the object ...

Modelling Observation Correlations for Active Exploration and Robust Object Detection

Today, mobile robots are expected to carry out increasingly complex task...

Detection and depth estimation for domestic waste in outdoor environments by sensors fusion

In this work, we estimate the depth in which domestic waste are located ...

FINO-Net: A Deep Multimodal Sensor Fusion Framework for Manipulation Failure Detection

Safe manipulation in unstructured environments for service robots is a c...

Weakly Aligned Feature Fusion for Multimodal Object Detection

To achieve accurate and robust object detection in the real-world scenar...

Multimodal Object Detection via Bayesian Fusion

Object detection with multimodal inputs can improve many safety-critical...