Learning End-to-end Multimodal Sensor Policies for Autonomous Navigation

05/30/2017
by   Guan-Horng Liu, et al.
0

Multisensory polices are known to enhance both state estimation and target tracking. However, in the space of end-to-end sensorimotor control, this multi-sensor outlook has received limited attention. Moreover, systematic ways to make policies robust to partial sensor failure are not well explored. In this work, we propose a specific customization of Dropout, called Sensor Dropout, to improve multisensory policy robustness and handle partial failure in the sensor-set. We also introduce an additional auxiliary loss on the policy network in order to reduce variance in the band of potential multi- and uni-sensory policies to reduce jerks during policy switching triggered by an abrupt sensor failure or deactivation/activation. Finally, through the visualization of gradients, we show that the learned policies are conditioned on the same latent states representation despite having diverse observations spaces - a hallmark of true sensor-fusion. Simulation results of the multisensory policy, as visualized in TORCS racing game, can be seen here: https://youtu.be/QAK2lcXjNZc.

READ FULL TEXT
research
07/09/2021

ARC: Adversarially Robust Control Policies for Autonomous Vehicles

Deep neural networks have demonstrated their capability to learn control...
research
05/19/2020

Multi-modal Sensor Fusion-Based Deep Neural Network for End-to-end Autonomous Driving with Scene Understanding

This study aims to improve the control performance and generalization ca...
research
09/26/2018

Learning Navigation Behaviors End to End

A longstanding goal of behavior-based robotics is to solve high-level na...
research
07/16/2019

A General Framework for Uncertainty Estimation in Deep Learning

End-to-end learning has recently emerged as a promising technique to tac...
research
02/04/2020

Learning Task-Driven Control Policies via Information Bottlenecks

This paper presents a reinforcement learning approach to synthesizing ta...
research
03/16/2022

Multiscale Sensor Fusion and Continuous Control with Neural CDEs

Though robot learning is often formulated in terms of discrete-time Mark...
research
07/25/2023

MAEA: Multimodal Attribution for Embodied AI

Understanding multimodal perception for embodied AI is an open question ...

Please sign up or login with your details

Forgot password? Click here to reset