Efficient Deep Visual and Inertial Odometry with Adaptive Visual Modality Selection

05/12/2022
by   Mingyu Yang, et al.
19

In recent years, deep learning-based approaches for visual-inertial odometry (VIO) have shown remarkable performance outperforming traditional geometric methods. Yet, all existing methods use both the visual and inertial measurements for every pose estimation incurring potential computational redundancy. While visual data processing is much more expensive than that for the inertial measurement unit (IMU), it may not always contribute to improving the pose estimation accuracy. In this paper, we propose an adaptive deep-learning based VIO method that reduces computational redundancy by opportunistically disabling the visual modality. Specifically, we train a policy network that learns to deactivate the visual feature extractor on the fly based on the current motion state and IMU readings. A Gumbel-Softmax trick is adopted to train the policy network to make the decision process differentiable for end-to-end system training. The learned strategy is interpretable, and it shows scenario-dependent decision patterns for adaptive complexity reduction. Experiment results show that our method achieves a similar or even better performance than the full-modality baseline with up to 78.8 will be shared in https://github.com/mingyuyng/Visual-Selective-VIO

READ FULL TEXT
research
10/27/2022

Learned Inertial Odometry for Autonomous Drone Racing

Inertial odometry is an attractive solution to the problem of state esti...
research
09/18/2022

EMA-VIO: Deep Visual-Inertial Odometry with External Memory Attention

Accurate and robust localization is a fundamental need for mobile agents...
research
06/27/2019

DeepVIO: Self-supervised Deep Learning of Monocular Visual Inertial Odometry using 3D Geometric Constraints

This paper presents an self-supervised deep learning network for monocul...
research
01/18/2021

Deep Inertial Odometry with Accurate IMU Preintegration

Inertial Measurement Units (IMUs) are interceptive modalities that provi...
research
09/25/2021

Fully Differentiable and Interpretable Model for VIO with 4 Trainable Parameters

Monocular visual-inertial odometry (VIO) is a critical problem in roboti...
research
03/21/2023

Online Learning of Wheel Odometry Correction for Mobile Robots with Attention-based Neural Network

Modern robotic platforms need a reliable localization system to operate ...
research
10/01/2018

Inertial-aided Motion Deblurring with Deep Networks

We propose an inertial-aided deblurring method that incorporates gyrosco...

Please sign up or login with your details

Forgot password? Click here to reset