DeepAI AI Chat
Log In Sign Up

A Flexible-Frame-Rate Vision-Aided Inertial Object Tracking System for Mobile Devices

by   Yo-Chung Lau, et al.
National Taiwan University

Real-time object pose estimation and tracking is challenging but essential for emerging augmented reality (AR) applications. In general, state-of-the-art methods address this problem using deep neural networks which indeed yield satisfactory results. Nevertheless, the high computational cost of these methods makes them unsuitable for mobile devices where real-world applications usually take place. In addition, head-mounted displays such as AR glasses require at least 90 FPS to avoid motion sickness, which further complicates the problem. We propose a flexible-frame-rate object pose estimation and tracking system for mobile devices. It is a monocular visual-inertial-based system with a client-server architecture. Inertial measurement unit (IMU) pose propagation is performed on the client side for high speed tracking, and RGB image-based 3D pose estimation is performed on the server side to obtain accurate poses, after which the pose is sent to the client side for visual-inertial fusion, where we propose a bias self-correction mechanism to reduce drift. We also propose a pose inspection algorithm to detect tracking failures and incorrect pose estimation. Connected by high-speed networking, our system supports flexible frame rates up to 120 FPS and guarantees high precision and real-time tracking on low-end devices. Both simulations and real world experiments show that our method achieves accurate and robust object tracking.


page 1

page 5


Estimating Metric Poses of Dynamic Objects Using Monocular Visual-Inertial Fusion

A monocular 3D object tracking system generally has only up-to-scale pos...

Instant 3D Object Tracking with Applications in Augmented Reality

Tracking object poses in 3D is a crucial building block for Augmented Re...

VIPose: Real-time Visual-Inertial 6D Object Pose Tracking

Estimating the 6D pose of objects is beneficial for robotics tasks such ...

Monocular visual-inertial SLAM algorithm combined with wheel speed anomaly detection

To address the weak observability of monocular visual-inertial odometers...

BundleFusion: Real-time Globally Consistent 3D Reconstruction using On-the-fly Surface Re-integration

Real-time, high-quality, 3D scanning of large-scale scenes is key to mix...

Distributed Client-Server Optimization for SLAM with Limited On-Device Resources

Simultaneous localization and mapping (SLAM) is a crucial functionality ...

Sensor Fusion of Camera, GPS and IMU using Fuzzy Adaptive Multiple Motion Models

A tracking system that will be used for Augmented Reality (AR) applicati...