PVT++: A Simple End-to-End Latency-Aware Visual Tracking Framework

11/21/2022
by   Bowen Li, et al.
8

Visual object tracking is an essential capability of intelligent robots. Most existing approaches have ignored the online latency that can cause severe performance degradation during real-world processing. Especially for unmanned aerial vehicle, where robust tracking is more challenging and onboard computation is limited, latency issue could be fatal. In this work, we present a simple framework for end-to-end latency-aware tracking, i.e., end-to-end predictive visual tracking (PVT++). PVT++ is capable of turning most leading-edge trackers into predictive trackers by appending an online predictor. Unlike existing solutions that use model-based approaches, our framework is learnable, such that it can take not only motion information as input but it can also take advantage of visual cues or a combination of both. Moreover, since PVT++ is end-to-end optimizable, it can further boost the latency-aware tracking performance by joint training. Additionally, this work presents an extended latency-aware evaluation benchmark for assessing an any-speed tracker in the online setting. Empirical results on robotic platform from aerial perspective show that PVT++ can achieve up to 60 on various trackers and exhibit better robustness than prior model-based solution, largely mitigating the degradation brought by latency. Code and models will be made public.

READ FULL TEXT

page 8

page 14

page 18

research
03/08/2021

Predictive Visual Tracking: A New Benchmark and Baseline Approach

As a crucial robotic perception capability, visual tracking has been int...
research
01/21/2021

All-Day Object Tracking for Unmanned Aerial Vehicle

Visual object tracking, which is representing a major interest in image ...
research
04/12/2021

Localization-Based Tracking

End-to-end production of object tracklets from high resolution video in ...
research
03/03/2022

Ad2Attack: Adaptive Adversarial Attack on Real-Time UAV Tracking

Visual tracking is adopted to extensive unmanned aerial vehicle (UAV)-re...
research
12/04/2019

Robust Visual Object Tracking with Natural Language Region Proposal Network

Tracking with natural-language (NL) specification is a powerful new para...
research
08/31/2023

D-VAT: End-to-End Visual Active Tracking for Micro Aerial Vehicles

Visual active tracking is a growing research topic in robotics due to it...

Please sign up or login with your details

Forgot password? Click here to reset