Multi-Object Tracking with Deep Learning Ensemble for Unmanned Aerial System Applications

by   Wanlin Xie, et al.
Lockheed Martin Corp.

Multi-object tracking (MOT) is a crucial component of situational awareness in military defense applications. With the growing use of unmanned aerial systems (UASs), MOT methods for aerial surveillance is in high demand. Application of MOT in UAS presents specific challenges such as moving sensor, changing zoom levels, dynamic background, illumination changes, obscurations and small objects. In this work, we present a robust object tracking architecture aimed to accommodate for the noise in real-time situations. We propose a kinematic prediction model, called Deep Extended Kalman Filter (DeepEKF), in which a sequence-to-sequence architecture is used to predict entity trajectories in latent space. DeepEKF utilizes a learned image embedding along with an attention mechanism trained to weight the importance of areas in an image to predict future states. For the visual scoring, we experiment with different similarity measures to calculate distance based on entity appearances, including a convolutional neural network (CNN) encoder, pre-trained using Siamese networks. In initial evaluation experiments, we show that our method, combining scoring structure of the kinematic and visual models within a MHT framework, has improved performance especially in edge cases where entity motion is unpredictable, or the data presents frames with significant gaps.


page 6

page 8

page 10

page 11


All-Day Object Tracking for Unmanned Aerial Vehicle

Visual object tracking, which is representing a major interest in image ...

An Integrated Visual System for Unmanned Aerial Vehicles Tracking and Landing on the Ground Vehicles

The vision of unmanned aerial vehicles is very significant for UAV-relat...

Towards Robust Visual Tracking for Unmanned Aerial Vehicle with Tri-Attentional Correlation Filters

Object tracking has been broadly applied in unmanned aerial vehicle (UAV...

Siamese Object Tracking for Vision-Based UAM Approaching with Pairwise Scale-Channel Attention

Although the manipulating of the unmanned aerial manipulator (UAM) has b...

Generating Synthetic Training Data for Deep Learning-Based UAV Trajectory Prediction

Deep learning-based models, such as recurrent neural networks (RNNs), ha...

Tracking Evolving labels using Cone based Oracles

The evolving data framework was first proposed by Anagnostopoulos et al....

Please sign up or login with your details

Forgot password? Click here to reset