Towards real-time and energy efficient Siamese tracking – a hardware-software approach

by   Dominika Przewlocka-Rus, et al.

Siamese trackers have been among the state-of-the-art solutions in each Visual Object Tracking (VOT) challenge over the past few years. However, with great accuracy comes great computational complexity: to achieve real-time processing, these trackers have to be massively parallelised and are usually run on high-end GPUs. Easy to implement, this approach is energy consuming, and thus cannot be used in many low-power applications. To overcome this, one can use energy-efficient embedded devices, such as heterogeneous platforms joining the ARM processor system with programmable logic (FPGA). In this work, we propose a hardware-software implementation of the well-known fully connected Siamese tracker (SiamFC). We have developed a quantised Siamese network for the FINN accelerator, using algorithm-accelerator co-design, and performed design space exploration to achieve the best efficiency-to-energy ratio (determined by FPS and used resources). For our network, running in the programmable logic part of the Zynq UltraScale+ MPSoC ZCU104, we achieved the processing of almost 50 frames-per-second with tracker accuracy on par with its floating point counterpart, as well as the original SiamFC network. The complete tracking system, implemented in ARM with the network accelerated on FPGA, achieves up to 17 fps. These results bring us towards bridging the gap between the highly accurate but energy-demanding algorithms and energy-efficient solutions ready to be used in low-power, edge systems.


eSLAM: An Energy-Efficient Accelerator for Real-Time ORB-SLAM on FPGA Platform

Simultaneous Localization and Mapping (SLAM) is a critical task for auto...

Flexible and Fully Quantized Ultra-Lightweight TinyissimoYOLO for Ultra-Low-Power Edge Systems

This paper deploys and explores variants of TinyissimoYOLO, a highly fle...

Optimisation of a Siamese Neural Network for Real-Time Energy Efficient Object Tracking

In this paper the research on optimisation of visual object tracking usi...

MATCHA: A Fast and Energy-Efficient Accelerator for Fully Homomorphic Encryption over the Torus

Fully Homomorphic Encryption over the Torus (TFHE) allows arbitrary comp...

Adaptive Subsampling for ROI-based Visual Tracking: Algorithms and FPGA Implementation

There is tremendous scope for improving the energy efficiency of embedde...

A Real-Time and Energy-Efficient Implementation of Difference-of-Gaussian with Flexible Thin-Film Transistors

With many advantageous features, softness and better biocompatibility, f...

Substream-Centric Maximum Matchings on FPGA

Developing high-performance and energy-efficient algorithms for maximum ...

Please sign up or login with your details

Forgot password? Click here to reset