Data-Model-Circuit Tri-Design for Ultra-Light Video Intelligence on Edge Devices

10/16/2022
by   Yimeng Zhang, et al.
2

In this paper, we propose a data-model-hardware tri-design framework for high-throughput, low-cost, and high-accuracy multi-object tracking (MOT) on High-Definition (HD) video stream. First, to enable ultra-light video intelligence, we propose temporal frame-filtering and spatial saliency-focusing approaches to reduce the complexity of massive video data. Second, we exploit structure-aware weight sparsity to design a hardware-friendly model compression method. Third, assisted with data and model complexity reduction, we propose a sparsity-aware, scalable, and low-power accelerator design, aiming to deliver real-time performance with high energy efficiency. Different from existing works, we make a solid step towards the synergized software/hardware co-optimization for realistic MOT model implementation. Compared to the state-of-the-art MOT baseline, our tri-design approach can achieve 12.5x latency reduction, 20.9x effective frame rate improvement, 5.83x lower power, and 9.78x better energy efficiency, without much accuracy drop.

READ FULL TEXT
research
12/05/2022

Algorithm and Hardware Co-Design of Energy-Efficient LSTM Networks for Video Recognition with Hierarchical Tucker Tensor Decomposition

Long short-term memory (LSTM) is a type of powerful deep neural network ...
research
07/12/2023

Flexible and Fully Quantized Ultra-Lightweight TinyissimoYOLO for Ultra-Low-Power Edge Systems

This paper deploys and explores variants of TinyissimoYOLO, a highly fle...
research
05/11/2021

3U-EdgeAI: Ultra-Low Memory Training, Ultra-Low BitwidthQuantization, and Ultra-Low Latency Acceleration

The deep neural network (DNN) based AI applications on the edge require ...
research
12/29/2020

Sensifi: A Wireless Sensing System for Ultra-High-Rate Applications

Wireless Sensor Networks (WSNs) are being used in various applications s...
research
06/17/2023

CStream: Parallel Data Stream Compression on Multicore Edge Devices

In the burgeoning realm of Internet of Things (IoT) applications on edge...
research
05/04/2021

A Power and Area Efficient Lepton Hardware Encoder with Hash-based Memory Optimization

Although it has been surpassed by many subsequent coding standards, JPEG...
research
08/04/2021

Spartus: A 9.4 TOp/s FPGA-based LSTM Accelerator Exploiting Spatio-temporal Sparsity

Long Short-Term Memory (LSTM) recurrent networks are frequently used for...

Please sign up or login with your details

Forgot password? Click here to reset