HERALD: Optimizing Heterogeneous DNN Accelerators for Edge Devices

by   Hyoukjun Kwon, et al.

Recent advances in deep neural networks (DNNs) have made DNNs the backbone of many applications on edge devices such as face recognition, object detection, and so on. To deal with massive computation requirements of DNN inferences within stringent energy and latency constraints, DNN accelerator (i.e., hardware specialized forDNN inferences), have emerged as a promising solution. Such advancement of hardware supporting DNNs has led to multiple DNN-based applications running at the same time on edge devices. They often run in parallel as background processes or as sub-tasks of a complex application. Thus, DNN workloads on a DNN accelerator now include a variety of layer operations and sizes from DNN models for diverse applications making them heterogeneous in layer granularity. Such heterogeneous workloads introduce a new major challenge for monolithic DNN accelerators because the efficiency of DNN accelerators relies on its dataflow, and different DNN layer types and shapes prefer different dataflows. In this work, we propose to tackle this challenge by designing heterogeneous DNN accelerators (HDAs) that deploy multiple DNN accelerators each optimized for different layer shapes and operations. To enable this approach, we propose HERALD, an optimization framework that explores the design space an HDA and layer schedules. Design time-optimized HDAs with the best energy-delay-product (EDP) HERALD identified provided 24.93 average across workloads and accelerators we evaluate compared to the best case of monolithic accelerators for each evaluation setting by deploying two complementary-style DNN accelerators.HERALD's scheduler employs heuristics that exploit the characteristics of DNN workloads, which provided 6.4 average compared to a baseline scheduler.


page 2

page 3

page 4

page 5

page 8


Enabling Flexibility for Sparse Tensor Acceleration via Heterogeneity

Recently, numerous sparse hardware accelerators for Deep Neural Networks...

Model-driven Cluster Resource Management for AI Workloads in Edge Clouds

Since emerging edge applications such as Internet of Things (IoT) analyt...

OmniBoost: Boosting Throughput of Heterogeneous Embedded Devices under Multi-DNN Workload

Modern Deep Neural Networks (DNNs) exhibit profound efficiency and accur...

Shared Memory-contention-aware Concurrent DNN Execution for Diversely Heterogeneous System-on-Chips

Two distinguishing features of state-of-the-art mobile and autonomous sy...

SALSA: Simulated Annealing based Loop-Ordering Scheduler for DNN Accelerators

To meet the growing need for computational power for DNNs, multiple spec...

On the Viability of using LLMs for SW/HW Co-Design: An Example in Designing CiM DNN Accelerators

Deep Neural Networks (DNNs) have demonstrated impressive performance acr...

CoSA: Scheduling by Constrained Optimization for Spatial Accelerators

Recent advances in Deep Neural Networks (DNNs) have led to active develo...

Please sign up or login with your details

Forgot password? Click here to reset