CoSA: Scheduling by Constrained Optimization for Spatial Accelerators

05/05/2021
by   Qijing Huang, et al.
3

Recent advances in Deep Neural Networks (DNNs) have led to active development of specialized DNN accelerators, many of which feature a large number of processing elements laid out spatially, together with a multi-level memory hierarchy and flexible interconnect. While DNN accelerators can take advantage of data reuse and achieve high peak throughput, they also expose a large number of runtime parameters to the programmers who need to explicitly manage how computation is scheduled both spatially and temporally. In fact, different scheduling choices can lead to wide variations in performance and efficiency, motivating the need for a fast and efficient search strategy to navigate the vast scheduling space. To address this challenge, we present CoSA, a constrained-optimization-based approach for scheduling DNN accelerators. As opposed to existing approaches that either rely on designers' heuristics or iterative methods to navigate the search space, CoSA expresses scheduling decisions as a constrained-optimization problem that can be deterministically solved using mathematical optimization techniques. Specifically, CoSA leverages the regularities in DNN operators and hardware to formulate the DNN scheduling space into a mixed-integer programming (MIP) problem with algorithmic and architectural constraints, which can be solved to automatically generate a highly efficient schedule in one shot. We demonstrate that CoSA-generated schedules significantly outperform state-of-the-art approaches by a geometric mean of up to 2.5x across a wide range of DNN networks while improving the time-to-solution by 90x.

READ FULL TEXT

page 1

page 4

page 9

page 10

10/28/2020

DNA: Differentiable Network-Accelerator Co-Search

Powerful yet complex deep neural networks (DNNs) have fueled a booming d...
09/13/2019

HERALD: Optimizing Heterogeneous DNN Accelerators for Edge Devices

Recent advances in deep neural networks (DNNs) have made DNNs the backbo...
08/24/2021

METRO: A Software-Hardware Co-Design of Interconnections for Spatial DNN Accelerators

Tiled spatial architectures have proved to be an effective solution to b...
11/28/2021

Automated Runtime-Aware Scheduling for Multi-Tenant DNN Inference on GPU

With the fast development of deep neural networks (DNNs), many real-worl...
01/26/2022

DiGamma: Domain-aware Genetic Algorithm for HW-Mapping Co-optimization for DNN Accelerators

The design of DNN accelerators includes two key parts: HW resource confi...
04/28/2021

Domain-specific Genetic Algorithm for Multi-tenant DNNAccelerator Scheduling

As Deep Learning continues to drive a variety of applications in datacen...
09/10/2018

DNN Dataflow Choice Is Overrated

Many DNN accelerators have been proposed and built using different micro...