SparDA: Accelerating Dynamic Sparse Deep Neural Networks via Sparse-Dense Transformation

01/26/2023
by   Ningxin Zheng, et al.
0

Due to its high cost-effectiveness, sparsity has become the most important approach for building efficient deep-learning models. However, commodity accelerators are built mainly for efficient dense computation, creating a huge gap for general sparse computation to leverage. Existing solutions have to use time-consuming compiling to improve the efficiency of sparse kernels in an ahead-of-time manner and thus are limited to static sparsity. A wide range of dynamic sparsity opportunities is missed because their sparsity patterns are only known at runtime. This limits the future of building more biological brain-like neural networks that should be dynamically and sparsely activated. In this paper, we bridge the gap between sparse computation and commodity accelerators by proposing a system, called Spider, for efficiently executing deep learning models with dynamic sparsity. We identify an important property called permutation invariant that applies to most deep-learning computations. The property enables Spider (1) to extract dynamic sparsity patterns of tensors that are only known at runtime with little overhead; and (2) to transform the dynamic sparse computation into an equivalent dense computation which has been extremely optimized on commodity accelerators. Extensive evaluation on diverse models shows Spider can extract and transform dynamic sparsity with negligible overhead but brings up to 9.4x speedup over state-of-art solutions.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/27/2021

Griffin: Rethinking Sparse Optimization for Deep Learning Architectures

This paper examines the design space trade-offs of DNNs accelerators aim...
research
04/15/2023

STen: Productive and Efficient Sparsity in PyTorch

As deep learning models grow, sparsity is becoming an increasingly criti...
research
08/12/2022

An Algorithm-Hardware Co-Optimized Framework for Accelerating N:M Sparse Transformers

The Transformer has been an indispensable staple in deep learning. Howev...
research
03/29/2023

PopSparse: Accelerated block sparse matrix multiplication on IPU

Reducing the computational cost of running large scale neural networks u...
research
03/03/2023

Sparsity May Cry: Let Us Fail (Current) Sparse Neural Networks Together!

Sparse Neural Networks (SNNs) have received voluminous attention predomi...
research
11/07/2019

Transformation of Dense and Sparse Text Representations

Sparsity is regarded as a desirable property of representations, especia...
research
02/09/2022

Compiler Support for Sparse Tensor Computations in MLIR

Sparse tensors arise in problems in science, engineering, machine learni...

Please sign up or login with your details

Forgot password? Click here to reset