Log In Sign Up

Alleviating Bottlenecks for DNN Execution on GPUs via Opportunistic Computing

by   Xianwei Cheng, et al.

Edge computing and IoT applications are severely constrained by limited hardware resource. This makes memory consuming DNN frameworks not applicable to edge computing. Simple algorithms such as direct convolution are finding their way in embedded machine learning. As one of the most widely used platforms for DNN acceleration, GPUs face the bottleneck of on-chip bandwidth. This work introduces a GPU DNN execution architecture that targets on relieving the on-chip bandwidth bottleneck by reducing data movement through opportunistic computing. We first investigate data access patterns in the hardware view rather than the software view. Then we propose two opportunistic computing techniques to predictably perform computation when data is available with the help of assistant warps. By moving computation to data, our techniques are able to significantly reduce data movement and relieve the DNN execution bottleneck. Our evaluation results show that the proposed technique can improve DNN application performance as much as 55


Modeling of Deep Neural Network (DNN) Placement and Inference in Edge Computing

With the edge computing becoming an increasingly adopted concept in syst...

Enabling On-Device Smartphone GPU based Training: Lessons Learned

Deep Learning (DL) has shown impressive performance in many mobile appli...

Lightning: Striking the Secure Isolation on GPU Clouds with Transient Hardware Faults

GPU clouds have become a popular computing platform because of the cost ...

HUGE2: a Highly Untangled Generative-model Engine for Edge-computing

As a type of prominent studies in deep learning, generative models have ...

A Customized NoC Architecture to Enable Highly Localized Computing-On-the-Move DNN Dataflow

The ever-increasing computation complexity of fastgrowing Deep Neural Ne...

CAVBench: A Benchmark Suite for Connected and Autonomous Vehicles

Connected and autonomous vehicles (CAVs) have recently attracted a signi...