ReaLPrune: ReRAM Crossbar-aware Lottery Ticket Pruned CNNs

11/17/2021
by   Biresh Kumar Joardar, et al.
0

Training machine learning (ML) models at the edge (on-chip training on end user devices) can address many pressing challenges including data privacy/security, increase the accessibility of ML applications to different parts of the world by reducing the dependence on the communication fabric and the cloud infrastructure, and meet the real-time requirements of AR/VR applications. However, existing edge platforms do not have sufficient computing capabilities to support complex ML tasks such as training large CNNs. ReRAM-based architectures offer high-performance yet energy efficient computing platforms for on-chip CNN training/inferencing. However, ReRAM-based architectures are not scalable with the size of the CNN. Larger CNNs have more weights, which requires more ReRAM cells that cannot be integrated in a single chip. Moreover, training larger CNNs on-chip will require higher power, which cannot be afforded by these smaller devices. Pruning is an effective way to solve this problem. However, existing pruning techniques are either targeted for inferencing only, or they are not crossbar-aware. This leads to sub-optimal hardware savings and performance benefits for CNN training on ReRAM-based architectures. In this paper, we address this problem by proposing a novel crossbar-aware pruning strategy, referred as ReaLPrune, which can prune more than 90 any accuracy loss. Experimental results indicate that ReaLPrune reduces hardware requirements by 77.2 unpruned CNNs. ReaLPrune also outperforms other crossbar-aware pruning techniques in terms of both performance and hardware savings. In addition, ReaLPrune is equally effective for diverse datasets and more complex CNNs

READ FULL TEXT

page 1

page 2

page 3

page 4

research
12/05/2017

On-Chip Communication Network for Efficient Training of Deep Convolutional Networks on Heterogeneous Manycore Systems

Convolutional Neural Networks (CNNs) have shown a great deal of success ...
research
07/14/2022

PR-DARTS: Pruning-Based Differentiable Architecture Search

The deployment of Convolutional Neural Networks (CNNs) on edge devices i...
research
03/20/2023

What does it take to catch a Chinchilla? Verifying Rules on Large-Scale Neural Network Training via Compute Monitoring

As advanced machine learning systems' capabilities begin to play a signi...
research
06/15/2020

Now that I can see, I can improve: Enabling data-driven finetuning of CNNs on the edge

In today's world, a vast amount of data is being generated by edge devic...
research
06/25/2021

A Photonic-Circuits-Inspired Compact Network: Toward Real-Time Wireless Signal Classification at the Edge

Machine learning (ML) methods are ubiquitous in wireless communication s...
research
10/21/2020

Performance Prediction for Convolutional Neural Networks in Edge Devices

Running Convolutional Neural Network (CNN) based applications on edge de...
research
11/21/2020

MacLeR: Machine Learning-based Run-Time Hardware Trojan Detection in Resource-Constrained IoT Edge Devices

Traditional learning-based approaches for run-time Hardware Trojan detec...

Please sign up or login with your details

Forgot password? Click here to reset