Pruning Coherent Integrated Photonic Neural Networks Using the Lottery Ticket Hypothesis

12/14/2021
by   Sanmitra Banerjee, et al.
0

Singular-value-decomposition-based coherent integrated photonic neural networks (SC-IPNNs) have a large footprint, suffer from high static power consumption for training and inference, and cannot be pruned using conventional DNN pruning techniques. We leverage the lottery ticket hypothesis to propose the first hardware-aware pruning method for SC-IPNNs that alleviates these challenges by minimizing the number of weight parameters. We prune a multi-layer perceptron-based SC-IPNN and show that up to 89 angles, which correspond to weight parameters in SC-IPNNs, can be pruned with a negligible accuracy loss (smaller than 5 consumption by up to 86

READ FULL TEXT

page 1

page 3

page 6

research
12/11/2021

CHAMP: Coherent Hardware-Aware Magnitude Pruning of Integrated Photonic Neural Networks

We propose a novel hardware-aware magnitude pruning technique for cohere...
research
03/15/2023

Deep Learning Weight Pruning with RMT-SVD: Increasing Accuracy and Reducing Overfitting

In this work, we present some applications of random matrix theory for t...
research
02/03/2018

An Area and Energy Efficient Design of Domain-Wall Memory-Based Deep Convolutional Neural Networks using Stochastic Computing

With recent trend of wearable devices and Internet of Things (IoTs), it ...
research
03/23/2020

Safe Crossover of Neural Networks Through Neuron Alignment

One of the main and largely unexplored challenges in evolvingthe weights...
research
05/30/2021

MLPruning: A Multilevel Structured Pruning Framework for Transformer-based Models

Pruning is an effective method to reduce the memory footprint and comput...
research
03/05/2023

Scalable Object Detection on Embedded Devices Using Weight Pruning and Singular Value Decomposition

This paper presents a method for optimizing object detection models by c...
research
07/22/2022

Characterizing Coherent Integrated Photonic Neural Networks under Imperfections

Integrated photonic neural networks (IPNNs) are emerging as promising su...

Please sign up or login with your details

Forgot password? Click here to reset