Efficient On-Chip Learning for Optical Neural Networks Through Power-Aware Sparse Zeroth-Order Optimization

12/21/2020
by   Jiaqi Gu, et al.
0

Optical neural networks (ONNs) have demonstrated record-breaking potential in high-performance neuromorphic computing due to their ultra-high execution speed and low energy consumption. However, current learning protocols fail to provide scalable and efficient solutions to photonic circuit optimization in practical applications. In this work, we propose a novel on-chip learning framework to release the full potential of ONNs for power-efficient in situ training. Instead of deploying implementation-costly back-propagation, we directly optimize the device configurations with computation budgets and power constraints. We are the first to model the ONN on-chip learning as a resource-constrained stochastic noisy zeroth-order optimization problem, and propose a novel mixed-training strategy with two-level sparsity and power-aware dynamic pruning to offer a scalable on-chip training solution in practical ONN deployment. Compared with previous methods, we are the first to optimize over 2,500 optical components on chip. We can achieve much better optimization stability, 3.7x-7.6x higher efficiency, and save >90 device variations and thermal crosstalk.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/27/2021

L2ight: Enabling On-Chip Learning for Optical Neural Networks via Efficient in-situ Subspace Optimization

Silicon-photonics-based optical neural network (ONN) is a promising hard...
research
04/04/2023

Physics-aware Roughness Optimization for Diffractive Optical Neural Networks

As a representative next-generation device/circuit technology beyond CMO...
research
11/11/2021

Silicon photonic subspace neural chip for hardware-efficient deep learning

As deep learning has shown revolutionary performance in many artificial ...
research
04/06/2019

Ring-Mesh: A Scalable and High-Performance Approach for Manycore Accelerators

There is an increasing number of works addressing the design challenge o...
research
04/03/2016

A New Learning Method for Inference Accuracy, Core Occupation, and Performance Co-optimization on TrueNorth Chip

IBM TrueNorth chip uses digital spikes to perform neuromorphic computing...
research
03/02/2023

Modeling and Exploration of Gain Competition Attacks in Optical Network-on-Chip Architectures

Network-on-Chip (NoC) enables energy-efficient communication between num...
research
03/29/2017

JetsonLEAP: a Framework to Measure Power on a Heterogeneous System-on-a-Chip Device

Computer science marches towards energy-aware practices. This trend impa...

Please sign up or login with your details

Forgot password? Click here to reset