Backpropagation through nonlinear units for all-optical training of neural networks

06/30/2020
by   Thomas D. Barrett, et al.
0

Backpropagation through nonlinear neurons is an outstanding challenge to the field of optical neural networks and the major conceptual barrier to all-optical training schemes. Each neuron is required to exhibit a directionally dependent response to propagating optical signals, with the backwards response conditioned on the forward signal, which is highly non-trivial to implement optically. We propose a practical and surprisingly simple solution that uses saturable absorption to provide the network nonlinearity. We find that the backward propagating gradients required to train the network can be approximated in a pump-probe scheme that requires only passive optical elements. Simulations show that, with readily obtainable optical depths, our approach can achieve equivalent performance to state-of-the-art computational networks on image classification benchmarks, even in deep networks with multiple sequential gradient approximations. This scheme is compatible with leading optical neural network proposals and therefore provides a feasible path towards end-to-end optical training.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 5

page 7

page 8

page 10

page 11

page 13

11/17/2020

ZORB: A Derivative-Free Backpropagation Algorithm for Neural Networks

Gradient descent and backpropagation have enabled neural networks to ach...
03/20/2022

Hybrid training of optical neural networks

Optical neural networks are emerging as a promising type of machine lear...
06/02/2020

Light-in-the-loop: using a photonics co-processor for scalable training of neural networks

As neural networks grow larger and more complex and data-hungry, trainin...
03/14/2020

Compatible Learning for Deep Photonic Neural Network

Realization of deep learning with coherent optical field has attracted r...
10/10/2019

Solving Optical Tomography with Deep Learning

This paper presents a neural network approach for solving two-dimensiona...
06/08/2019

Class-specific Differential Detection in Diffractive Optical Neural Networks Improves Inference Accuracy

Diffractive deep neural networks have been introduced earlier as an opti...
09/05/2019

Dispersion Characterization and Pulse Prediction with Machine Learning

In this work we demonstrate the efficacy of neural networks in the chara...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.