• Photonics Research
  • Vol. 9, Issue 3, B71 (2021)
Xianxin Guo1、2、3、5、†,*, Thomas D. Barrett2、6、†,*, Zhiming M. Wang1、7、*, and A. I. Lvovsky2、4、8、*
Author Affiliations
  • 1Institute of Fundamental and Frontier Sciences, University of Electronic Science and Technology of China, Chengdu 610054, China
  • 2Clarendon Laboratory, University of Oxford, Oxford OX1 3PU, UK
  • 3Institute for Quantum Science and Technology, University of Calgary, Calgary, Alberta T2N 1N4, Canada
  • 4Russian Quantum Center, Skolkovo 143025, Moscow, Russia
  • 5e-mail: xianxin.guo@physics.ox.ac.uk
  • 6e-mail: thomas.barrett@physics.ox.ac.uk
  • 7e-mail: zhmwang@uestc.edu.cn
  • 8e-mail: alex.lvovsky@physics.ox.ac.uk
  • show less
    DOI: 10.1364/PRJ.411104 Cite this Article Set citation alerts
    Xianxin Guo, Thomas D. Barrett, Zhiming M. Wang, A. I. Lvovsky. Backpropagation through nonlinear units for the all-optical training of neural networks[J]. Photonics Research, 2021, 9(3): B71 Copy Citation Text show less

    Abstract

    We propose a practical scheme for end-to-end optical backpropagation in neural networks. Using saturable absorption for the nonlinear units, we find that the backward-propagating gradients required to train the network can be approximated in a surprisingly simple pump-probe scheme that requires only simple passive optical elements. Simulations show that, with readily obtainable optical depths, our approach can achieve equivalent performance to state-of-the-art computational networks on image classification benchmarks, even in deep networks with multiple sequential gradient approximation. With backpropagation through nonlinear units being an outstanding challenge to the field, this work provides a feasible path toward truly all-optical neural networks.

    zj(l)=iwji(l)ai(l1),

    View in Article

    wji(l)=zj(l)zj(l)wji(l)=δj(l)ai(l1),

    View in Article

    δj(l)=kzk(l+1)zk(l+1)zj(l)=g(zj(l))ρj(l+1),

    View in Article

    EP,out=g(EP,in)=exp(α0/21+EP,in2)EP,in,

    View in Article

    Epr,outEpr,in=exp(α0/21+EP,in2).

    View in Article

    g(EP,in)=[1+α0EP,in2(1+EP,in2)2]exp(α0/21+EP,in2).

    View in Article

    =i12(zi(L)ti)2,

    View in Article

    p(z)=12πσexp(z22σ2),(B1)

    View in Article

    S=|f(z)g(z)p(z)dz|2[f(z)]2p(z)dz·[g(z)]2p(z)dz.(B2)

    View in Article

    Isat=ωΓ2σ0=16.6  μWmm2,(C1)

    View in Article

    Xianxin Guo, Thomas D. Barrett, Zhiming M. Wang, A. I. Lvovsky. Backpropagation through nonlinear units for the all-optical training of neural networks[J]. Photonics Research, 2021, 9(3): B71
    Download Citation