Massively parallel universal linear transformations using a wavelength-multiplexed diffractive optical network

Jingxi Li; Tianyi Gan; Bijie Bai; Yi Luo; Mona Jarrahi; Aydogan Ozcan

doi:10.1117/1.AP.5.1.016003

[1] D. R. Solli, B. Jalali. Analog optical computing. Nat. Photonics, 9, 704-706(2015).

[2] G. Wetzstein et al. Inference in artificial intelligence with deep optics and photonics. Nature, 588, 39-47(2020).

[3] B. J. Shastri et al. Photonics for artificial intelligence and neuromorphic computing. Nat. Photonics, 15, 102-114(2021).

[4] H. Zhou et al. Photonic matrix multiplication lights up photonic accelerator and beyond. Light Sci. Appl., 11, 30(2022).

[5] D. Mengu et al. At the intersection of optics and deep learning: statistical inference, computing, and inverse design. Adv. Opt. Photonics, 14, 209-290(2022).

[6] L. Cutrona et al. Optical data processing and filtering systems. IRE Trans. Inf. Theory, 6, 386-400(1960).

[7] J. J. Hopfield. Neural networks and physical systems with emergent collective computational abilities. Proc. Natl. Acad. Sci. U. S. A., 79, 2554-2558(1982).

[8] D. Psaltis, N. Farhat. Optical information processing based on an associative-memory model of neural nets with thresholding and feedback. Opt. Lett., 10, 98-100(1985).

[9] N. H. Farhat et al. Optical implementation of the Hopfield model. Appl. Opt., 24, 1469-1475(1985).

[10] K. Wagner, D. Psaltis. Multilayer optical learning networks. Appl. Opt., 26, 5061-5076(1987).

[11] D. Psaltis et al. Holography in artificial neural networks. Nature, 343, 325-330(1990).

[12] K. Vandoorne et al. Parallel reservoir computing using optical amplifiers. IEEE Trans. Neural Networks, 22, 1469-1481(2011).

[13] A. Silva et al. Performing mathematical operations with metamaterials. Science, 343, 160-163(2014).

[14] K. Vandoorne et al. Experimental demonstration of reservoir computing on a silicon photonics chip. Nat. Commun., 5, 3541(2014).

[15] J. Carolan et al. Universal linear optics. Science, 349, 711-716(2015).

[16] J. Chang et al. Hybrid optical-electronic convolutional neural networks with optimized diffractive optics for image classification. Sci. Rep., 8, 12324(2018).

[17] N. M. Estakhri, B. Edwards, N. Engheta. Inverse-designed metastructures that solve equations. Science, 363, 1333-1338(2019).

[18] J. Dong et al. Optical reservoir computing using multiple light scattering for chaotic systems prediction. IEEE J. Sel. Top. Quantum Electron., 26, 7701012(2020).

[19] U. Teğin et al. Scalable optical learning operator. Nat. Comput. Sci., 1, 542-549(2021).

[20] Y. Shen et al. Deep learning with coherent nanophotonic circuits. Nat. Photonics, 11, 441-446(2017).

[21] A. N. Tait et al. Neuromorphic photonic networks using silicon photonic weight banks. Sci. Rep., 7, 7430(2017).

[22] X. Lin et al. All-optical machine learning using diffractive deep neural networks. Science, 361, 1004-1008(2018).

[23] J. Bueno et al. Reinforcement learning in a large-scale photonic recurrent neural network. Optica, 5, 756-760(2018).

[24] Y. Zuo et al. All-optical neural network with nonlinear activation functions. Optica, 6, 1132-1137(2019).

[25] T. W. Hughes et al. Wave physics as an analog recurrent neural network. Sci. Adv., 5, eaay6946(2019).

[26] J. Feldmann et al. All-optical spiking neurosynaptic networks with self-learning capabilities. Nature, 569, 208-214(2019).

[27] M. Miscuglio, V. J. Sorger. Photonic tensor cores for machine learning. Appl. Phys. Rev., 7, 031404(2020).

[28] H. Zhang et al. An optical neural chip for implementing complex-valued neural network. Nat. Commun., 12, 457(2021).

[29] J. Feldmann et al. Parallel convolutional processing using an integrated photonic tensor core. Nature, 589, 52-58(2021).

[30] X. Xu et al. 11 TOPS photonic convolutional accelerator for optical neural networks. Nature, 589, 44-51(2021).

[31] L. G. Wright et al. Deep physical neural networks trained with backpropagation. Nature, 601, 549-555(2022).

[32] F. Ashtiani, A. J. Geers, F. Aflatouni. An on-chip photonic deep neural network for image classification. Nature, 606, 501-506(2022).

[33] D. Liu et al. Training deep neural networks for the inverse design of nanophotonic structures. ACS Photonics, 5, 1365-1369(2018).

[34] W. Ma, F. Cheng, Y. Liu. Deep-learning-enabled on-demand design of chiral metamaterials. ACS Nano, 12, 6326-6334(2018).

[35] J. Peurifoy et al. Nanophotonic particle simulation and inverse design using artificial neural networks. Sci. Adv., 4, eaar4206(2018).

[36] I. Malkiel et al. “Plasmonic nanostructure design and characterization via deep learning. Light Sci. Appl., 7, 60(2018).

[37] Z. Liu et al. Generative model for the inverse design of metasurfaces. Nano Lett., 18, 6570-6576(2018).

[38] S. So, J. Rho. Designing nanophotonic structures using conditional deep convolutional generative adversarial networks. Nanophotonics, 8, 1255-1261(2019).

[39] W. Ma et al. Probabilistic representation and inverse design of metamaterials based on a deep generative model with semi-supervised learning strategy. Adv. Mater., 31, 1901111(2019).

[40] S. An et al. A deep learning approach for objective-driven all-dielectric metasurface design. ACS Photonics, 6, 3196-3207(2019).

[41] J. Jiang et al. Free-form diffractive metagrating design based on generative adversarial networks. ACS Nano, 13, 8872-8878(2019).

[42] C. Qian et al. Deep-learning-enabled self-adaptive microwave cloak without human intervention. Nat. Photonics, 14, 383-390(2020).

[43] Z. Liu et al. Compounding meta-atoms into metamolecules with hybrid artificial intelligence techniques. Adv. Mater., 32, 1904790(2020).

[44] H. Ren et al. Three-dimensional vectorial holography based on machine learning inverse design. Sci. Adv., 6, eaaz4261(2020).

[45] C. Zuo, Q. Chen. Exploiting optical degrees of freedom for information multiplexing in diffractive neural networks. Light Sci. Appl., 11, 208(2022).

[46] D. Mengu et al. Analysis of diffractive optical neural networks and their integration with electronic neural networks. IEEE J. Sel. Top. Quantum Electron., 26, 3700114(2020).

[47] J. Li et al. Class-specific differential detection in diffractive optical neural networks improves inference accuracy. Adv. Photonics, 1, 046001(2019).

[48] T. Yan et al. Fourier-space diffractive deep neural network. Phys. Rev. Lett., 123, 023901(2019).

[49] D. Mengu, Y. Rivenson, A. Ozcan. Scale-, shift-, and rotation-invariant diffractive optical networks. ACS Photonics, 8, 324-334(2020).

[50] D. Mengu et al. Misalignment resilient diffractive optical networks. Nanophotonics, 9, 4207-4219(2020).

[51] M. S. S. Rahman et al. Ensemble learning of diffractive optical networks. Light Sci. Appl., 10, 14(2021).

[52] J. Li et al. Spectrally encoded single-pixel machine vision using diffractive networks. Sci. Adv., 7, eabd7690(2021).

[53] O. Kulce et al. All-optical information-processing capacity of diffractive surfaces. Light Sci. Appl., 10, 25(2021).

[54] T. Zhou et al. Large-scale neuromorphic optoelectronic computing with a reconfigurable diffractive processing unit. Nat. Photonics, 15, 367-373(2021).

[55] H. Chen et al. Diffractive deep neural networks at visible wavelengths. Engineering, 7, 1483-1491(2021).

[56] C. Liu et al. A programmable diffractive deep neural network based on a digital-coding metasurface array. Nat. Electron., 5, 113-122(2022).

[57] D. Mengu et al. Classification and reconstruction of spatially overlapping phase images using diffractive optical networks. Sci. Rep., 12, 8446(2022).

[58] Y. Luo et al. Computational imaging without a computer: seeing through random diffusers at the speed of light. eLight, 2, 4(2022).

[59] D. Mengu et al. Diffractive interconnects: all-optical permutation operation using diffractive networks. Nanophotonics(2022).

[60] D. Mengu, A. Ozcan. All-optical phase recovery: diffractive computing for quantitative phase imaging. Adv. Opt. Mater., 10, 2200281(2022).

[61] B. Bai et al. To image, or not to image: class-specific diffractive cameras with all-optical erasure of undesired objects. eLight, 2, 14(2022).

[62] Ç. Işıl et al. Super-resolution image display using diffractive decoders. Sci. Adv., 8, eadd3433(2022).

[63] C. Qian et al. Performing optical logic operations by a diffractive neural network. Light Sci. Appl., 9, 59(2020).

[64] P. Wang et al. Orbital angular momentum mode logical operation using optical diffractive neural network. Photonics Res., 9, 2116-2124(2021).

[65] Y. Luo, D. Mengu, A. Ozcan. Cascadable all-optical NAND gates using diffractive networks. Sci. Rep., 12, 7121(2022).

[66] Y. Luo et al. Design of task-specific optical systems using broadband diffractive neural networks. Light Sci. Appl., 8, 112(2019).

[67] M. Veli et al. Terahertz pulse shaping using diffractive surfaces. Nat. Commun., 12, 37(2021).

[68] Z. Huang et al. All-optical signal processing of vortex beams with diffractive deep neural networks. Phys. Rev. Appl., 15, 014037(2021).

[69] O. Kulce et al. All-optical synthesis of an arbitrary linear transformation using diffractive surfaces. Light Sci. Appl., 10, 196(2021).

[70] J. Li et al. Polarization multiplexed diffractive computing: all-optical implementation of a group of linear transformations through a polarization-encoded diffractive network. Light Sci. Appl., 11, 153(2022).

[71] T. Ishihara et al. An optical neural network architecture based on highly parallelized WDM-multiplier-accumulator. IEEE/ACM Workshop on Photonics-Opt. Technol. Oriented Networking, Inf. and Comput. Syst. (PHOTONICS), 15-21(2019).

[72] R. Hamerly et al. Edge computing with optical neural networks via WDM weight broadcasting. Proc. SPIE, 11804, 118041R(2021).

[73] A. Totovic et al. Programmable photonic neural networks combining WDM with coherent linear optics. Sci. Rep., 12, 5605(2022).

[74] TSL-570|SANTEC CORPORATION: The photonics pioneer.

[75] MEMS-VCSEL swept-wavelength laser sources.

[76] D. P. Kingma, J. Ba. Adam: a method for stochastic optimization(2014).

[77] I. Loshchilov, F. Hutter. Decoupled weight decay regularization, 18(2019).