Artificial Intelligence Nanophotonics: Optical Neural Networks and Nanophotonics

Haitao Luan; Xi Chen; Qiming Zhang; Haoyi Yu; Min Gu

doi:10.3788/AOS202141.0823005

[1] McCulloch W S, Pitts W. A logical calculus of the ideas immanent in nervous activity[J]. The bulletin of Mathematical Biophysics, 5, 115-133(1943).

[2] Krenker A, Bester J, Kos A. Introduction to the artificial neural networks[M]. ∥Artificial Neural Networks - Methodological Advances and Biomedical Applications. London: InTech, 25693(2011).

[3] Thompson R. The neurobiology of learning and memory[J]. Science, 233, 941-947(1986).

[4] Dayhoff J E. DeLeo J M. Artificial neural networks[J]. Cancer, 91, 1615-1635(2001).

[5] Dzierma Y, Schuermann M, Melchior P et al. Optimizing adjuvant stereotactic radiotherapy of motor-eloquent brain metastases: sparing the nTMS-defined motor cortex and the hippocampus[J]. Frontiers in Oncology, 11, 628007(2021).

[6] Sengupta B, Stemmler M B. Power consumption during neuronal computation[J]. Proceedings of the IEEE, 102, 738-750(2014).

[7] Schwabe R J, Zelinger S, Key T S et al. Electronic lighting interference[J]. IEEE Industry Applications Magazine, 4, 43-48(1998).

[8] Chaisakul P, Marris-Morini D, Frigerio J et al. Integrated germanium optical interconnects on silicon substrates[J]. Nature Photonics, 8, 482-488(2014).

[9] Woods D, Naughton T J. Photonic neural networks[J]. Nature Physics, 8, 257-259(2012).

[10] Feitelson D G. Optical computing: a survey for computer scientists[J]. Applied Optics, 28, 2182-2183(1989).

[11] Zhang Q M, Yu H Y, Barbiero M et al. Artificial neural networks enabled by nanophotonics[J]. Light: Science & Applications, 8, 42(2019).

[12] Athale R A, Psaltis D, Wagner K. Optical computing: introduction by the guest editors to the feature in the 1 May 1988 issue[J]. Applied Optics, 27, 1641-1642(1988).

[13] Engheta N. Circuits with light at nanoscales: optical nanocircuits inspired by metamaterials[J]. Science, 317, 1698-1702(2007).

[14] Goodman J W, Dias A R, Woody L M. Fully parallel, high-speed incoherent optical method for performing discrete Fourier transforms[J]. Optics Letters, 2, 1-3(1978).

[15] Caulfield H J, Dolev S. Why future supercomputing requires optics[J]. Nature Photonics, 4, 261-263(2010).

[16] Xiang L, Wang Q, Nie D et al. Deep embedding convolutional neural network for synthesizing CT image from T1-Weighted MR image[J]. Medical Image Analysis, 47, 31-44(2018).

[17] Molesky S, Lin Z, Piggott A Y et al. Inverse design in nanophotonics[J]. Nature Photonics, 12, 659-670(2018).

[18] Haddad W, Hui Q, Bailey J. Human brain networks: spiking neuron models, multistability, synchronization, thermodynamics, maximum entropy production, and anesthetic cascade mechanisms[J]. Entropy, 16, 3939-4003(2014).

[19] Harris K M, Jensen F E, Tsao B. Three-dimensional structure of dendritic spines and synapses in rat hippocampus (CA1) at postnatal day 15 and adult ages: implications for the maturation of synaptic physiology and long-term potentiation[J]. The Journal of Neuroscience, 12, 2685-2705(1992).

[20] Koch C, Segev I. The role of single neurons in information processing[J]. Nature Neuroscience, 3, 1171-1177(2000).

[21] Nahmias M A, Shastri B J, Tait A N et al. Neuromorphic photonics[J]. Optics and Photonics News, 29, 34-41(2018).

[22] Rosenblatt F. The perceptron: a probabilistic model for information storage and organization in the brain[J]. Psychological Review, 65, 386(1958).

[23] Minsky M, Papert S A[M]. Perceptrons(2017).

[24] Rumelhart D E, Hinton G E, Williams R J. Learning representations by back-propagating errors[J]. Nature, 323, 533-536(1986).

[25] Lecun Y. PhD thesis: modeles connexionnistes de l'apprentissage (connectionist learning models)[D]. Paris: Université de Paris(1987).

[26] Lecun Y, Boser B, Denker J S et al. Backpropagation applied to handwritten zip code recognition[J]. Neural Computation, 1, 541-551(1989).

[27] Cun Y L, Boser B, Denker J S et al. Handwritten digit recognition with a back-propagation network. [C]∥Advances in Neural Information Processing Systems, June 1, 1990, San Francisco, CA, United States. San Francisco: Morgan Kaufmann, 396-404(1990).

[28] Yamashita R, Nishio M. Do R K G, et al. Convolutional neural networks: an overview and application in radiology[J]. Insights into Imaging, 9, 611-629(2018).

[29] Ning Y S, He S, Wu Z Y et al. A review of deep learning based speech synthesis[J]. Applied Sciences, 9, 4050(2019).

[30] Psaltis D, Farhat N. Optical information processing based on an associative-memory model of neural nets with thresholding and feedback[J]. Optics Letters, 10, 98-100(1985).

[31] Farhat N H, Psaltis D, Prata A et al. Optical implementation of the Hopfield model[J]. Applied Optics, 24, 1469-1475(1985).

[32] Psaltis D, Brady D, Gu X G et al. Holography in artificial neural networks[J]. Nature, 343, 325-330(1990).

[33] Yeh S L, Lo R C, Shi C Y. Optical implementation of the Hopfield neural network with matrix gratings[J]. Applied Optics, 43, 858-865(2004).

[34] Bueno J, Maktoobi S, Froehly L et al. Reinforcement learning in a large-scale photonic recurrent neural network[J]. Optica, 5, 756-760(2018).

[35] Lu T, Wu S, Xu X et al. Two-dimensional programmable optical neural network[J]. Applied Optics, 28, 4908-4913(1989).

[36] Kirchain R, Kimerling L. A roadmap for nanophotonics[J]. Nature Photonics, 1, 303-305(2007).

[37] Tait A N, de Lima T F, Zhou E et al. Neuromorphic photonic networks using silicon photonic weight banks[J]. Scientific Reports, 7, 7430(2017).

[38] Shen Y C, Harris N C, Skirlo S et al. Deep learning with coherent nanophotonic circuits[C]∥2017 IEEE Photonics Society Summer Topical Meeting Series (SUM), July 10-12, 2017, San Juan, PR, USA., 189-190(2017).

[39] Lin X, Rivenson Y, Yardimci N T et al. All-optical machine learning using diffractive deep neural networks[J]. Science, 361, 1004-1008(2018).

[40] Chang J, Sitzmann V, Dun X et al. Hybrid optical-electronic convolutional neural networks with optimized diffractive optics for image classification[J]. Scientific Reports, 8, 12324(2018).

[41] Li X, Lan T H, Tien C H et al. Three-dimensional orientation-unlimited polarization encryption by a single optically configured vectorial beam[J]. Nature Communications, 3, 998(2012).

[42] Gan Z, Cao Y, Evans R A et al. Three-dimensional deep sub-diffraction optical beam lithography with 9 nm feature size[J]. Nature Communications, 4, 2061(2013).

[43] Yue Z, Xue G, Liu J et al. Nanometric holograms based on a topological insulator material[J]. Nature Communications, 8, 15354(2017).

[44] Yue Z J, Ren H R, Wei S B et al. Angular-momentum nanometrology in an ultrathin plasmonic topological insulator film[J]. Nature Communications, 9, 4413(2018).

[45] Gu M, Fang X Y, Ren H R et al. Optically digitalized holography: a perspective for all-optical machine learning[J]. Engineering, 5, 363-365(2019).

[46] Miscuglio M, Hu Z B, Li S R et al. Massively parallel amplitude-only Fourier neural network[J]. Optica, 7, 1812-1819(2020).

[47] Chen H, Feng J N, Jiang M W et al[2021-03-15]. Diffractive deep neural networks at visible wavelengths [2021-03-15].https:∥www.sciencedirect.com/science/article/pii/S2095809921000448?via%3Dihub..

[48] Goi E, Chen X, Zhang Q M et al. Nanoprinted high-neuron-density optical linear perceptrons performing near-infrared inference on a CMOS chip[J]. Light: Science & Applications, 10, 40(2021).

[49] Hell S W, Wichmann J. Breaking the diffraction resolution limit by stimulated emission: stimulated-emission-depletion fluorescence microscopy[J]. Optics Letters, 19, 780-782(1994).

[50] Betzig E, Patterson G H, Sougrat R et al. Imaging intracellular fluorescent proteins at nanometer resolution[J]. Science, 313, 1642-1645(2006).

[51] Johnson S G, Mekis A, Fan S et al. Molding the flow of light[J]. Computing in Science & Engineering, 3, 38-47(2001).

[52] Jahani S, Jacob Z. All-dielectric metamaterials[J]. Nature Nanotechnology, 11, 23-36(2016).

[53] Mokkapati S. Beck F J, de Waele R, et al. Resonant nano-antennas for light trapping in plasmonic solar cells[J]. Journal of Physics D: Applied Physics, 44, 185101(2011).

[54] Hsiao H H, Chu C H, Tsai D P. Metasurfaces: fundamentals and applications of metasurfaces[J]. Small Methods, 1, 1770041(2017).

[55] Minovich A E, Miroshnichenko A E, Bykov A Y et al. Functional and nonlinear optical metasurfaces[J]. Laser & Photonics Reviews, 9, 195-213(2015).

[56] Baranov D G, Wersäll M, Cuadra J et al. Novel nanostructures and materials for strong light-matter interactions[J]. ACS Photonics, 5, 24-42(2018).

[57] Pelton M, Aizpurua J, Bryant G. Metal-nanoparticle plasmonics[J]. Laser & Photonics Review, 2, 136-159(2008).

[58] Borel P I, Harpøth A, Frandsen L H et al. Topology optimization and fabrication of photonic crystal structures[J]. Optics Express, 12, 1996-2001(2004).

[59] Piggott A Y, Lu J, Lagoudakis K G et al. Inverse design and demonstration of a compact and broadband on-chip wavelength demultiplexer[J]. Nature Photonics, 9, 374-377(2015).

[60] Shen B, Wang P, Polson R et al. An integrated-nanophotonics polarization beamsplitter with 2.4×2.4 μm 2 footprint[J]. Nature Photonics, 9, 378-382(2015).

[61] Huntington M D, Lauhon L J, Odom T W. Subwavelength lattice optics by evolutionary design[J]. Nano Letters, 14, 7195-7200(2014).

[62] Feichtner T, Selig O, Kiunke M et al. Evolutionary optimization of optical antennas[J]. Physical Review Letters, 109, 127701(2012).

[63] Wiecha P R, Arbouet A, Girard C et al. Evolutionary multi-objective optimization of colour pixels based on dielectric nanoantennas[J]. Nature Nanotechnology, 12, 163-169(2017).

[64] Kristensen A. Yang J K W, Bozhevolnyi S I, et al. Plasmonic colour generation[J]. Nature Reviews Materials, 2, 16088(2017).

[65] Malkiel I, Mrejen M, Nagler A et al. Plasmonic nanostructure design and characterization via deep learning[J]. Light, Science & Applications, 7, 60(2018).

[66] Ma W, Cheng F, Liu Y M. Deep-learning-enabled on-demand design of chiral metamaterials[J]. ACS Nano, 12, 6326-6334(2018).

[67] Liu D J, Tan Y X, Khoram E et al. Training deep neural networks for the inverse design of nanophotonic structures[C]∥2019 Conference on Lasers and Electro-Optics (CLEO), May 5-10, 2019, San Jose, CA, USA., 1-2(2019).

[68] Alagappan G, Png C E. Modal classification in optical waveguides using deep learning[J]. Journal of Modern Optics, 66, 557-561(2019).

[69] Alagappan G, Png C E. Deep learning models for effective refractive indices in silicon nitride waveguides[J]. Journal of Optics, 21, 035801(2019).

[70] Kiarashinejad Y, Abdollahramezani S, Zandehshahvar M et al. Deep learning reveals underlying physics of light-matter interactions in nanophotonic devices[J]. Advanced Theory and Simulations, 2, 1900088(2019).

[71] Bessonov A A, Kirikova M N, Petukhov D I et al. Layered memristive and memcapacitive switches for printable electronics[J]. Nature Materials, 14, 199-204(2015).

[72] Ohno T, Hasegawa T, Tsuruoka T et al. Short-term plasticity and long-term potentiation mimicked in single inorganic synapses[J]. Nature Materials, 10, 591-595(2011).

[73] Gao S, Liu G, Yang H L et al. An oxide Schottky junction artificial optoelectronic synapse[J]. ACS Nano, 13, 2634-2642(2019).

[74] Li Y, Zhong Y, Zhang J et al. Activity-dependent synaptic plasticity of a chalcogenide electronic synapse for neuromorphic systems[J]. Scientific Reports, 4, 4906(2014).

[75] Zhang J Y, Dai S L, Zhao Y W et al. Recent progress in photonic synapses for neuromorphic systems[J]. Advanced Intelligent Systems, 2, 1900136(2020).

[76] Lee M, Lee W, Choi S et al. Brain-inspired photonic neuromorphic devices using photodynamic amorphous oxide semiconductors and their persistent photoconductivity[J]. Advanced Materials, 29, 1700951(2017).

[77] Iwasaki T, Itagaki N, Den T et al. Combinatorial approach to thin-film transistors using multicomponent semiconductor channels: an application to amorphous oxide semiconductors in In-Ga-Zn-O system[J]. Applied Physics Letters, 90, 242114(2007).

[78] Wang Z, Yin M, Zhang T et al. Engineering incremental resistive switching in TaOx based memristors for brain-inspired computing[J]. Nanoscale, 8, 14015-14022(2016).

[79] Hu D C, Yang R, Jiang L et al. Memristive synapses with photoelectric plasticity realized in ZnO1-x/AlOy heterojunction[J]. ACS Applied Materials & Interfaces, 10, 6463-6470(2018).

[80] Zhu X J, Lu W D. Optogenetics-inspired tunable synaptic functions in memristors[J]. ACS Nano, 12, 1242-1249(2018).

[81] Wang Y, Lü Z, Chen J R et al. Photonic flash memory: photonic synapses based on inorganic perovskite quantum dots for neuromorphic computing[J]. Advanced Materials, 30, 1870287(2018).

[82] Shao L, Wang H L, Yang Y et al. Optoelectronic properties of printed photogating carbon nanotube thin film transistors and their application for light-stimulated neuromorphic devices[J]. ACS Applied Materials & Interfaces, 11, 12161-12169(2019).

[83] Wang H, Zhao Q, Ni Z et al. A ferroelectric/electrochemical modulated organic synapse for ultraflexible, artificial visual-perception system[J]. Advanced Materials, 30, e1803961(2018).

[84] Jeon S, Ahn S E, Song I et al. Gated three-terminal device architecture to eliminate persistent photoconductivity in oxide semiconductor photosensor arrays[J]. Nature Materials, 11, 301-305(2012).

[85] Bera A, Peng H Y, Lourembam J et al. A versatile light-switchable nanorod memory: wurtzite ZnO on perovskite SrTiO3[J]. Advanced Functional Materials, 23, 4977-4984(2013).

[86] Robertson J. Band offsets of wide-band-gap oxides and implications for future electronic devices[J]. Journal of Vacuum Science & Technology B: Microelectronics and Nanometer Structures, 18, 1785-1791(2000).

[87] Park Y, Choong V, Gao Y et al. Work function of indium tin oxide transparent conductor measured by photoelectron spectroscopy[J]. Applied Physics Letters, 68, 2699-2701(1996).

[88] Azpiroz J M, Mosconi E, Bisquert J et al. Defect migration in methylammonium lead iodide and its role in perovskite solar cell operation[J]. Energy & Environmental Science, 8, 2118-2127(2015).

[89] Löper P, Stuckelberger M, Niesen B et al. Complex refractive index spectra of CH3NH3PbI3 perovskite thin films determined by spectroscopic ellipsometry and spectrophotometry[J]. The Journal of Physical Chemistry Letters, 6, 66-71(2015).

[90] Sun J, Oh S, Choi Y et al. Optoelectronic synapse based on IGZO-alkylated graphene oxide hybrid structure[J]. Advanced Functional Materials, 28, 1804397(2018).

[91] Seo S, Jo S H, Kim S et al. Artificial optic-neural synapse for colored and color-mixed pattern recognition[J]. Nature Communications, 9, 5106(2018).

[92] Lü Z, Chen M, Qian F S et al. Photonic synapse: mimicking neuroplasticity in a hybrid biopolymer transistor by dual modes modulation[J]. Advanced Functional Materials, 29, 1970212(2019).

[93] Ham S, Choi S, Cho H et al. Photonic artificial synapses: photonic organolead halide perovskite artificial synapse capable of accelerated learning at low power inspired by dopamine-facilitated synaptic activity[J]. Advanced Functional Materials, 29, 1970031(2019).