Delay-weight plasticity-based supervised learning in optical spiking neural networks

Yanan Han; Shuiying Xiang; Zhenxing Ren; Chentao Fu; Aijun Wen; Yue Hao

doi:10.1364/PRJ.413742

[1] S. Ghosh-Dastidar, H. Adeli. Spiking neural networks. Int. J. Neural Syst., 19, 295-308(2009).

[2] R. Brette, M. Rudolph, T. Carnevale, M. Hines, D. Beeman, J. M. Bower, M. Diesmann, A. Morrison, P. H. Goodman, F. C. Harris, M. Zirpe, T. Natschläger, D. Pecevski, B. Ermentrout, M. Djurfeldt, A. Lansner, O. Rochel, T. Vieville, E. Muller, A. P. Davison, S. El Boustani, A. Destexhe. Simulation of networks of spiking neurons: a review of tools and strategies. J. Comput. Neurosci., 23, 349-398(2007).

[3] N. Kasabov, K. Dhoble, N. Nuntalid, G. Indiveri. Dynamic evolving spiking neural networks for on-line spatio- and spectro-temporal pattern recognition. Neural Netw., 41, 188-201(2013).

[4] K. Roy, J. Akhilesh, P. Priyadarshini. Towards spike-based machine intelligence with neuromorphic computing. Nature, 575, 607-617(2019).

[5] W. Q. Zhang, B. Gao, J. S. Tang, P. Yao, H. Q. Wu. Neuro-inspired computing chips. Nat. Electron., 3, 371-382(2020).

[6] T. Clarence, M. Arlija, N. Kasabov. Spiking neural networks: background, recent development and the NeuCube architecture. Neural Process. Lett., 3, 1675-1701(2020).

[7] A. Cariani. Temporal codes and computations for sensory representation and scene analysis. IEEE Trans. Neural Netw., 15, 1100-1111(2004).

[8] S. M. Bohte. The evidence for neural information processing with precise spike-times: a survey. Natural Comput., 3, 195-206(2004).

[9] A. Mohemmed, S. Schliebs. Training spiking neural networks to associate spatio-temporal input-output spike patterns. Neurocomputing, 107, 3-10(2013).

[10] S. B. Laughlin, R. R. de Ruyter van Steveninck, J. C. Anderson. The metabolic cost of neural information. Nat. Neurosci., 1, 36-41(1998).

[11] S. B. Laughlin. Energy as a constraint on the coding and processing of sensory information. Curr. Opinion Neurobiol., 11, 475-480(2001).

[12] H. Paugam-Moisy, S. Bohte. Computing with spiking neuron networks. Handbook of Natural Computing, 335-376(2012).

[13] J. Hu, H. Tang, K. C. Tan, H. Li, L. Shi. A spike-timing-based integrated model for pattern recognition. Neural Comput., 25, 450-472(2013).

[14] F. Ponulak, A. Kasinski. Introduction to spiking neural networks: information processing, learning and applications. Acta Neurobiol. Experim., 71, 409-433(2011).

[15] H. Jörntell, C. Hansel. Synaptic memories upside down: bidirectional plasticity at cerebellar parallel fiber-Purkinje cell synapses. Neuron, 52, 227-238(2006).

[16] R. Gütig, H. Sompolinsky. The tempotron: a neuron that learns spike timing-based decisions. Nat. Neurosci., 9, 420-428(2006).

[17] F. Ponulak, A. Kasinski. Supervised learning in spiking neural networks with ReSuMe: sequence learning, classification, and spike shifting. Neural Comput., 22, 467-510(2010).

[18] I. Sporea, A. Grüning. Supervised learning in multilayer spiking neural networks. Neural Comput., 25, 473-509(2013).

[19] S. R. Kulkarni, B. Rajendran. Spiking neural networks for handwritten digit recognition—supervised learning and network optimization. Neural Netw., 103, 118-127(2018).

[20] C. Hong, X. Wei, J. Wang, B. Deng, H. Yu, Y. Che. Training spiking neural networks for cognitive tasks: a versatile framework compatible with various temporal codes. IEEE Trans. Neural Netw. Learn. Syst., 31, 1285-1296(2020).

[21] S. Boudkkazi, L. Fronzaroli-Molinieres, D. Debanne. Presynaptic action potential waveform determines cortical synaptic latency. J. Physiol., 589, 1117-1131(2011).

[22] J. W. Lin, D. S. Faber. Modulation of synaptic delay during synaptic plasticity. Trends Neurosci., 25, 449-455(2002).

[23] C. W. Eurich, K. Pawelzik, U. Ernst, J. D. Cowan, J. G. Milton. Dynamics of self-organized delay adaptation. Phys. Rev. Lett., 82, 1594-1597(1999).

[24] X. B. Gong, Y. B. Wang, B. Ying. Delay-induced firing behavior and transitions in adaptive neuronal networks with two types of synapses. Sci. China Chem., 56, 222-229(2012).

[25] M. Dhamala, V. K. Jirsa, M. Ding. Enhancement of neural synchrony by time delay. Phys. Rev. Lett., 92, 074104(2004).

[26] S. Ghosh-Dastidar, H. Adeli. Improved spiking neural networks for EEG classification and epilepsy and seizure detection. Integr. Comput.-Aided Eng., 14, 187-212(2007).

[27] P. Adibi, M. R. Meybodi, R. Safabakhsh. Unsupervised learning of synaptic delays based on learning automata in an RBF-like network of spiking neurons for data clustering. Neurocomputing, 64, 335-357(2005).

[28] S. Ghosh-Dastidar, H. Adeli. A new supervised learning algorithm for multiple spiking neural networks with application in epilepsy and seizure detection. Neural Netw., 22, 1419-1431(2009).

[29] A. Taherkhani, A. Belatreche, Y. Li, L. P. Maguire. EDL: an extended delay learning based remote supervised method for spiking neurons. Neural Information Processing, 190-197(2015).

[30] A. Taherkhani, A. Belatreche, Y. Li, L. P. Maguire. DL-ReSuMe: a delay learning-based remote supervised method for spiking neurons. IEEE Trans. Neur. Netw. Learn. Syst., 26, 3137-3149(2015).

[31] A. Taherkhani, A. Belatreche, Y. Li, L. P. Maguire. A supervised learning algorithm for learning precise timing of multiple spikes in multilayer spiking neural networks. IEEE Trans. Neur. Netw. Learn. Syst., 29, 5394-5407(2018).

[32] M. L. Zhang, J. B. Wu, A. Belatreche, Z. H. Pan, X. R. Xie, Y. S. Chua. Supervised learning in spiking neural networks with synaptic delay-weight plasticity. Neurocomputing, 409, 103-118(2020).

[33] W. Xu, S. Y. Min, H. Hwang, T. W. Lee. Organic core-sheath nanowire artificial synapses with femtojoule energy consumption. Sci. Adv., 2, e1501326(2016).

[34] I. Sourikopoulos, S. Hedayat, C. Loyez, F. Danneville, V. Hoel, E. Mercier, A. Cappy. A 4-fJ/spike artificial neuron in 65 nm CMOS technology. Front. Neurosci., 11, 123(2017).

[35] J. D. Zhu, Y. C. Yang, R. D. Jia, Z. X. Liang, W. Zhu, Z. A. Rehman, L. Bao, X. X. Zhang, Y. M. Cai, L. Song, R. Huang. Ion gated synaptic transistors based on 2D van der Waals crystals with tunable diffusive dynamics. Adv. Mater., 30, 1800195(2018).

[36] B. Irem, L. G. Manuel, S. R. Nandakumar, M. Timoleon, P. Thomas, R. Bipin, L. Yusuf, S. Abu, E. Evangelos. Neuromorphic computing with multi-memristive synapses. Nat. Commun., 9, 2514(2018).

[37] Y. Zhou, N. Xu, B. Gao, Y. Y. Chen, B. Y. Dong, Y. Li, Y. H. He, X. S. Miao. Complementary graphene-ferroelectric transistors (C-GFTs) as synapses with modulatable plasticity for supervised learning. IEEE International Electron Devices Meeting (IEDM), 1-4(2019).

[38] S. G. Hu, G. C. Qiao, Y. A. Liu, L. M. Rong, Q. Yu, Y. Liu. An improved memristor model connecting plastic synapse synapses and nonlinear spiking neuron. J. Phys. D, 52, 275402(2019).

[39] J. Ohtsubo, R. Ozawa, M. Nanbu. Synchrony of small nonlinear networks in chaotic semiconductor lasers. Jpn. J. Appl. Phys., 54, 072702(2015).

[40] A. Hurtado, I. D. Henning, M. J. Adams. Optical neuron using polarization switching in a 1550 nm-VCSEL. Opt. Express, 18, 25170-25176(2010).

[41] A. Hurtado, K. Schires, I. Henning, M. Adams. Investigation of vertical cavity surface emitting laser dynamics for neuromorphic photonic systems. Appl. Phys. Lett., 100, 103703(2012).

[42] M. P. Fok, Y. Tian, D. Rosenbluth, P. R. Prucnal. Pulse lead/lag timing detection for adaptive feedback and control based on optical spike-timing-dependent plasticity. Opt. Lett., 38, 419-421(2013).

[43] Q. Li, Z. Wang, Y. Le, C. Sun, X. Song, C. Wu. Optical implementation of neural learning algorithms based on cross-gain modulation in a semiconductor optical amplifier. Proc. SPIE, 10019, 100190E(2016).

[44] S. Y. Xiang, Y. H. Zhang, X. X. Guo, J. F. Li, A. J. Wen, W. Pan, Y. Hao. Cascadable neuron-like spiking dynamics in coupled VCSELs subject to orthogonally polarized optical pulse injection. IEEE J. Sel. Top. Quantum Electron., 23, 1700207(2017).

[45] T. Deng, J. Robertson, A. Hurtado. Controlled propagation of spiking dynamics in vertical-cavity surface-emitting lasers: towards neuromorphic photonic networks. IEEE J. Sel. Top. Quantum Electron., 23, 1800408(2017).

[46] Z. G. Chen, R. Carlos, W. H. P. Pernice, C. D. Wrigh, H. Bhaskara. On-chip photonic synapse. Sci. Adv., 3, e1700160(2017).

[47] S. Y. Xiang, J. K. Gong, Y. H. Zhang, X. X. Guo, A. Wen, Y. Hao. Numerical implementation of wavelength-dependent photonic spike timing dependent plasticity based on VCSOA. IEEE J. Quantum Electron., 54, 8100107(2018).

[48] Y. H. Zhang, S. Y. Xiang, J. K. Gong, X. X. Guo, A. J. Wen, Y. Hao. Spike encoding and storage properties in mutually coupled vertical-cavity surface-emitting lasers subject to optical pulse injection. Appl. Opt., 57, 1731-1737(2018).

[49] I. Chakraborty, G. Saha, A. Sengupta, K. Roy. Toward fast neural computing using all-photonic phase change spiking neurons. Sci. Rep., 8, 12980(2018).

[50] I. Chakraborty, G. Saha, K. Roy. Photonic in-memory computing primitive for spiking neural networks using phase-change materials. Phys. Rev. Appl., 11, 014063(2019).

[51] Y. H. Zhang, S. Y. Xiang, X. Guo, A. Wen, Y. Hao. All-optical inhibitory dynamics in photonic neuron based on polarization mode competition in a VCSEL with an embedded saturable absorber. Opt. Lett., 44, 1548-1551(2019).

[52] J. Robertson, E. Wade, Y. Kopp, J. Bueno, A. Hurtado. Toward neuromorphic photonic networks of ultrafast spiking laser neurons. IEEE J. Sel. Top. Quantum Electron., 26, 7700715(2020).

[53] B. W. Ma, W. W. Zou. Demonstration of a distributed feedback laser diode working as a graded-potential-signaling photonic neuron and its application to neuromorphic information processing. Sci. China Inf. Sci., 63, 160408(2020).

[54] S. Y. Xiang, Y. H. Zhang, J. K. Gong, X. X. Guo, L. Lin, Y. Hao. STDP-based unsupervised spike pattern learning in a photonic spiking neural network with VCSELs and VCSOAs. IEEE J. Sel. Top. Quantum Electron., 25, 1700109(2019).

[55] J. Feldmann, N. Youngblood, C. D. Wright, H. Bhaskaran, W. H. P. Pernice. All-optical spiking neurosynaptic networks with self-learning capabilities. Nature, 569, 208-215(2019).

[56] C. Ríos, N. Youngblood, Z. Cheng, M. L. Gallo, W. H. P. Pernice, C. D. Wright, A. Sebastian, H. Bhaskaran. In-memory computing on a photonic platform. Sci. Adv., 5, eaau5759(2019).

[57] Z. W. Song, S. Y. Xiang, Z. X. Ren, G. Q. Han, Y. Hao. Spike sequence learning in a photonic spiking neural network consisting of VCSELs-SA with supervised training. IEEE J. Sel. Top. Quantum Electron., 26, 1700209(2020).

[58] S. Y. Xiang, Z. X. Ren, Y. H. Zhang, Z. W. Song, Y. Hao. All-optical neuromorphic XOR operation with inhibitory dynamics of a single photonic spiking neuron based on VCSEL-SA. Opt. Lett., 45, 1104-1107(2020).

[59] S. Y. Xiang, Z. X. Ren, Z. W. Song, Y. H. Zhang, X. X. Guo, G. Q. Han, Y. Hao. Computing primitive of fully-VCSELs-based all-optical spiking neural network for supervised learning and pattern classification. IEEE Trans. Neural Netw.(2020).

[60] S. Moallemi, R. Welker, J. Kitchen. Wide band programmable true time delay block for phased array antenna applications. IEEE Dallas Circuits and Systems Conference (DCAS), 1-4(2016).

[61] G. Wetzstein, A. Ozcan, S. Gigan, S. Fan, D. Englund, M. Soljačić, C. Denz, D. A. B. Miller, D. Psaltis. Inference in artificial intelligence with deep optics and photonics. Nature, 588, 39-47(2020).

[62] A. Roy, S. Govil, R. Miranda. An algorithm to generate radial basis function (RBF)-like nets for classification problems. Neural Netw., 8, 179-201(1995).

CLP Journals