In the past few decades, the computing capability of conventional digital computers based on complementary metal oxide semiconductor (CMOS) transistors has been increased greatly as predicted by Gordon Moore^[1]. With the rapid development of artificial intelligence (AI) and internet of things, an exponential growth in the amount of data has led to urgent requirements for high efficiency and ultralow power consumption for the data-centric applications. In a standard digital computer based on the von Neumann architecture, the memory and processor are physically separated. Significant data movement between memory and processor is costly in terms of time and energy in today’s von Neumann systems. Meanwhile, Moore’s law has been slowing down significantly in recent years^[2]. Thus, innovative non-von Neumann computational approaches are highly desired in the More-than-Moore era.

The human brain, which is believed to be the most complex intelligent system in the universe, exhibits ultralow power consumption, massive parallelism, robust fault tolerance, self-adaptation, and self-learning ability. The architecture of the human brain differs from that of a digital computer. Generally, a biological neural network is composed of roughly 10¹¹ neurons and 10¹⁵ synapses, and is normally represented by spiking neural network (SNN)^[3]. In the brain, the information is represented and transmitted by action potential, i.e., spike^{[4, 5]}. The spike signals transmitted between the neurons connected through synapses with synaptic plasticity. Spike timing-dependent plasticity (STDP) is one of the most widely studied synaptic plasticity mechanism^[6-8].

Inspired by the network architecture and principles of the human brain, the neuromorphic computing system has drawn tremendous attention in the next generation of computing technology. Nowadays, both the digital and analog hardware paradigms based on the CMOS technologies have been fully developed^{[9, 10]}. In addition, extensive efforts have also been made to mimic the functions of biological neurons and synapses by various electronic neuromorphic devices. In recent years, remarkable progress has been made from materials to devices, circuits, and architectures in the field of electronic neuromorphic computing^[9-12]. However, the operation speed of the electronic neuromorphic computing system is restricted due to the bandwidth–connection density trade-off^[13].

As a complementary approach, the photonics platform has gained increasing attention for hardware neuromorphic computing, due to the fascinating advantages such as high speed, wide bandwidth, and massive parallelism. The photonics neuromorphic computing shows great promise in the applications which require low latency, low power consumption, and high bandwidth. Nevertheless, the photonics neuromorphic computing is still in its infancy compared to the electronics counterpart. Nahmias et al. proposed a photonic leaky integrate-and-fire (LIF) neuron based on vertical-cavity surface-emitting lasers with saturable absorber (VCSELs-SA)^[14]. With the help of Yamada model, the neuron-like spiking dynamics was reproduced in the VCSEL-SA model. On the other hand, there are also some proposed photonics synapses. For instance, Gholipour et al. developed photonic synapses based on amorphous gallium lanthanum oxysulphide microfibers^[15]. Cheng et al. fabricated and demonstrated an on-chip photonic synapse via phase-change materials combined with integrated silicon nitride waveguide^[16]. Specifically, Feldmann et al. implemented an all-optical SNN with self-learning capacity based on a nanophotonic chip, and successfully demonstrated supervised and unsupervised learning in the optical domain^[17]. For more detail on the spiking neurons and synapses based on photonic devices, please refer to Refs. [13, 18–21].

Generally, the photonics neurons and synapses are studied separately with different devices, therefore leading to the need for further developing high-performance photonics neuron and synaptic devices. In addition, algorithm is another key issue that limits the progress of photonic SNN development.

Herein, we review some recent progress on the devices, architecture, and algorithm of photonic neural computing in our research groups. First, we introduce the photonic neuron at the device level. Then, we review the progress on the photonic STDP. Subsequently, we focus on several photonic neural networks at the system level. Finally, we summarize the challenges and opportunities faced by photonic neural computing, and propose promising solutions and perspectives.

Photonic devices operating in the excitable regime are dynamically analogous to the biological neurons exhibit spiking dynamics. While the operating speed of photonic devices are many orders of magnitude faster than their biological counterparts. There are various optical neurons reported experimentally and numerically in recent years^{[14, 22-39]}. Here, we focus on the lasers-based optical neuron. Some recent progress on the optical neurons based on conventional VCSEL, VCSEL-SA, as well as distributed feedback laser (DFB) are reviewed.

A VCSEL-SA can be employed as a photonics neuron because it possesses the excitability property behaves analogously to the LIF neuron model. The excitability properties and the spike latency of the VCSELs-SA are shown in Fig. 1^[39]. Fig. 1(a) shows that the VCSEL-SA could generate excitatory neuron-like spike and realize the temporal encoding. The spike latency, defined as the interval between the start time of stimulus pulse and the timing corresponding to the maximum value of spikes generated by VCSEL-SA, varied with different stimuli strengths and central timings. As presented in Fig. 1(b), when the stimuli pulse power exceeds a threshold value, the VCSEL-SA can emit a spike. In addition, such threshold behavior can be tuned by different bias current. Fig. 1(c) indicated that the spiking timing can be continuously adjusted by different stimulus strength.

Note, the inhibitory dynamics is also important for the neural information processing. We further revealed the inhibitory dynamics based on the polarization mode competition effect in a VCSEL-SA with two coexisting polarization-resolved modes^[33]. The schematic diagram of inhibitory photonic neuron based on VCSEL-SA is presented in Fig. 2(a). The inhibition behavior was characterized by the spike amplitude and first spike latency. As presented in Fig. 2(b), the neuron-like inhibition behavior could be achieved thanks to the polarization mode competition. Furthermore, as presented in Fig. 3, such inhibition dynamics could also be used to realize the spike-based XOR in a single step with a single VCSEL-SA^[34]. Additionally, the XOR operation was also achieved successfully with two pseudo-random return-to-zero sequences.

The VCSELs have become promising candidates for artificial neuronal models due to the polarization switching or optical injection induced nonlinear dynamics. Recently, the controllable and reproducible excitation and inhibition behaviors at sub-nanosecond speeds for a commercially available VCSEL subject to the successive external stimulus have been experimentally and theoretically demonstrated^{[22, 24-30, 36]}. Under an external injection characterized by a constant optical power level with sudden perturbation in the form of short power drops, the VCSEL-neuron can transmit from the injection-locking state into the unlocking state after the arrival of the perturbation due to the broken equilibrium and a spiking event is triggered. Moreover, the number of fired spikes can be controlled through adjusting the temporal duration of incoming perturbations, as shown in Figs. 4(b1)–4(b5)^{[26, 28]}. Also, under the external injection in the form of a constant level with sudden power raises for increasing perturbation duration, the spiking regime could be entirely suppressed during the perturbation period, as shown in Figs. 4(c1)–4(c5)^[30]. Moreover, the communication of excitatory and inhibitory spiking signals at sub-nanosecond speeds between two coupled VCSEL-neurons have also been demonstrated experimentally and numerically^{[26-28, 30]}.

The DFB exhibits similar processing characteristics with the graded-potential-signaling-based neuron observed in the nervous system, such as temporal integration and pulse facilitation. A commercially available DFB was demonstrated to play a role in three applications of neuromorphic information processing for pattern recognition, single-wavelength STDP implementation, and sound azimuth measurement, as shown in Fig. 5^[37]. To further investigate the spatiotemporal processing potential of DFB neurons, a network architecture was proposed to be equipped with N tunable weights on each input branch for complex pattern recognition^[38]. The schematic of the network is illustrated by Fig. 6(a). Successful pattern recognition was demonstrated among the input sets when N = 3 and 4, as given in Figs. 6(b) and 6(c), respectively. Additionally, the network can learn a target pattern by the assistance of the STDP learning module also realized by a DFB, which is convenient for the potential integration.

In brains, synaptic plasticity is believed to be closely related to the learning and memory. In this section, we review the emulation of the synaptic function based on the semiconductor optical amplifier (SOA) and vertical-cavity semiconductor optical amplifier (VCSOA). The photonic STDP is focused on.

STDP is a long-term synaptic plasticity observed experimentally in biological synapses by Bi and Poo^[6]. Thanks to the cross-gain modulation in SOA, the implementation of optical STDP scheme based on SOA was demonstrated^[40-43]. The result in Fig. 7 shows that the STDP curve is similar to that measured in biological experiments, but at a much faster time scale^[41]. Moreover, the height and width of the measured STDP learning window decrease as the SOA current increases.

Note that, for the photonic STDP based on SOA, the operation current of SOA is relatively high, i.e., several tens of or hundreds of mA. When operating below threshold, the VCSEL can also be regarded as VCSOA. The photonic STDP based on the VCSOA was proposed and demonstrated experimentally and numerically^[44]. The schematic diagram is illustrated in Fig. 8(a). In our experiment, the VCSOA was biased at 1.42 mA, the optical pulse power (width) was 70 μW (100 ps). The energy consumption for biasing the VCSOA and triggering the STDP function could be estimated as several femtojoules per spike, which is much lower than the microelectronic counterparts^{[45, 46]}. The experimental measurements are shown in Fig. 8(b). When two optical pulses with different time interval were injected into the VCSOA, the lagged pulse experienced different gain due to the carrier depletion caused by the leading pulses. With the increase of time interval, the output power of the lagged pulse increased gradually. The numerical results are presented in Figs. 8(c) and 8(d), which agree well with the experimental measurements. Fig. 8(e) shows that the STDP curve can be achieved in VCSOA with low bias current. The calculated STDP window was about 1 ns, indicating that the STDP operation rate was nearly 1 GHz, which is higher than the conventional electronics^{[45, 46]}. Furthermore, such photonic STDP curve can also be achieved in real-time with a single VCSEL with dual-polarized pulsed optical injection^[47].

In this section, we review the progress on the optical neural computing at the system level. We consider several network architectures and algorithms including photonic SNN, photonic convolutional neural network (CNN), photonic matrix computation, photonic reservoir computing (RC), and photonic reinforcement learning.

To design the algorithm for a photonic SNN, we proposed a novel framework of a fully VCSEL-based all-optical SNN and developed a self-consistent unified neuron-synapse-learning model that allows a complete learning-to-inference workflow^{[35, 39]}. The unsupervised learning was implemented in a photonic SNN consisting of VCSELs-SA^[35]. The input pattern is encoded into spikes at different timings by different VCSELs-SA. As shown in Fig. 9, the photonic SNN with photonic STDP enables the POST neuron to spike at the first spike timing of the input pattern in an unsupervised manner. In addition, supervised learning was also realized in a photonic SNN^{[39, 48]}. As illustrated in Fig. 10, a supervised spike sequence learning task was implemented in a two-layer fully-connected photonic SNN consisting of excitable VCSELs-SA^[48]. In this photonic SNN, the classical remote supervised method (ReSuMe) learning algorithm based on the photonic STDP is adopted to train the POST to fire the desired spike sequence. Fig. 10(c) illustrates the learning process in a typical run. The corresponding spike sequence distance (SSD) as a function of the learning epoch shown in Fig. 10(e) indicates that the learning process was convergent. Therefore, the photonic SNN successfully reproduced a desirable output spike sequence in response to a spatiotemporal input spike pattern by means of the iteration algorithm to update synaptic weights continuously.

The spatiotemporal pattern classification based on supervised learning was further demonstrated based on the spatiotemporal design of the photonic SNN shown in Fig. 11(a)^[39]. Optical character numbers were trained and tested by the proposed SNN. As shown in Fig. 11(b), the network was trained with a clean character image, and then, the inference was tested with a set of noisy patterns. The results show that this all-optical SNN can recognize ten numbers by a supervised learning algorithm. Besides, the training convergence can be optimized by using different bias current of VCSOA as presented in Fig. 11(c). As illustrated in Fig. 11(d), the accuracy rate of the trained network was robust to small noise strength of the optical digital character.

In addition, sound azimuth detection was emulated in a fully connected photonic SNN consisting of VCSELs-SA^[49]. Fig. 12(a) shows the schematic diagram of the proposed photonic SNN which composed of two presynaptic neurons (PREs) considered as the ear sensors, and two postsynaptic neurons (POSTs). The difference between the precise spike timing of two POSTs ( ) was used as an indication of the sound azimuth. The dependence of on the relative timing of the spikes of two PREs ( ) was revealed as illustrated in Fig. 12(b). Furthermore, the effective detection range were identified for different weights as shown in Fig. 12(c). The results demonstrated that the proposed photonic SNN used for sound azimuth detection was biologically plausible, and has higher resolution compared with the biological system.

Besides, the winner-take-all mechanism was also achieved successfully in the photonic SNN consisting of VCSELs-SA as shown in Figs. 13(a) and 13(b)^[50]. Two information processing tasks including pattern recognition and max-pooling operation were performed as shown in Figs. 13(c)–13(f). The results hold great promise for the development of energy-efficient and high-speed photonic SNN.

Furthermore, associative learning and forgetting processes were emulated in a photonic SNN^[51]. The schematic diagram of the photonic associative learning network is shown in Fig. 14(a). Fig. 14(b) shows that both the associative learning and forgetting processes could be achieved thanks to the photonic STDP rule. The pattern recall based on the associative learning was further demonstrated in the photonic SNN presented in Fig. 14(c). Complete pattern and incomplete pattern of number 8 are shown in Figs. 14(d1) and 14(d2), respectively. Fig. 14(d3) [Fig. 14(d4)] shows the initial output (final output) of number 8 before [after] associative learning process. The evolution of synapse weight corresponding to the pattern recall of number 8 is presented inFig. 14(e1). Without loss of generality, Figs. 14(d5, d6, d7, d8) and 14(e2) show the pattern recall process of number 5 and the corresponding weight evolution. Obviously, the incomplete pattern can be recovered and pattern recall was realized based on the photonic associative learning network.

The optical implementation of CNN with fast operation speed and high energy efficiency is appealing due to its outstanding feature extraction ability^[52-54]. The high-accuracy optical convolution unit (OCU) with cascaded acousto-optical modulator arrays is illustrated in Figs. 15(a)–15(c)^[52]. The input data and convolutional kernel were fed into the modulator arrays to carry out the operation. With the hardware reusing scheme, complicated CNNs can be conducted by the units. In Fig. 15(d), convolution results on the digital computer and the proposed OCU are shown to support the feasibility.

A more complete optical CNN implementation incorporating patching scheme was demonstrated in^[53]. In Figs. 16(a) and 16(b), the introduction of optical delay lines to execute data manipulations promises low latency and power consumption. Meanwhile, wavelength-division-demultiplexing (WDM) was used to improve the computational capacity by parallel wavelength channels. Illustrated by Figs. 16(c)–16(f), the serialized input waveforms of digit 2 and 4 were successful to form delayed copies as the experimental demonstration of optical patching. Consequently, the scale of input modulator arrays was largely cut down.

Matrix computations form the most widely used computational tools in science and engineering, and are the basic components of neural networks for deep learning. While the electronic matrix computations suffer from limited bandwidth. Alternatively, the optical methods offer a high-speed and low-loss solution. Optical matrix computation is also essential in the optical neural computing.

The Mech-Zehnder interferometer (MZI) mesh had been demonstrated for optical matrix computation^{[55, 56]}. As shown in Fig. 17(a), the optical matrix computing processor can perform fundamental matrix computations including XB = C, AB = X and AX = C, where A, B, C are known matrices, and X is the matrix to be solved. Fig. 17(b) shows an example to self-configure a transmission matrix. An optical PageRank algorithm was further demonstrated based on the matrix computing processor. Furthermore, the optical matrix computation core could be applied for polarization processing^{[57, 58]}. The micrograph of chip is presented in Fig. 17(a) and some experimental results are depicted in Fig. 18. The polarization processor could implement multiple polarization processing functions, including polarization multiple-input-multiple-output (MIMO) descrambler, polarization controller and polarization analyzer, which are the basic building blocks of polarization processing. More functions could be realized by using an additional two-dimensional output grating. A numerical gradient descent algorithm was employed to self-configure and self-optimize these functions. Recently, the 4-port general linear network has been applied for multi-channel optical switching, MIMO descramble and tunable filter by learning strategy^[59]. Fig. 19 shows the typical demonstration of the three functions respectively. The works suggested great potential for chip-scale reconfigurable and fully programmable photonic computing and optical signal processors with artificial intelligent algorithm.

RC is a brain-inspired computational paradigm originated from recurrent neural network suitable for time series processing^[60-62]. In the training process of RC systems, only the output weights are modified, while the input and reservoir weights are fixed randomly. Specifically, the time-delay RC system based on a single nonlinear node with delay feedback has been demonstrated in electronic, optoelectronic, and all optical delay systems^[63-65]. In recent years, we have also made some attempts to the time-delay RC via photonics approaches^[66-70].

An attempt at the time-delay RC systems based on VCSEL is that we proposed a four-channels RC system based on polarization dynamics in mutually coupled VCSELs^[67]. As shown in Fig. 20(a), four channels RC were realized in two orthogonal polarization modes of two VCSELs. The outputs obtained from the four channels were combined into one output for post-processing as displayed in Fig. 20(b). The four-channels RC could produce comparable prediction performance but at a faster information processing rate compared with the one-channel RC as shown in Fig. 20(c).

We also have some attempts at the time-delay RC system based on semiconductor nanolaser (SNL), due to the potential of realizing photonic integrated RC system^{[69, 70]}. For example, a high-speed neuromorphic SNL-based RC system under electrical modulation was proposed^[69]. The conceptual scheme of the SNL-based RC system is presented in Fig. 21(a). The numerical simulation results are shown in Fig. 21(b), which show that a larger Purcell factor F and larger spontaneous emission coupling factor β could extend the range of high prediction performance of SNL-based RC system.

Reinforcement learning is also a fundamental learning mechanism inspired by the brain^[71]. In reinforcement learning, an agent learns from interaction process with the environment, aims to maximize the benefits via certain learning strategies. Decision-making is a basic component of reinforcement learning, which requires the agent to make decision quickly and accurately in an uncertain and dynamically changing environment^[72]. Photonic decision making provides possibilities in applications of ultrafast processing.

The photonic reinforcement learning based on laser chaos with time-delay signature concealment was demonstrated by introducing a phase-modulated Sagnac loop in mutually delay-coupled semiconductor lasers (PMSL-MC)^[73], as shown in Fig. 22. The multi-armed bandit problem was solved in parallel with the utilization of dual-channel chaotic signals. The comparison between the PMSL-MC system and conventional mutually-coupled semiconductor lasers system (CSL-MC) further demonstrated that the system with dual-channels chaotic signals can make decision in parallel and converge faster.

A further parallelized scheme for photonic decision making was experimentally demonstrated in a globally-coupled chaotic semiconductor lasers network as shown in Fig. 23^[74]. Triple-channel chaotic signals were applied to solve an 8-armed bandit problem with a parallel architecture given in Fig. 23(b). In Figs. 23(c) and 23(d), the results suggested that the chaotic signals with better time delay signature concealment generally contributed to better decision-making performance. The adaptability of the strategy to environmental change was further demonstrated as in Fig. 23(e). Moreover, such a scheme was also scalable as demonstrated in Fig. 23(f), where both 8-armed and 16-armed bandit problems were solved successfully.

We have reviewed some representative photonic neural computing in devices, architectures, and algorithms. To further pave the way of photonics neuromorphic computing, there are still some problems that need to be addressed. The optical neuron and synapse are generally designed separately, which results in the different time scales. The optical neuron and synapse with similar time scale have to be further developed to meet up the requirements of on-line training. The design of photonic neural computing system on a chip still requires further exploration. Further attempts have to be made to reduce the area and increase the integration level.

Note, at the present stage, we think it may be more realistic to adopt an ex-situ training approach for the training process of the photonic neural network. Nevertheless, it is promising to build an integrated hardware photonic neural network for realizing the inference process. At present, the optical neuron based on spiking laser could be readily realized with III–V compound semiconductor technology such as indium phosphide (InP) and gallium arsenide (GaAs), while the weight array could be successfully implemented with silicon waveguides or resonators. It is believed that the rapid development of photonic integrated technologies will lead to a bright future for the field of photonic neural computing. For instance, a hybrid III–V and silicon photonics platform may be a candidate to realize an integrated hardware photonic neural network for inference task^[75-84], in which spiking lasers array in a bonded InP layer could be interconnected via a silicon layer^{[13, 78]}.

This work was supported in part by the National Outstanding Youth Science Fund Project of National Natural Science Foundation of China (62022062), by the National Natural Science Foundation of China (61974177, 61674119), by the Fundamental Research Funds for the Central Universities.