Artificial neural networks excel in learning from data to perform classification, regression, and prediction tasks of vital importance [1]. As data information content increases, simulating these models becomes computationally expensive on conventional computers, making specialized signal processors crucial for intelligent systems. Training large networks is also costly in terms of energy consumption and carbon footprint [2]. Photonics provides a valuable alternative toward sustainable computing technologies. For this reason, machine learning through photonic components is gaining enormous interest [3]. In fact, many mathematical functions, which enable complex feature extraction from data, find a native implementation on optical platforms. Pioneering attempts using photosensitive masks [4] and volume holograms [5,6] have been recently developed into coherent optical neural networks that promise fast and energy-efficient optical computing [7–16]. These schemes exploit optical units such as nanophotonic circuits [7], on-chip frequency combs [14–16], and spatial light modulators to perform matrix multiplications in parallel [17] or to carry out convolutional layers [18–20]. Training consists in adjusting the optical response of each physical node [21], also by adopting external optical signals [22], which is demanding [23]. Moreover, photonic neural networks based on nanofabrication still have a considerable energy impact. A general and promising idea to overcome the issue is to adapt machine-learning paradigms and make them inclined to optical platforms. In this paper, we pursue this approach by constructing an easy-to-train optical architecture that requires only free-space optical propagation.

Photonic architectures that do not need control of the entire network are particularly attractive. A remarkable method for their design and training is reservoir computing [24–26], in which the nonlinear dynamics of a recurrent system processes data, and training occurs only on the readout layer. Optical reservoir computing has demonstrated striking performance on dynamical series prediction by using delay-based setups [27–30], laser networks [31], multimode waveguides [32], and random media [33]. Since several interesting complex systems can be exploited as physical reservoirs, the inverse approach is also appealing, i.e., the scheme can be trained for learning dynamical properties of the reservoir itself [34].

Extreme learning machines (ELMs) [35,36], or closely related schemes based on random neural networks [37,38], support vector machines [39] and kernel methods [40], are a powerful paradigm in which only a subset of connections is trained. ELMs perform comparably with basic deep and recurrent neural networks on the majority of learning tasks [36]. The basic mechanism is to use the network to establish a nonlinear mapping between the data set and a high-dimensional feature space, where a properly trained classifier performs the separation. Training occurs at the readout layer; however, at variance with reservoir computing, ELMs have no recurrences in the connection topology and do not possess dynamical memory. In optics, an interesting case of this general approach has been implemented by using speckle patterns emerging from multiple scattering [41] and multimode fibers [42] as a feature mapping. Although in principle many optical phenomena can form the building block of the architecture [43,44], the general potential of the ELM framework for photonic computing remains largely unexplored.

Here, we propose a photonic extreme learning machine (PELM) for performing classification and regression tasks using coherent light. We find that a minimal optical setting composed by an input element, which encodes embedded input data into the optical field, a wave layer, corresponding to propagation in free space, and a nonlinear detector, enables simple and efficient learning. The encoding method plays a crucial role in determining the learning ability of the machine. The architecture is experimentally implemented via phase modulation of a visible laser beam by a spatial light modulator (SLM). We benchmark the realized device on problems of different classes, achieving performance comparable with digital ELMs. These include a classification accuracy exceeding 92% on the well-known MNIST database, which overcomes fabricated diffractive neural networks [8]; further, it is comparable with convolutional artificial networks that employ photonic accelerators [15,16,19]. Given the massive parallelism provided by spatial optics and the ease of training, our approach is ideal for big data, i.e., extensive data sets with large dimension samples. Our PELM is intrinsically stable and adaptable over time, as it does not require engineered or sensitive components. It can potentially operate in dynamic environments as an intelligent device performing on-the-fly optical signal processing.

An ELM is a feed-forward neural network in which a set of input signals is connected to at least one output node by a large structure of generalized artificial neurons [35]. An outline of the architecture is illustrated in Fig. 1(a). The hidden neurons form the computing reservoir. Unlike for neurons in a deep neural network, they do not require any training, and their individual responses can be unknown to the user [36]. Given an input data set with $N$ samples $\mathbf{X}=[{\mathbf{X}}_1,\dots ,{\mathbf{X}}_N]$, the reservoir furnishes an hidden-layer output matrix $\mathbf{H}=[\mathbf{g}({\mathbf{X}}_1),\dots ,\mathbf{g}({\mathbf{X}}_N)]$, where $\mathbf{g}(\mathbf{x})$ is a nonlinear feature mapping, $\mathbf{H}$ is linearly combined with a set of weights $\beta$ to give the output $\mathbf{Y}=\mathbf{Y}(\mathbf{H};\mathbf{\beta })$ performing the classification, and $\mathbf{Y}=\mathbf{H}\mathbf{\beta }$. To train an ELM classifier, the optimal output weights ${\beta }_i$ are the sole parameters that need to be determined, a problem that can be solved via ridge regression. Details on this technique are reported in Appendix A.

We transfer the ELM principle into the optical domain by considering the three-layer structure illustrated in Fig. 1(b). In the encoding layer, the input vectors ${\mathbf{X}}_i$ are embedded into the phase and/or amplitude of the field by an optical modulator. The reservoir consists of linear optical propagation and nonlinear detection of the final state. The output is recovered in the readout layer, where weights ${\beta }_i$ are applied to $M$ measured channels. The $\beta$ set is trained by solving the regression problem via digital hardware. For an extensive training set, with size larger than the number of channels ($N\gg M$), an effective solution reads as [36] $$\mathbf{\beta }={({\mathbf{H}}^{\mathrm{T}}\mathbf{H}+c\mathbf{I})}^{-1}{\mathbf{H}}^{\mathrm{T}}\mathbf{T},$$(1)where $\mathbf{T}$ indicates the training targets, $\mathbf{I}$ the identity matrix, and $c$ is a regularizing constant. $\mathbf{H}$ contains projections of input samples in the feature space, where data are separated according to their features. This mapping is the key ingredient for learning and occurs entirely by means of optical propagation. Mapping functions can be constructed starting from the elementary components of the optical setup. For a general PELM based on the scheme in Fig. 1(b), we construct the feature mapping $$H_{ji}={\mathbf{g}}_i({\mathbf{X}}_j)=G{[\mathbf{M}·p({\mathbf{X}}_j)·q(\mathbf{W})]}_i,$$(2)which describes linear propagation of input optical data and their nonlinear detection. In Eq. (2), $G$ is a detection function, $\mathbf{M}$ is the complex transfer matrix modelling field transmission, $p$ and $q$ are two encoding functions, and $\mathbf{W}$ is a fixed character of the encoder that we term embedding matrix. $\mathbf{W}$ has no direct equivalent in the basic ELM model and serves taking into account that any laser beam has spatial amplitude modulations and phase inhomogeneities. The elements of this matrix can be understood as biases or as entries of a fan-in matrix connecting the input signal to the reservoir. We remark that, at variance with the ELM model, in the optical architecture, input and output data, network connections, and processing functions are all complex quantities. This fact can add an important degree of freedom to the network operation [45].

In the PELM architecture, each operation is defined by the corresponding optical elements. Therefore, Eq. (2) can represent the feature space of different optical settings. For instance, in the optical classifier demonstrated by Saade et al. [41], in which amplitude modulated data propagate through a scattering medium, $p({\mathbf{X}}_j)={\mathbf{X}}_j$ (amplitude encoding) and $\mathbf{M}$ is a random complex Gaussian matrix. We validate and experimentally realize a three-component system composed by a phase-only SLM, free space, and a camera. In our case, Eq. (2) can thus be specified as follows. Phase encoding of the input data by spatial light modulation corresponds to $p(\mathbf{x})=\exp (i\mathbf{x})$, and, since a single encoder is employed, $p=q$, and $\mathbf{H}=G[\mathbf{M}·\exp i(\mathbf{X}+\mathbf{W})]$. The nonlinear function $G$ models the detection of the transmitted field. Using the saturation effect of the camera pixels, we have $G(I)\simeq I/(I+I_s)$, with $I={|A|}^2$, optical amplitude $A$, and saturation intensity $I_s$. For free-space optical propagation, i.e., the light distribution in the far field or in the focal plane of a lens, $\mathbf{M}$ corresponds to the discrete Fourier transform [46].

We first validate the free-space PELM architecture and assess the condition for learning via numerical simulations. We consider digit recognition and train our classifier on the MNIST handwritten digit database. Figure 2 reports the learning properties for two representative phase-encoding methods: (i) noise embedding and (ii) Fourier embedding. In (i), $\mathbf{W}$ is a disordered matrix modelling a distortion on the encoder (see Appendix A for details). It remains unchanged during both training and testing. Each input is encoded by phase modulation in $[0,\pi ]$, and the embedding signal is encoded in the same phase interval. The effect of the noise embedding is illustrated in Fig. 2(a), where a typical digit is shown as a phase mask. Figure 2(b) shows the classification performance of the PELM with $M=1600$ channels when varying the mean noise amplitude. The machine always reaches accuracies close to that allowed by training. The classification error shows a sharp decrease as noise increases from zero, and it converges to a plateau already for small perturbations. Results varying the noise correlation length at fixed noise level are in Fig. 2(c). This behavior indicates that optimal learning occurs for small-scale noise, i.e., when $\mathbf{W}$ has its maximum information content (rank $L$). The relevance of $\mathbf{W}$ for learning extends to embedding matrices without randomness. In (ii), $\mathbf{W}$ is a modulated carrier signal [Fig. 2(d)]. We are superimposing the input data on a carrier pulse; feature mapping corresponds to analyzing the intensity of the entire spectrum after a nonlinear amplifier. In this case, the learning process can be interpretable: the learned features point out resonances between the input signal and the carrier pulse. As shown in Fig. 2(e), a few frequencies in the carrier are sufficient for the learning transition. The PELM is much more accurate in the classification as the frequency content of the embedding signal is larger. These results reveal that the embedding matrix plays a key role in enabling data-set learning. Although, in our experiments, we control this matrix via the encoder, we note that it can be intrinsic in any optical setup, e.g., a distortion of the optical wavefront.

Another key hyperparameter of the scheme is the number of features (channels) $M$, which sets the dimensionality of the feature (measurement) space. Figure 2(f) reports the testing error with varying $M$. The performance rapidly increases with the number of features, whereas only hundreds of channels are necessary for a proficient PELM. For $M\gtrsim N/20$, the error approaches 2%, a value very close to the optimal accuracy achievable with learning machines based on the same paradigm [47]. In these digital machines, the main computational cost both for training and performing the classification is imputable to large matrix multiplications and their processing with nonlinear functions. In the optical device we carry out, these operations occur simply by free-space light propagation and detection. The only calculation we kept on digital hardware is the offline training via simple regression.

The experimental setup for the free-space photonic extreme learning machine is illustrated in Fig. 3(a) and detailed in Appendix B. A phase-only SLM encodes both the input data set and the embedding matrix on the phase spatial profile of the laser beam. In practice, distinct attributes of a data set are arranged in separate blocks, a data sample corresponds to a set of phase values on these blocks, and a preselected phase mask is superimposed [inset in Fig. 3(a)]. Phase information self-mixes during light propagation, a linear process that corresponds to the matrix multiplication in Eq. (2). The far field is detected on a camera that performs the nonlinear activation function. As shown in Fig. 3(b), the resulting acquired image is a saturated function of the optical amplitude in the lens focal plane. From this data matrix, we extract the $M$ channels to construct the PELM feature space. An example of an input data measured on a feature space is reported in Fig. 3(c). In analogy with the numerical implementation, the device is trained by finding the weights ${\beta }_i$ that, when applied the optical output, allow performing the classification.

We test the optical device on various computing tasks and data sets. The aim is to point out that, when the PELM is fully effective on a given task, it can be also easily and rapidly retrained for a different application. The main results are summarized in Fig. 4 and demonstrate the learning capability of the optical device. We first use the MNIST data set to prove classification on a large-scale multiclass problem. When trained using $M=4096$ output channels, the PELM reaches a mean classification accuracy of 92% $(\pm 0.005)$ over $N_t=10\text{,}000$ test images [Figs. 4(a) and 4(b)]. We obtain accuracy that exceeds recent optical convolutional processors [15] and is comparable with optical deep neural networks [8], but without the fabrication of specific optical layers and the heavy training they require. The best classification performance is not sensitive to the embedding matrix employed on the encoder, which experimentally confirms our numerical findings (Fig. 2). Figure 4(b) shows a confusion matrix measured using the random embedding method, which gives 92.18% accuracy, compared with the 92.06% [Fig. 4(a)] when using Fourier embedding. Our free-space PELM hence surpasses diffractive neural networks [8] and competes with cutting-edge photonic hardware in terms of accuracy. In particular, convolutional opto-electronic setups also achieve a coarse accuracy of 92% [19]. Ultrafast photonic processors reach 95% precision with the help of electronic layers [15], whereas similar photonic accelerators operate with 88% accuracy [16].

By only updating the input training set and the output weights, the PELM can be quickly reconfigured for classifying different objects. We consider a binary class problem, i.e., the mushroom task, where the goal is to separate poisonous from edible mushrooms from a series of features (see Appendix C for details). Figure 4(c) shows that, on this data set, we achieve 95.4% accuracy, using $N_t=4124$ test samples and $N=4000$ training points, which is close to the precision of digital ELMs [36]. The excellent properties of the experimental device as a classifier generalizes to regression problems. We test the abalone data set (see Appendix C), a task where the input data ciphers a sea snail, and the output is expected to furnish its age. It is a solid benchmark since it has low-dimensional inputs ($L=8$) but contains extreme events that are difficult to predict. Figure 4(d) shows experimental abalone predictions obtained via a feature space of dimension $M=2000$. The testing error measured as normalized root mean square displacement (NRMSD) is $\approx 0.12$, i.e., regression is performed with remarkable accuracy by using only linear optical propagation [44].

A key point of the PELM architecture is a testing error that rapidly converges to its optimal value as the feature space dimension increases. Experimental results demonstrating this property are shown in Figs. 4(e) and 4(f), respectively, for the MNIST and abalone data set. Good classification/regression performance is maintained even for a very small number of trained channels, if compared with the data set size. For instance, useful abalone predictions can be obtained with only $M=128$ channels, i.e., with a training process that consists only in inverting a modest matrix ($128\times 128$ elements). Heuristically, we find that the accuracy reaches a limiting value as $M\sim N/20$, in agreement with numerical simulation for the same machine hyperparameters. The same behavior is found for training errors, which indicates that input data are already completely separated by the optical setup.

It is worth noting that the device does not need training at each use. Once we learn a given data set, we store the corresponding output weights and program a different problem; then, we are able to accomplish any of the posed tasks on-demand. The diverse classification and regression instances in Fig. 4 can be tested without any sequential order, simply specifying the type of task they belong to.

The limit provided by training errors in Figs. 4(e) and 4(f) allows us to identify the various factors toward improving the computational resolution of the optical machine. We ascribe residual errors to practical nonidealities of the device. The finite precision of both encoding and readout, including noise in optical intensities, introduces errors that are absent in the digital implementation [48]. Considering a camera precision of 8 bit in the numerical model, we find a relevant decrease of the maximum accuracy. This indicates that improved performance is attainable by fine-tuning the optical components. Flickering effects of the phase modulator, which give a tiny but variable inaccuracy each time the machine is interrogated, are identified as another relevant source of error. Moreover, optical and mechanical fluctuations of the tabletop optical line during training can result in further inconsistencies. These fluctuations can be compensated in future experiments using incremental learning [36], a refined technique that allows us to adaptively adjust the weights while training is ongoing. Training by use of sparse regression methods may be also useful when dealing with larger data sets.

The learning principle of the PELM lies on the basis of various kernel methods [49]. In kernel classification, mapping of the input data to the high-dimensional feature space is not explicit, and correlations between samples are evaluated through a kernel function over the input space. Such a kernel $\mathbf{K}$ contains the inner product of all pairs of data points in the feature space, i.e., all the information sufficient to perform data separation. When trained via ridge regression, the output $\mathbf{Y}$ of a kernel classifier can be generally expressed as $$\mathbf{Y}=\mathbf{K}{(\mathbf{K}+c\mathbf{I})}^{-1}\mathbf{T},$$(3)where $\mathbf{K}=k({\mathbf{X}}_i,{\mathbf{X}}_j)$ is an $N\times N$ kernel matrix, with $k$ being a given kernel function. Comparing Eqs. (3) and (1) indicates that, in our PELM scheme, the free-space optical setup acts as an effective photonic kernel [41]. To provide a comparison with standard digital kernels, we have evaluated their performance on the MNIST data set. We exploit the modularity of the approach, i.e., we use the same training algorithm [Eq. (3)] with different kernel functions. We focus on the relevant case of the element-wise Gaussian kernel $k({\mathbf{X}}_i,{\mathbf{X}}_j)=\exp (-\gamma \Vert {\mathbf{X}}_i-{\mathbf{X}}_j{\Vert }^2$), with tunable parameter $\gamma$. Using this kernel, we obtain an average classification accuracy of 96%, to be compared with the 85% given by a linear kernel ($\mathbf{K}=\mathbf{X}{\mathbf{X}}^{\mathrm{T}}$). Smaller classification errors can be obtained with more elaborate kernel functions [41]. Although the evaluation of the kernel matrix elements can be time-efficient, training requires storing and inverting this large matrix ($N\times N$), which becomes unfeasible on large-scale data sets. This enormous memory consumption represents a major drawback, which makes digital kernels energetically inefficient with respect to the photonic implementation. In fact, in the PELM scheme, we make use of explicit feature mapping, and the output matrix used for training has a much smaller size ($M\times M$). This implies that the computational cost for training the device does not depend on the data set size $N$ but only on the selected number of channels $M$. The advantage of the proposed approach extends also to kernel methods based on different algorithms. For instance, a support-vector machine on MNIST takes about ${10}^7$ multiply-adds (MACs) per recognition [50], whereas in our PELM the optical processors enables adequate classification of the single handwritten digit using only ${10}^4$ digital MAC operations.

Besides computing effectiveness comparable with its digital counterpart, the PELM hardware offers several advantages that are promising for fast processing of big data, especially for inputs that naturally present themselves as optical signals. In fact, unlike deep optical neural networks [51,52], training is simple and scales favorably with the data set size. It can also be performed online with affordable computational resources. Once trained, forward information propagates optically in a fully passive way and in free space. This can provide a key advantage with respect to ELM algorithms and kernel methods both in terms of scalability and latency. In fact, the matrix multiplication in Eq. (2), which maps input data to the feature space, on a digital computer requires a time and memory consumption that grows quadratically with the data dimension $L$. The optical hardware performs this operation totally in parallel, independently of the input data size, and without power dissipation. Since only $M$ scalar multiplications are necessary from the optical detection to the final classification, the PELM has a low latency, which is independent on the data-set dimension. This is a crucial property in applications that require fast responses, such as in real-time computer vision. The main speed limitation of our free-space PELM is related to the frame rate of the input optical modulator. Liquid-crystal SLMs currently have typical frame rates of the order of 100 Hz, but phase modulators based on micro-electro-mechanical, electro-optical, or acousto-optical technologies are approaching GHz frequencies [53,54]. With similar components available, PELM hardware could perform $R=L\times M\times {10}^9$ operations per second (OPS), a value that, for large $L$, can overcome the current limits of electronic computing (peta OPS).

The absence of any optical element or medium along the network path, which differentiates our scheme from all other optical neuromorphic devices previously realized, is a valuable aspect for various reasons. The tiny optical absorption and scattering of light in air imply that our scheme needs low-power optical sources ($\mathrm{\mu W}$ or lower), i.e., it requires minimal energy consumption. More importantly, our device is extremely stable to mechanical and optical perturbations, because its operation does not depend on components that must be kept completely static after training. This suggests that the proposed optical processor is promising for use in dynamic environments. In fact, tolerance to external disturbances requires only the relative alignment between the SLM plane and the camera plane, which can be carried out with mechanical or optical stabilization. We thus expect the free-space PELM can operate even when being subject to shocks. For small perturbations, tolerance is already guaranteed by the finite spatial extent of the input and output modes. Specifically, the output channels are made by averaging over several camera pixels (see Appendix B), which reduces the effect of vibrations. Therefore, we expect that re-training the device would be necessary only when the perturbation is large enough that a change of the channel positions cannot be retrieved via some feedback mechanism.

In view of edge-computing applications, the PELM can be further adapted toward ease of use and energy efficiency. In fact, using optical signals from the environment as input data, the encoding operation performed by the SLM is reduced to the sole embedding. This operation can be integrated directly into optical propagation, removing the need for a phase modulator. For example, we can divide the coherent input field and recombine it after a digital micromirror device (DMD) has performed the embedding. The intensity extracted from a selected plane along the optical path gives the output matrix, which can be enriched with phase and spectral information. Similar schemes promise a miniaturization of our free-space device and its operation as a fast and stable optical processor.

Nowadays, artificial intelligence is permeating the field of photonics [55,56] and viceversa [57]. Optical platforms are enabling computing functionalities with unique features, ranging from photonic Ising machines for complex optimization problems [58–65] to optical devices for cryptography [66]. We have realized a novel photonic neuromorphic computing device that is able to perform classification and feature extraction only by exploiting optical propagation in free space. Our results demonstrate that fabricated optical networks, or complex physical processes and materials, are not mandatory ingredients for performing effective machine learning on a optical setup. All the essential factors for learning can be included in optical propagation via encoding and decoding methods. On this principle, we demonstrated a photonic extreme-learning machine that, given its unique adaptability and intrinsic stability, is attractive for future processing of optical data in real-time and dynamic conditions. More generally, our approach envisions the exceptional possibility of harnessing any wave system as an intelligent device that learns from data, in photonics, and far beyond.

Acknowledgment. We thank MD Deen Islam and V. H. Santos for technical support in the laboratory.