Optical neuromorphic computing offers an alternative approach to realize artificial neural computing. Compared to digital neural computing, it has several potential advantages such as ultra-fast speed and ultra-low energy consumption. In this paper, we introduce a platform to realize artificial neural computing based on metasurfaces (Zhicheng Wu, Ming Zhou, Erfan Khoram, Boyuan Liu, Zongfu Yu. Neuromorphic metasurface[J]. Photonics Research, 2020, 8(1): 01000046). Metasurfaces were developed to perform arbitrary wavefront engineering. Their optical functions are realized by the resonant scattering of nanoscale scatterers. And the scatterers are fabricated on a flat surface. Besides, the metasurface is also compatible with today’s nanofabrication and can be mass-produced at a low cost. Based on the property of metasurface, here, we use it to perform neural computing. In the following, we describe the design procedures and show it utility by the recognition of handwritten digits.
The concept is illustrated in Fig. 1. An object, such as a handwritten digit, is illuminated by a plane wave. The scattered light is then processed by a multi-layer neuromorphic metasurface. Each layer consists of arrays of scatterers. By changing the size of the scatterers, we can control the amplitude and the phase of the scattered light as shown in Fig. 1. The array of scatterers leads to strong interference when the light passes through the metasurface. With a few layers of metasurfaces, the output light becomes a focused beam and is directed toward a specific spatial location, which corresponds to the value of the handwritten digit.
Fig. 1. (a) Schematic of the neuromorphic metasurface. The neuromorphic metasurface consists of multiple layers of nanostructures, which are composed of an array of scatterers on top of a dielectric substrate. A handwritten digit is illuminated by a plane wave and the scattered light then is processed by the neuromorphic metasurface. By changing the sizes of scatterers, the phase and amplitude of the transmitted light after each layer can be modified. After multiple layers, the transmitted field can be focused on specific photodetectors, which are labeled by the values of the handwritten digits, i.e. 0 to 9. (b) Intensity distribution of the transmitted light after each layer in a 3-layer neuromorphic metasurface. Handwritten digits of 7 and 2 with different writing styles are used as examples. Despite the different writing styles, the transmitted light is always focused on the spot corresponding to the value of the handwritten digit. Here, we normalize the intensity of the transmitted light after each layer to its maximum for clarity.
Here we use two-dimensional metasurfaces to illustrate the design process, which means the input light is vectorized as a 1D array and nanoribbons are chosen as scattering elements. The widths of the nanoribbons are the trainable parameters. They can be learned through a training process similar to training a neural network. First, we build a forward propagation model based on wave optics. To obtain the electromagnetic properties of the metasurface without losing speed, we use locally periodic approximation. It assumes the geometry of ribbons in a small region doesn't change too much so that the metasurface can be treated as locally periodic. Under this assumption, the amplitude and phase of the field right after a scattering element can be calculated by a full-wave simulation assuming a periodic boundary condition. After getting the local field for each element, we can assemble the field along the plane right after the metasurface. Then we use near-to-far field transformation to calculate far-field distribution. Using the far-field distribution of one layer as an input of the following layer, we can cascade the multi-layer metasurfaces. At this point, with a given input field, the output of the multi-layer metasurface can be calculated using the forward propagation model.
Gaussian distribution is chosen as the training target. The peak of the distribution is the locations where we would like to make the output light to focus. Stochastic gradient optimization is used to minimize the difference between the metasurface output and the target output. Ribbon widths are updated towards the opposite direction of the gradient and modified by the learning rate. The gradient of the loss with respect to the width of a ribbon at a certain layer is calculated following the chain rule.
The neuromorphic metasurface starts to show its utility even with just two layers of metasurfaces. It can achieve 80% accuracy for MNIST classification, which means that 8 out 10 times, this double-layer metasurface can focus the light on the right location based on the meaning of the handwritten digit. It is a remarkable focusing effect compared with traditional metasurfaces that focus all light to a single spot. The accuracy reaches 90% when 6 layers are used.
Considering its advantages such as ultra-fast speed, ultra-low energy consumption, and easy fabrication, neuromorphic metasurface has promising application value in the future. Besides, there are also many works need to be done. For example, here we did not use nonlinear activation. In this simple recognition task, nonlinear activation does not significantly enhance performance. But nonlinear activation is crucial for more complex tasks such as face recognition. So, nonlinear materials such as a layer of saturable absorber should be considered to add into multi-layer metasurfaces.
About the Authors
Zongfu Yu is an associate professor of Dugald C. Jackson and Vilas Associate at Madison. Graduate students Zhicheng Wu, Ming Zhou, Erfan Khoram, and Boyuan Liu contributed to this research. The DARPA Young Faculty Award program supported the study.
Photonics Lab, led by Zongfu Yu, not only focuses on basic scientific research on light, but also focuses on combining cutting-edge nanoscience and artificial intelligence to solve challenging application problems. The research team homepage: https://photonics.engr.wisc.edu/.
