Chirality is one of important properties of subjects^{1, 2}. Especially in the aspects of medicine and biology, it is very normal that the opposite handedness of enantiomers leads to entirely different pharmacological effects. How to discriminate chiral signals attracts wide attention in recent years, but most of chiral biomolecules present weak optical signals in ultraviolet band, which increases difficulties for detection. On the other hand, it is reported that surface plasmons enhance circular dichroism (CD)^3-6, and different kinds of plasmonic metamaterial exhibit a giant nonlinear optical activity (NOA)^7-11. They possess significant optical chirality around the resonant frequencies in the visible and near-infrared band. Therefore, chiral metasurfaces^12-19, metamaterials^19-22 and nanoparticles^23-26 are widely used in chiral detection. The nearfield interaction between chiral molecules and metal nanostructures causes a far-field spectrum shift which can be considered as the basis of chiral discrimination²⁷. Complex nanostructures usually possess intrinsic chirality, and the background CD signals of nanostructures cover the interaction effects from biomolecules. Emphasizing the CD signals of biomolecules depends on shapes and optical properties of the nanostructure. It is significant to design particular structures for accurate chiral detection. Diverse types of biomolecules require different nanostructures, but the relationships between biomolecules and corresponding nanostructures are very intricate. Finding appropriate nanostructures demands extensive theoretical simulation and explorative experiments.

In recent decades, it is found that various artificial nanostructures achieve the manipulation of light based on their unique geometric distributions. However, the structure design process consumes huge amounts of computing in iterative electromagnetic simulations²⁸. Advanced optimization algorithms rather than simple parameter sweep methods are involved in the nanophotonic design to tackle these challenges. For example, genetic algorithm and topology optimization succeed in the design of metasurface and obtain exceptional optical features^29-32. As the development of artificial intelligence, the application of intelligent algorithms has been an active topic for nanophotonics^{28, 33-35}. Due to their huge advantages on large-scale data processing, intelligent algorithms deal with the complex problem which traditional optimization algorithms cannot solve. They contribute to the design of diverse functional nanostructures including meta-lens³⁶, meta-grating^37-39, coupler^40-41, beam splitters^42-43 and photonic crystals⁴⁴, etc. As one of the most popular artificial intelligence algorithms, deep learning models successfully decrease the cost of nanostructure design, enlarge the computable parameter space and help search for better nanostructures than classic design methods provide. However, deep learning models are usually limited by the small volume and inappropriate distribution of raw dataset²⁸.

In this work, we present a method of nanostructure design based on reinforcement learning, which changes data generation manners and replaces data fitting with explorations of parameters. The algorithm proposes automatically the nanostructures that it wants, and learns from the spectra of these nanostructures. Artificial neural networks (ANNs) guide searching for potentially better nanostructures which are about to be calculated by electromagnetic simulations, and these results are used to update the ANNs. With this algorithm, we obtain numerous different nanostructures with a sharp peak in CD spectra in order to emphasize the interaction effects between chiral biomolecules and plasmonic. The experiment proves that the microfluidic chips with the optimized chiral structures succeed in differentiation of enantiomers. The resonance wavelength shifts between enantiomers of glucose with opposite chirality reach 7 nm. According to the cathodoluminescence (CL) spectroscopy, the structures designed by artificial intelligence possess different near-field modes for RCP and LCP components.

The circular dichroism (CD) spectrum of a typical chiral reflective metasurface illustrates the difference in reflectivity (transmissivity) between left-circularly polarized light (LCP) and right-circularly polarized light (RCP), and it is defined as

CD=RRCP−RLCPRRCP+RLCP,

where R_RCP and R_LCP denote the reflectivity of the metasurface with a right-circularly polarized input and a left-circularly polarized input.

The metasurface whose CD spectrum presents a sharp peak emphasizes the phenomenon of the frequency shift caused by chiral biomolecules. When a metasurface is covered by biomolecular solution, its CD spectrum will be modulated by the biomolecules which leads to the change of the resonance frequency. As for ordinary chiral metasurface, the modulation effect is limited, because the CD spectra shift is combined with the noise of background spectra. Therefore, it is effective to design a nanostructure with a sharp chiral peak for detection of the chiral molecules.

Reinforcement learning is involved to design structures with strong chirality. The specific optimization workflow for optical properties of nanostructures based on reinforcement learning is shown in Fig. 1. We use ANNs to predict the chirality of a new nanostructure instead of electromagnetic simulations. Unlike classic supervised learning, the design process is considered as the exploration of the parameter space. Predictions of ANNs which are updated according to the feedback from the environment guide the optimization. The exploration of new nanostructures and the update of models are simultaneous. The introduction of reinforcement learning improves the quality of the training dataset and declines electromagnetic simulations.

When involving classic supervised learning to design structures of nano-photonics, it is a possible contradiction that enough training data might consume so much computing resource that machine learning does not develop the design process. Data plays a crucial role in classic supervised learning. Trained deep learning models summarize the relationship between the input data and the output data, so they cannot achieve the prediction of inputs which have nothing in common with training dataset, even for the most advanced artificial neural networks. Obviously, sufficient data is the base of supervised learning, but the generation of data cost large amounts of computing resource and time. We obtain the training data through numerical electromagnetic simulations which take much more time than the calculation of ANNs. When the training dataset covers the whole parameter space densely, involving ANNs is meaningless, because the cost of training data generation reaches or even exceeds the traditional design method. In this work, it is difficult for normal supervised learning to accurately predict spectra with a dramatic change because of the above problem. It is necessary for accurate prediction for optical response of a structure with ANNs that there are similar structures in the training dataset. As the first step of the classic supervised machine learning, it is very difficult to estimate the optical responses of structures without electromagnetic simulation while the generation of training dataset, so the structures are usually selected randomly. However, the electromagnetic simulation of all kinds of nanostructures demands more computing costs than the classical nano-photonic design does. Normal supervised learning cannot achieve the design under the limitation of calculation resources.

Reinforcement learning is an effective way for the construction of a training dataset shown in Fig. 1(a), which focuses on nanostructures with strong circular dichroism rather than all of nanostructures. Numerical electromagnetic simulations of significantly chiral structures are indispensable, because it is the main goal for ANNs to calculate spectra and distinguish structures with high values of circular dichroism. It is important to decrease the ratio of weakly chiral nanostructures in the training dataset for the decline of the simulation cost. The arrangement does not only reduce the time spent in electromagnetic simulation significantly, but also ensures that ANNs calculate optical responses of chiral structures correctly. Although ANNs trained according to this training set cannot predict spectra of nanostructures with weak chirality accurately, their results usually exhibit stochasticity and most of random spectra present weak circular dichroism. The ANNs still differentiate whether a structure has optical chirality or not, so the inaccuracy does not hinder the chiral structure design. Therefore, the training dataset including few achiral structures reduces computing resources and maintains the major function of ANNs.

The exploration of the parameter space is achieved through ANNs and Bayesian optimization algorithms. The first part of the optimization is the generation of the initial dataset. We create different nanostructures randomly and calculate the corresponding CD spectra by solving the Maxwell’s equations through the finite difference time-domain (FDTD) method. As Fig. 1(b) shows, the next step is to train several ANNs in order to obtain mapping relations between the nanostructure geometry and the optical response. New structures (shown in Fig. 1(c)) are designed with Bayesian optimization algorithm on the basis of predictions from ANNs. Although ANNs cannot provide accurate prediction at the period of the exploration because of the limited training data, they often recognize structures that possibly possess strong chirality. As a primary selection, predictions of ANNs generate possibly optimized structures and decrease the computing recourses spent on structures with weak chirality.

Figure 1(d) illustrates the process of updating the dataset. It is a useful method for judging whether the check of a structure is necessary or not to compare results from different ANNs. Structures whose optical responses estimated by several neural networks are different significantly need to be calculated again by FDTD and added to the dataset, because the inaccuracies of ANNs prove that there are few similar structures in the current dataset and it is possible that the structures possess strong chirality. On the other hand, when predictions of a structure from several ANNs are highly consistent, the predictions are credible and the dataset covers similar structures. The structure does not need check with numerical electromagnetic simulations, and it can be the optimized chirality structure. After the data updated, ANNs are retrained and generate new structures. Although the initial training data generation is random, ANNs ensure that new structures added to it have the potential of strong chirality, and this selection contributes to construct a training dataset which contains chiral structures.

The above steps compose a self-consisted cycle which searches for nanostructures with optimized chirality. Reinforcement learning achieves the combination of parameter explorations and model updates, and declines the cost of data generation. Through this method, we obtain lots of chiral structures to detect chiral biomolecules.

Figure 2(a–c) illustrates the basic structure of the chiral metasurface. Gold cuboid nanostructures are placed on a SiO₂/Si substrate with a period of 400 nm × 400 nm. The thickness of SiO₂ is 100 nm, and the height of nanostructures is 40 nm. These nanostructures are composed of several cuboids whose side lengths are an integral multiple of 10 nm, so they can be parameterized as shown in Fig. 2(c). Each spatial period is divided into 40 × 40 units encoded with 0 or 1. Code 1 represents that a gold cuboid exists at that position, and Code 0 means that the substrate is exposed. Therefore, the geometry of a nanostructure is expressed as a 40 × 40 square matrix.

Figure 2(d) shows the architecture of ANNs which predict reflective spectra of metasurfaces with a right-circularly polarized incidence (R_RCP). ANNs achieve the mapping between square matrixes which describe nanostructures and corresponding spectra. According to the symmetry of the system, the mirror symmetry transformation exchanges values of R_RCP and R_LCP, so R_LCP can be obtained based on the prediction for the flipped structure. R_RCP and R_LCP determine CD spectrum by the definition. As several types of classical Convolutional Neural Networks (CNN), ANNs consist of convolutional layers, pooling layers, activation layers and fully connection layers. It is because CNNs exhibit their advantages on image processing that we choose them to analyze optical properties of metasurfaces. Details and hyperparameters of neural networks are given in Supplementary information.

The above ANN models predict optical responses successfully after the whole design process. In the first step of the design process, 1000 random nanostructures are generated as the initial dataset. As the design process searches for new structures with a strong chirality continuously, the quantity of the dataset is enlarged to 6000. In order to estimate the performance of the ANNs involved, 80% of data composes of training data and 20% data left is used as test data. After data enhancement (See Supplementary information), the training loss defined as the mean square error between results of simulations and ANNs decline to 1.22×10⁻⁴, and the test loss is 3.31×10⁻⁴. A prediction example is shown in Fig. 2(e, f). The outputs of ANNs describe reflective spectra with a right-circularly polarized incidence accurately. As stated above, we utilize mirror reflection transformation to obtain the reflectivity in the case of a left-circularly polarized incidence and calculate corresponding CD spectra. Figure 2(f) illustrates that ANNs achieve accurate predictions of CD spectra although the spectra possess a sharp peak.

In order to prove the chirality of structures designed by the algorithm proposed, we fabricate three structures whose chiral peaks distribute in different wavelengths. Figure 3(a, e, i) illustrate scanning electron microscope (SEM) images of the structures, and Fig. 3(b, f, j) compare the corresponding CD spectra obtained from simulations and experiments. It proves the rationality of the design method that simulation curves are consistent with experiment results. The deviation between theory and experiment can be caused by machining errors.

The cathodoluminescence (CL) spectroscopy is used to analyze optical near-field resonance modes, and the CL mapping images shown in Fig. 3(c, d, g, h, k, l) explain the origin of the chirality of the proposed structures. The CL emission signals excited by an electron beam are collected with a bandpass filter and a circular polarizer in order to differentiate LCP and RCP components. Figure 3(c, g, k) illuminate that quadrupole modes dominate LCP components of near-field distributions, but as shown in Fig. 3(d, h, l), RCP components exhibit a comparatively intense dipole mode. Dipole radiation usually causes a stronger far-field signal than quadrupole radiation does, so the reflective distinction between LCP and RCP originate from the different near-field modes excited. When chiral biomolecules are coupled to nanostructures, it is supposed that the change of near-field modes leads to the frequency shift of chiral peaks. For LCP and RCP components, the ratio of dipole and quadrupole modes varies differently, because chiral biomolecules possess the divergent optical responses to LCP lights and RCP lights. The different variation results in the far-field CD spectra change.

The frequency shift of CD spectra caused by chiral biomolecules is detected in experiments. The nanostructures locate at the bottom of microchannel structure made of Polydimethylsiloxane (PDMS) as shown in Fig. 4(a). We measure reflective spectra of LCP and RCP to calculate CD spectra when the solutions of diverse chiral molecules pass the microchannels. A vertical incident light excites the coupling of structures and chiral biomolecules in solution, and the spectra collected by the objective lens above are shown in Fig. 4(b, c, d). L-glucose and D-glucose are a pair of enantiomers which have opposite chirality, and we select them as an example to test the resolving power of the microfluidic chip. It is obvious that far-field CD spectra is modulated by the refractive index of solution, because near-field optical modes depend on the refractive index of the environment. Therefore, NaCl solution which possesses the same refractive index as glucose solution is involved as a controlled experiment.

The experiment results based on three kinds of nanostructures illustrate consistently that the L-glucose and R-glucose triggers frequency shifts in different directions compared to the CD spectra of NaCl solution. The shifts originate from the coupling of chiral molecules and nanostructures rather than the change of refractive index. It is a simple theory explanation that the effective refractive indexes of chiral solution for LCP and RCP are different, which leads to distinct near-field electromagnetic modes. As the CL mapping images show, the chiral resonance peaks of the metal nanostructures appear when dipole modes excited by LCP abate significantly. Enantiomers with opposite chirality possess contrary variation of effective refractive index, so the orientations of frequency shifts are different.

As shown in Fig. 4(c), the amplitudes of frequency shifts are possibly unequal. When chiral biomolecules locate at hot spots, the localized field enhancement effects of metallic nanostructures cause that molecular circular dichroism in the ultraviolet band extended to the visible and near-infrared band. The LCP and RCP modes of chiral nanostructure possess different resonant frequencies, so the enhanced CD signals from enantiomers occur in different wavelengths. Variations of CD spectrum are dependent on the near-field electromagnetic modes of metallic nanostructures. When the LCP and RCP modes exhibit asymmetric resonant frequency deviations from the CD peak, the shifts induced by biomolecules are probably different.

At present, the chiral detection methods of biomolecules include the chiral assemblies of plasmonic nanostructures enabled by biomolecules^{1-4, 46}, chiral metasurfaces^12-19, chiral metamaterials^19-21, plasmonic nanocavities^47-48 and chiroptical active nanoparticles. Most of them focus on the macromolecules like DNA, RNA and proteins. The chiral detection of small molecules mainly rely on nanoparticles and chiral assemblies. It is reported that chiral nanoparticle films display chiral binding to glucose enantiomers and induce selective catalysis and glucose enantiomers exhibit highly selective photooxidation under CPL illumination^{4, 46}. This method obtains interfacial assemblies of the monolayer NP film through evaporating mixed solution and achieves a high sensitivity. Unlike the complex process of the above method, our approach utilizes microfluidic chips to detect the chiral biomolecules constantly, which is reusable and more convenient to practice. Compared to other chiral metasurfaces, our nanostructures differentiate one type of small molecules, which can inspire more exploration of chiral detection based on metasurfaces.

In conclusion, a nanophotonics design method based on reinforcement learning is proposed to devise microfluidic chips which achieve differentiation of chiral molecules. The algorithm combines parameter explorations with model updates and successfully obtains nanostructures which possesses sharp peaks in CD spectra. ANNs do not only learn from the given training data but propose the structures whose spectra they need to know, so the process is an automatic exploration instead of supervised learning. They accurately predict the reflective spectra with a right-circularly polarized incidence within 6000 simulations, even though the reflectivity is very sensitive to the wavelength. The CL spectroscopy analyzes the near-field electromagnetic modes of nanostructures designed by the above method, and it is considered as the origin of the chirality of the nanostructures that LCP and RCP excite different near-field modes. When the solution of enantiomers with opposite chirality flows into microfluidic chips, a pair of contrary resonance frequency shifts are detected in CD spectra. It is because the coupling of chiral biomolecules and nanostructures changes the near-field modes. Therefore, the microfluidic chips designed by artificial intelligence differentiate enantiomers correctly.

The above algorithm improves classic explorations of parameter spaces with machine learning and achieves the optimization for spectrum. However, the physical meanings of spectrum are not important for the algorithm, which means that other optical properties can be optimized through this method. The paradigm of reinforcement learning can be extended to design nanostructures for other functions. In this work, the algorithm is restricted by the physical limits of the system, but the designed nanostructures successfully differentiate the enantiomers of glucose. It is reported that coupled chiral plasmonic nanostructures including plasmonic nanocavities are more suitable for chiral detection^47-48. The algorithm has the potential for diverse nanophotonic circumstances, and the combination with these plasmonic nanostructures probably form better chiral detection. As an instance, the study illuminates the physics mechanism and the practicability of nanostructures proposed by artificial intelligence. It can inspire new applications of deep learning algorithm in nanophotonics and help explore physical limits.

This work is supported by the National Science Foundation of China (Grant Nos. 12027807, 62225501, and 11974002), National Key Research and Development Program of China (Grant No. 2020YFA0211300, 2020YFA0906900, and 2021YFF1200500), PKU-Baidu Fund Project (Grant No. 2020BD023), and High-performance Computing Platform of Peking University.

Z. Y. Fang supervised the project. Y. X. Chen designed the algorithm, fabricated metal nanostructures and measured circular dichroism spectra. F. Y. Zhang fabricated microfluidics chips and participated in biomolecular testings. X. He contributed to the fabrication of nanostructures. Z. B. Dang measured CL signals. C. X. Luo proposed the implementation scheme of microfluidics chips. Z. C. Liu, P. Peng and Y. Li participated in the design of neural networks. All the authors contributed to the discussion and analysis of the project.

The authors declare no competing financial interests.

Supplementary information for this paper is available at https://doi.org/10.29026/oes.2023.220019