Electromagnetic (EM) metasurfaces use periodic or quasi-periodic macroscopic basic units to simulate the atoms or molecules on the microscale of traditional material science. These macroscopic basic units could interact with external electric fields via resonance effects and express unique EM properties [1–6] to realize various novel functional devices such as cloaks [7–10], concentrators [11], illusion optics devices [12,13], special lenses [14,15], and diffuse reflections [16]. Recently, a series of special EM metasurfaces called coding, digital, and programmable metasurfaces [17] (‘coding metasurfaces’ for short in the remainder of this article) have gained more attention due to the unique methods for manipulating the EM waves. By encoding the phase responses of the metasurfaces as digital numbers ‘0’ or ‘1’, the EM property of each meta-unit in the coding metasurfaces could be switched in real time when controlled by a field-programmable gate array (FPGA), which allows people to design metasurfaces in digital space rather than the analog domain. Further, the digital representation of meta-units could link the EM space and digital world and has been widely used in manipulating amplitude [18,19], polarization [20–22], and orbital angular momentum [23–25], yielding the concept of information metasurfaces [26–29] and many other novel applications [30–33].

With the development of communication technology, people need to deal with more and more information in daily life, which adds the burden of the whole society and promotes the development of artificial intelligence (AI) to help people deal with various information handling tasks such as speech recognition [34–36], image recognition [37–39], automatic translation [40–42], and robot control [43–45]. The growth of AI has also brought changes to the design of EM metasurfaces, especially coding metasurfaces. The digital representation of the coding metasurfaces makes it more convenient to put AI technology into the design of the meta-unit states [46–48] and has already sprung out various interesting applications such as the smart system [49], high-resolution imager [47,50], and recognizer [51].

Despite the wide usage of AI in coding metasurfaces, the design of meta-unit states for holographic imaging by coding metasurfaces still rests on iteration optimization algorithms like the Gerchberg–Saxton (GS) algorithm [52,53] or greedy algorithm [54]. Recently, a valuable trail has been made for using supervised deep neural networks (DNNs) to recover holograms from the generated speckles distorted by a thin diffuser [55]. However, the demand for thousands of training samples acquired by real measurements remarkably increases the cost in the usage of this supervised DNN. Here, we propose a new method based on AI or deep learning that could rapidly generate the coding pattern (metasurface hologram) of the 1 bit coding metasurface when a target holographic image is given. The deep learning structure composed of unsupervised variational auto-encoder (VAE) and conditional generative adversarial networks (cGANs) is presented in our method. The use of VAE [56], as an unsupervised algorithm, could remarkedly reduce the time consumption for the preparation of training data, and the EM propagation model described by the rigorous dyadic Green’s function (DGF) [53] is used to make the VAE structure possible while keeping the physical interpretability. We merge the Wasserstein distance [57–59] [by the form of Wasserstein GAN (WGAN) [60,61]] and mean square error (MSE) in the design of loss function to make the distributions of generated holographic images and target holographic images closer. The structure of cGAN [62–64] is also used to avoid the confusion of the generative network. We design a 40×40 one bit coding metasurface working at the frequency of 35 GHz to validate our intelligent method, and the simulation and experimental results both validate the efficiency and reliability of the proposed approach.

We accept rigorous DGF [53] as the basic computational kernel of the forward propagation from the source currents of meta-units to the near-field EM distribution: G→→(r,r′)=(I→→+∇∇k2)g(r,r′),(1)where I→→ is a 3×3 dyadic identity matrix, r and r′ are source and field points, respectively, and g(r,r′)=e−jkR4πR(2)is the free-space Green’s function, in which R=|r−r′|. The forward propagation formula could be represented as E(r)=−jωμ∫Vdr′G→→(r,r′)·J(r′),(3)where E(r) and J(r′) represent the electric field at the field point and the current at the source point, respectively. Owing to the discrete array form of the coding meta-units, the source currents can also be expressed in a discrete form. For convenience, we use a current element to represent a coding meta-unit and discrete the EM field into M points at the same time. Then, Eq. (3) could reduce to E(rm)=−jωμ∑n=1NG→→(rm,rn′)·J(rn′),m=1,…,M,(4)where N is the number of coding meta-units. Further, we organize the scalar components of E(rm) and J(rn′) in Eq. (4) for all m and n into column vectors E and J, respectively. As we can see from Eq. (4), the relationship between E(rm) and J(rn′) is linear, and, hence, they could be connected with a complex-value coefficient matrix. Then, Eq. (4) could ultimately reduce to E=W·J,(5)where W is the coefficient matrix that links the source J and field E. We only care about the vertical polarization EM waves and the relative amplitude of the near-field EM distribution. Hence, the current vector J could be described as J=Jr⊙Jφ,(6)where Jr is a complex-value vector and represents the current part that is directly caused by the incident EM waves, which is proportional to the incident electric-field values; ⊙ means element-wise multiplication; and Jφ represents the current part that is controlled by each coding meta-unit, which is a real-value vector whose elements are ‘1’ or ‘−1’ to represent the phase of 0 or π, corresponding to code ‘0’ or ‘1’. Then, Eq. (5) could be rewritten as $$\begin{gathered} E=W·Jr⊙Jφ=W·diag(Jr)·Jφ=Wr·Jφ, \\ Wr=W·diag(Jr). \end{gathered}$$(7)Thus, the forward propagation process could be represented as a form of matrix multiplication from a real-value vector Jφ to the near-field electric-field distribution E. The design objective of VAE-cGAN is to get the current vector Jφ to generate a target near-field electric-field distribution E, whose amplitude distribution represents a target holographic image.

The discriminator adopts the structure of cGAN [63] and WGAN [61], which is responsible for calculating the approximate Wasserstein distance between the generated holographic image and the target holographic image. It will be trained in the adversarial process towards the generator and improve the imaging quality of the generated holographic image.

The mainstream deep learning platform such as TensorFlow and Pytorch could not directly deal with the complex values, which will cause difficulty in the backpropagation process from loss function to Jφ. Thus, we need to derive the backpropagation partial derivative equation from the generated holographic image (the amplitude of E) to Jφ to get rid of the calculation of complex values, which is expressed as (using numerator layout) ∂|E|∂Jφ=|E|⊙−1⊙Re[E*⊙∂E∂Jφ]=|E|⊙−1⊙Re[E*⊙Wr],(8)where the superscript notation ⊙−1 means taking the reciprocal element-wisely and * means taking the conjugate matrix. Then, the backpropagation process from loss function to Jφ can be calculated by real values, ∂Loss∂Jφ=∂Loss∂|E|·∂|E|∂Jφ.(9)

We merge the MSE loss and Wasserstein distance as our final loss function. The MSE loss and Wasserstein distance are both criterions to indicate the distance between two distributions. Although using the Wasserstein distance evaluation function alone could also reach an equally good result, we still add the MSE evaluation to the loss function of the generator because the MSE evaluation could act as the ‘lubricant’ for the training process to accelerate and stabilize the convergence of the generator and discriminator. The discriminator cannot tell fake samples apart from real ones at the beginning of the training process; thus, it should be trained to learn a K-Lipschitz continuous function [60,61] so as to compute the Wasserstein distance. Therefore, at the beginning of training process, the generator is not able to get any effective guides from the discriminator for the updates of parameters, which would raise the risk of ‘mode collapse’. Luckily, the MSE evaluation function could help to guide the updates of the generator before the discriminator is well trained so as to prevent the generator from the ‘mode collapse’. This is the reason why we add it into the final loss function of the generator.

The MSE loss is described as $$\begin{gathered} MSE=1Nsum[(|E|−|Et|)⊙2], \\ ∂MSE∂|E|=2N[(|E|−|Et|)]T, \end{gathered}$$(10)where |Et| represents the target holographic image and superscript ⊙2 means element-wise square.

The mathematical expectation of p-norm distance between the discrete sequences sampled from distributions P1 and P2 is the same as that between P1 and P3. However, it is obvious that P2 is more visually similar to P1 than P3. Thus, the p-norm criterion may fail to indicate the distance between two sparse distributions, just as demonstrated in Figs. 3(b) and 3(c), in which the target holographic image [Fig. 3(b)] can be expressed as a sparse matrix, making it difficult to find the direction of optimization and fall into local minimum with the generator trained using only MSE loss, and eventually output a wrong holographic image [Fig. 3(c)]. Luckily, the Wasserstein distance could help denote this distribution difference, and the generator trained by the Wasserstein distance could generate the correct holographic image [Fig. 3(d)].

For the loss function of the discriminator, we use the loss function of WGAN [61] to simulate the calculation of Wasserstein distance and introduce the concept of cGAN [63] to match the target holographic image with the generated holographic image: Loss_D=Ex˜∼Pg|Pr[D(x˜)]−Ex∼Pr|Pr[D(x)]+λEx^∼Px^[(‖∇x^D(x^)‖2−1)2],(12)where Pg and Pr represent the distributions of the generated holographic images and target holographic images, respectively. Meanwhile, by adding the MSE loss in Eq. (10), our loss function for the generator could be expressed as Loss_G=−Ex˜∼Pg|Pr[D(x˜)]+MSE.(13)Figure 3(d) shows the generated holographic image output by the joint optimization of MSE loss and Wasserstein distance, which matches well with the target holographic image [Fig. 3(b)] and demonstrates the necessity of the mixed distance criterion.

For the experimental process, we input a testing target holographic image into the trained generator and get its output current vector Jφ. Then, the generated Jφ is binarized to ‘1’ or ‘−1’, which corresponds to the reflection phase of 0 or π (the coding meta-unit state of 0 or 1), respectively. The simulated holographic images are calculated by Eq. (7) with binarized current vectors Jφ. The coding metasurface [Fig. 1(b)] is set using an FPGA with the coding meta-unit states generated from Jφ. Last, we measure the holographic images radiated by the digital coding metasurface holograms in a standard microwave chamber.

The mean MSE and PSNR values of our VAE-cGAN evaluated with the whole testing dataset are 0.0382 and 14.33, respectively. Compared with 0.0407 (mean-MSE) and 14.05 (mean-PSNR) of the GS algorithm, our VAE-cGAN shows a better capability in searching the global optimum than the GS algorithm.

We propose a new intelligent non-iterative approach (VAE-cGAN) based on deep learning methods for metasurface holograms. The usage of an unsupervised VAE structure makes our system easily trained from scratch, and the introduction of Wasserstein distance criterion improves the imaging quality of holographic images. After the VAE-cGAN is well trained, we just need to use the generator part to generate the coding patterns (metasurface holograms) corresponding to the target holographic images. The non-iterative structure of the generator enables the realization of holographic imaging with high quality and high efficiency, which are validated by both simulation and experimental results.

When deploying our trained generator on a neural network chip, our system could become a real-time holographical imager to rapidly generate the desired holographic images. It could be expected that our intelligent metasurface hologram system could become an efficient tool for microwave or even optical holograms [69], and more valuable applications could be explored in wireless communications [32], smart EM environment, health monitoring [51,70], and so on.