In the past decade, demand for deep learning-based technologies has exploded, gradually penetrating multiple optical technology fields and driving the development of many corresponding technologies. Meanwhile, optical industries such as aerospace observation, AR/VR consumer electronics, mobile phone photography, and ultrashort-throw projectors are booming. This introduces complex design requirements for optical systems. The performance requirements of these optical systems have increased, and optical elements have become more complex. Free-form surfaces and metasurfaces have far more freedom than traditional spherical and low-order aspheric surfaces. This allows for further optimization of the independent variable parameters. Therefore, free-form surfaces and metasurfaces provide more freedom for optical system design. Moreover, free-form surfaces and metasurfaces can reduce the number of required optical components.
However, traditional optical design, manufacturing, and testing methods are not competitive for free-form surfaces and metasurfaces. In a traditional spherical optical system design, the degrees of freedom and the power orders of the independent variables are low. Therefore, iterative optimization and optical design methods are based on linear equations. In addition, solving the inverse partial differential equations can improve the completion of optical design tasks. With the demand for high-performance optical systems, the numbers of free-form surfaces and metasurfaces have significantly increased, providing a larger design space for optical systems. For free-form surfaces and metasurfaces, early iterative optimization and direct-solution optical design methods face many difficulties and challenges. The introduction of artificial intelligence (AI) technology has facilitated the development of many technologies, such as optical imaging and optical physical field regulation. System design methods have now entered a new era: the “AI optical design era”.
Deep-learning-based technologies have powerful computing, data evolution, and nonlinear inverse solving capabilities, which provide new ideas and methods for more complex optical system designs. From a mathematical perspective, AI deep learning methods are used to solve the mathematical equation of the relationship between the optical surface shape and optical aberration. AI optical design methods are not only a breakthrough at the algorithm level, but also make full use of the new hardware “computer power” in the AI era. Although most traditional inverse solutions rely on iterative optimization, AI optical design methods are based on data-driven and physical-model-driven approaches. The iterative optimization process is performed in advance during the training process without the need for real-time iterative optimization to achieve the initial optical system design quickly and accurately.
The classical optical electromagnetic theory can be used to guide the construction of neural networks for deep learning. Physical models such as aberration theory and wave aberration can be used to design loss functions that match real optical engineering problems. This loss function design significantly improves the degree of matching between deep learning networks and actual engineering problems. The rapid and accurate characteristics of AI deep learning are based on the successful training of neural networks. Additionally, deep learning-based methods are optimized through training and learning data, resulting in an intelligent and optimized design process that benefits from the data used for training in each training session.
From traditional iterative optimization to AI deep-learning optimization, optical system design methods are not completely independent or separate. This review discusses the internal path connection and development logic of the optical system design method, and looks forward to future and potential development directions. First, the development trends of optical system design requirements and optical surface shape complexity are introduced. Second, the concepts of traditional optical design methods are introduced and problems are analyzed. Subsequently, optical design optimization algorithms based on AI deep learning are introduced, which are divided and categorized according to surface types. These include spherical and low-order aspherical surfaces, free-form surfaces, diffractive elements, metasurfaces, and the co-design of optical systems and computational imaging. The principles and time consumption of traditional design algorithms and AI deep-learning algorithms are compared for different surface types (Table 1). Finally, we look forward to the future direction of development in the “AI optical design era”.
From traditional iterative optimization to AI, optical system design methods cannot be analyzed and discussed separately. In traditional convex optimization algorithms, the partial differential solution consumes extremely large CPU threads. Moreover, the interference and diffraction models involving physical optics not only consume CPU threads, but also require the real-time memory space of the computer to perform multidimensional matrix operations. AI deep learning optical design technology provides new ideas on the algorithm as well as new means for computing the hardware of a GPU or TPU. Many optical algorithms have significantly improved both the algorithms (software) and parallel computing (hardware), demonstrating that AI optical design is superior to the traditional optical system design method based on convex optimization planning, in both algorithm and hardware ‘computing power.’ An AI optical design can be used to quickly obtain the initial structure of an optical system. It can be developed in conjunction with a classic optical system design method based on convex optimization. The optical system design idea, based on an AI deep-learning architecture, is a very young breakthrough technical idea. A large number of optical technicians still need to combine practical engineering problems for further development.
