Optical design of a compact and high-transmittance compressive sensing imaging system enabled by freeform optics

Dewen Cheng; Hailong Chen; Tong Yang; Jun Ke; Yang Li; Yongtian Wang

doi:10.3788/COL202119.112202

Abstract

Using compressive sensing for imaging has many applications, and it is an important branch of computational imaging. In this Letter, freeform surfaces are introduced in the hardware optical system design of a compressive sensing imager. The system works under the medium wave infrared band and realizes a

16 \times

compression with a field-of-view of

7.5 ° \times 6 °

. Good imaging performance is achieved in both the entire system and the freeform objective optics. Compared with the system using all spherical lenses, the volume of the freeform system is about 1/3 smaller, and the total transmittance is about 56% higher, which shows the benefits of using freeform surfaces for compressive sensing and computational imaging.

Keywords

compactness and high-transmittance compressive sensing freeform optics joint optimization

1. Introduction

Computational imaging can be seen as indirectly forming images from measurements using algorithms that rely on a significant amount of computing^[1,2]. Among the wide range of applications of computational imaging, compressive sensing (CS) is a very important branch and has been extensively investigated in recent fifteen years. CS is a mathematical framework on how a high-resolution image can be inferred from a relatively small number of measurements, based on the fact that the image is sparse and thus compressible in some basis^[3]. For example, CS can be realized by firstly imaging the scene in the object space to a mask, which modulates the image in proportion to the elements of the random vectors^[4]. Then, the scene is refocused onto the final image plane. The mask can be realized by using programmable devices such as a spatial light modulator (SLM). Using proper codes and recovery algorithms, the large resolution image can be obtained by using low-resolution measurement. Although CS can be used in many areas, the application of CS in infrared (IR) imagers received considerable attention and research. This is because the IR detector or focal plane array (FPA) generally has a limited resolution and format, while the limited existing types of high-resolution FPA are very expensive, which constrain its applications in wide field-of-view (FOV) and high-resolution imaging. Using CS theory, this problem can be overcome. Some related systems have been proposed and developed^[4,5]. However, these systems may include many IR lenses (some of them may have a large size), which will significantly increase the total weight, volume, cost, and complexity of the whole system, while decreasing the transmittance and reducing the total energy received by the detector. To deal with this problem, freeform surfaces can be used in the IR CS imaging system design.

Freeform optical surfaces can be characterized as non-rotationally symmetric surfaces, and they can offer many more degrees of freedom for imaging system design. They can correct the aberrations well in non-symmetric systems and help to achieve better imaging performance, higher system specifications, more compact system configuration, and fewer elements. Using freeform optical surfaces in imaging systems can be seen as a revolution in the field of optical design^[6,7]. Up to now, freeform surfaces have been successfully used in many areas^[8–20]. If we introduce freeform surfaces into the design of IR CS systems, the bulky IR lenses can be replaced by a reduced number of freeform lenses or mirrors, which will significantly reduce the complexity, volume, and weight of the system while improving the total transmittance.

In this Letter, we demonstrate a design of a CS imager using freeform optics. This is the first time, to the best of our knowledge, that freeform surfaces are introduced in the design of computational imaging and CS systems. The entire system works under the medium wave IR band (MIR). A novel system configuration is proposed. An off-axis freeform three-mirror objective system and coaxial relay optics are integrated to realize CS. The design method and strategy of a freeform CS imaging system are proposed. The design and integration process of the individual parts, as well as the joint design of the entire CS system, are demonstrated, including pupil matching, light obscuration elimination, system structure control, and system specifications control. The final system realizes a 16 times compression and an FOV of $7.5 ° \times 6 °$ . Good imaging performance is achieved in both the entire system and the freeform objective optics. Compared with the system using all spherical lenses (a design has also been proposed in this Letter), large, bulky, and expensive IR lenses are not used in the freeform design. The volume of the freeform system is about 1/3 smaller, and the total transmittance is about 56% higher. The total number of optical elements is reduced. The design results show the benefits of freeform surfaces and open a new and promising way to achieve high-performance and compact computational imaging systems. Related design methods and system configuration can also be used in the design of imaging optics for other applications.

Sign up for Chinese Optics Letters TOC. Get the latest issue of Chinese Optics Letters delivered right to you！Sign up now

2. Basic Principles and System Specifications

The basic principles of CS used for the IR systems in this Letter are summarized as follows^[4]. As shown in Fig. 1, the scene is firstly imaged to the SLM using imaging optics (the objective optics). Then, the scene is refocused onto an FPA. To realize CS, the size of the mask has to match the number of points in the scene and spatially modulate the light. However, the large size of the mask significantly increases the difficulty in the mask design and information recovery. Therefore, block-based CS can be used^[21,22]. The SLM is divided into many contiguous element blocks. Each block of $N \times N$ elements in the SLM ( $N \times N$ points in the scene) corresponds to a single element on the image plane. The masks are also $N \times N$ , and, in this way, the design of the mask and information recovery will be more simplified. Details of CS algorithms (which are not the main topics of this Letter) can be found in Refs. [3,4,21 –25].

Figure 1.Sketch of the MIR CS system.

In this Letter, a digital micromirror device (DMD) is taken as the SLM. The DMD used in the system has $1920 \times 1080$ pixels, and the size of each pixel is $10.8 μm \times 10.8 μm$ . However, in this design we use only $1280 \times 1024$ pixels to code the image. Each micromirror in the DMD has two states: 12° tilt angle or $- 12 °$ tilt angle. The objective optics form the image on the DMD. Then, a subsequent optical system can relay the image to the FPA when the mirrors are in one of the two states (“on” state), while the light coming from the object space is deflected away from the FPA but into a cold plate when the mirrors are in the other state (“off” state). The specific masks and spatial light modulation are realized by controlling the states of the different mirror elements in the DMD. In this Letter, we use a commercially cooled IR detector with an F-number of 2 working in the MIR band ( $3.7 - 4.8 μm$ ) directly. The FPA in the detector has $320 \times 256$ pixels. The size of each pixel is $15 μm \times 15 μm$ . A cold stop is located above the FPA. A silicon window and a germanium filter are located above the cold stop. Each element on the FPA corresponds to a block of $4 \times 4$ elements on the DMD. As a result, this mapping realizes a 16 times compression at native resolution. The focal length of the objective system is set to be 105.52 mm. The rectangular DMD corresponds to a rectangular FOV of $7.5 ° \times 6 °$ [as shown in Fig. 2(a)]. Since each mirror can only rotate around the diagonal axis of the square mirror pixel, the DMD needs to be rotated by 45° clockwise or anticlockwise [as shown in Fig. 2(b)] to keep the whole system plane-symmetric and make sure the reflected central ray is parallel to the plane of symmetry (YOZ plane or meridional plane). In addition, due to the plane symmetry of the system, the FOV coverage used in the actual design process can fall into the $+ x$ direction (or $- x$ direction), as shown in Fig. 2(c). The cooled detector should be also rotated by 45° to match the object (DMD) of the relay optics. The reduction ratio of the relay optics can be calculated by the above specifications, which should be 1/2.88. Two different designs using the above system specifications and devices will be presented in the following sections.

Figure 2.FOV coverage of the system. (a) Rectangular FOV. (b) The FOV coverage is rotated. (c) The FOV coverage used in the actual design process.

3. System Design Using Traditional Spherical Lenses

We first discuss the design using traditional imaging optics. In Ref. [4], the authors have presented a good design (the system specifications are different from those of this Letter). Both the objective optics and the relay optics use traditional spherical lenses (11 IR lenses are used in total). Here, we present a similar design for the requirements given in this Letter, as shown in Fig. 3. Nine IR lenses are used in total. The coaxial objective optics consist of three IR spherical lenses. Light coming from the object space is focused on the DMD plane (a ZnSe window is located above the DMD plane), which is the intermediate image plane of the entire system. In the “on” state, the DMD mirrors are tilted at 12°. The light will be reflected by 24° and travel back through objective lenses. Then, the light will be deflected into the relay optics, which consist of six spherical lenses, and finally will be focused on the FPA and form the image. In the “off” state, the DMD mirrors are tilted at $- 12 °$ . The light will also be reflected by 24° (but in a different direction compared with the “on” state) and travel back through the objective lens to a cold plate. The modulation transfer function (MTF) plots for the entire system and the objective optics are shown in Fig. 4. It should be noted that during the design and analysis in optical design software (such as CODE V), the typical field points are sampled among the FOV coverage shown in Fig. 2(c). Although the above two traditional designs can achieve the design requirements and realize CS, these systems use many IR lenses (many of them have a large size), which will lead to increased complexity as well as system volume (about $46 mm \times 220 mm \times 245 mm$ , considering only the optical elements) and weight. Additionally, the transmittance of the whole system will be low, and the total energy reaching the FPA is limited. The light travels through 16 refractive elements (four refractive elements in the objective optics are used twice) and three reflective elements in the system. If the reflectivity is assumed to be 4% for a refractive lens and 96% for a mirror surface, and the absorptivity of a lens is assumed to be 2%, then the total transmittance is about ${0.96}^{16} \times {0.96}^{3} \times {0.98}^{16} = 33.3 %$ .

Figure 3.Layout of the system using traditional spherical lenses.

Figure 4.MTF plots of the traditional design. (a) Entire system. (b) Objective optics.

4. System Design Using Freeform Optics

To overcome the drawbacks of the traditional design, in this Letter, we represent an alternative design with the same system specifications, which use freeform surfaces in the system. The objective optics and the relay optics are firstly designed separately and then are integrated as one system for final joint optimization. The detailed design processes are depicted as follows.

As the design requires high imaging performance over the full FOV in both the objective optics and the final entire system, and strict requirements in system configuration should be fulfilled (such as large chief ray angle on DMD, and elimination of light obscuration), all three mirrors are freeform surfaces, and the surface type is an XY polynomials surface up to the sixth order with a base conic at this design stage. Because the optical system is symmetric about the YOZ plane, only the even items of $x$ in the XY polynomials are used. In the “on” state, the DMD mirrors are tilted at 12°. The light will be reflected by 24° and travel to the relay optics. As the relay optics are expected to be a coaxial system using spherical lenses, the incident angle of the chief ray of the central field on the DMD plane should be constrained as 24°. In this way, the outgoing direction of this ray will be perpendicular to the DMD plane, which matches the requirements of a coaxial system. In the “off” state, the DMD mirrors are tilted at $- 12 °$ , and the light will be reflected to a cold plate. Some other constraints have to be used to keep the system feasible. The focal lengths of the system in the $x$ and $y$ directions are calculated using the ABCD matrix method and then controlled. The distortion of the system is controlled using real ray tracing data. To eliminate light obscuration or avoid surface interference, the marked distances $L_{1}$ to $L_{3}$ shown in Fig. 5(a) are controlled. The chief ray of the central field is controlled to intersect at the center of each freeform surface (the vertex of the freeform surface). Large conic constants may lead to freeform surfaces which are ill-behaved outside the controlled aperture and often are not very effective at improving the image performance. Here, we controlled the values of conic constants to be within $\pm 10$ . Other values (not too large) are also applicable. Another important constraint that should be considered is the pupil matching of the objective optics and relay optics. The exit pupil location of the objective optics should coincide with the entrance pupil location of the subsequent relay optics. In this design, we constrain the exit pupil locations in both the $x$ and $y$ directions to be 220 mm (other values are also feasible) after the DMD plane. These locations can be calculated using the ABCD matrix method. The locations of primary mirror (M1) and tertiary mirror (M3) are constrained to be close in the $z$ direction. In this way, these two mirrors may be fabricated on a monolithic substrate and therefore reduce the assembly difficulty. During optimization, the number of sample fields is gradually increased. The design result of the freeform objective optics is shown in Fig. 5(a). The MTF plot is given in Fig. 5(b). The average RMS spot diameter among the full FOV is about 7.7 µm.

Figure 5.Design result of the freeform objective system. (a) System layout. (b) MTF plot.

The relay optics are used to reimage the scene from the DMD to the FPA. Here, a rotationally symmetric coaxial system is designed. The relay optics consist of four spherical lenses. The exit pupil of the system should match the cold stop of the cooled detector. In this way, the cold stop can be used with high efficiency in order to significantly limit the unwanted background radiation and increase the signal-to-noise ratio. In this design, the cold stop is set as the aperture stop of the system directly. The field type used in the design is object height. The maximum height is half of the diagonal length of the DMD (used area). During the optimization, the reduction ratio of this system is controlled to be 1/2.88. The distance between the object plane and the first IR lens should not be too small to avoid light obscuration in the final integrated system. To realize the pupil matching of the objective optics and relay optics, the location of entrance pupil (in both the $x$ and $y$ directions) of the relay optics is constrained to be at 220 mm after the object plane. The system layout is shown in Fig. 6(a). The MTF plot of the relay optics is given in Fig. 6(b), which shows good imaging performance.

Figure 6.Design result of the coaxial relay optics. (a) System layout. (b) MTF plot.

After the individual design of the objective optics and the relay optics, the next step is to integrate two systems as one and then apply joint optimization. After direct integration, the imaging performance is dropped, the first IR lens is very close to the light coming from M3, and this distance should be controlled during the joint optimization to avoid obscuration. Two zooms are optimized simultaneously: zoom #1 is the full system, and zoom #2 is the freeform three-mirror objective system (part of zoom #1, setting the image plane at the DMD plane). Both zooms should have good imaging performance. The optimization of two zooms is similar to the methods given in Sections 4. The cold stop should be the aperture stop of the entire system. In addition, we hope M2 is approximately conjugate to the cold stop (which means M2 is still a pupil surface). If the cold stop is set to be aperture stop in zoom #1, the chief ray intersections of different fields with M2 should be controlled at the vertex of M2. During optimization, the surface order of the freeform surface for three mirrors is gradually upgraded to 10 (only the even items of $x$ are used). The layout of the final system is plotted in Fig. 7, which shows the optical paths of the “on” state and “off” state. The MTF plots for the full system and the three-mirror objective system are given in Fig. 8. The complexity of the system is reduced compared with the traditional design, and bulky IR lenses are not used. The entire system volume is about $52 mm \times 156 mm \times 200 mm$ (considering only the optical elements), which is about 1/3 smaller than volume of the traditional system given in Section 3. The light travels through eight refractive elements (the window of the DMD is used twice) and four reflective elements in the traditional system. If the reflectivity is also assumed to be 4% for a refractive lens and 96% for a reflective mirror surface, and the absorptivity of a lens is also assumed to be 2%, then the total transmittance is about ${0.96}^{8} \times {0.96}^{4} \times {0.98}^{8} = 52.1 %$ , which is about 56% higher than that of the traditional design, which is quite important in the IR system design. Considering that the absorption of the three bulky IR lenses in the objective optics of the traditional design will actually have higher absorptivity than other elements, the traditional design has in fact a lower transmittance than the calculated value given in Section 3. Comparisons of the two systems are listed in Table 1 (DMD and detector are not counted in element numbers).

Items	Traditional Design	New Design
FOV	$7.5 ° \times 6 °$	$7.5 ° \times 6 °$
Compression	$16 \times$	$16 \times$
Number of elements	11	7
System volume	$46 mm \times 220 mm \times 245 mm$	$52 mm \times 156 mm \times 200 mm$
Total transmittance	33.3%	52.1%

Table 1. Comparisons of the Two Systems

View all Tables

Figure 7.Final design result of the MIR CS system using freeform surfaces.

Figure 8.MTF plots of MIR CS system using freeform surfaces. (a) Entire system (zoom #1). (b) Freeform objective optics (zoom #2).

To predict the actual performance of the as-built system and guide the manufacturing and assembly of the hardware system, tolerance analysis is further conducted. The analysis is performed for two zooms simultaneously in the sensitivity analysis mode using the wavefront differential tolerance analysis method in CODE V^[26]. The performance metrics are the MTF value in both the $x$ and $y$ directions at 34 cycles/mm frequency for zoom #1 and 20 cycles/mm frequency for zoom #2. The tolerance items as well as the corresponding values are listed in Table 2. (In this analysis, M1 and M3 are still seen as individual elements. If M1 and M3 are integrated as a single element, the tolerance items can be reduced.) The compensators used in the analysis are the locations and tilts of the DMD, relay optics, and the IR detector. The cumulative probability curves for two zooms are shown in Fig. 9. The 97.7% probable MTF values at the evaluation frequencies are above 0.50 and 0.36 for two zooms, respectively.

Figure 9.Cumulative probability curves of tolerance for two zooms. (a) and (b) are the results of the tolerance analysis of zoom #1 for MTF in the x and y directions, respectively. (c) and (d) are the results of the tolerance analysis of zoom #2 for MTF in the x and y directions, respectively.

Location	Tolerance type	Value
All surfaces in the freeform objective optics and relay optics	DLX (single surface x displacement)	0.02 mm
	DLY (single surface y displacement)	0.02 mm
All surfaces in the freeform objective optics	DLZ (single surface z displacement)	0.02 mm
	DLA (single surface α-tilt)	2 arcmin
	DLB (single surface β-tilt)	2 arcmin
	DLG (single surface γ-tilt)	10 arcmin
	RSE (random RMS surface error)	55 nm
All surfaces in the relay optics	DLA (single surface α-tilt)	1 arcmin
	DLB (single surface β-tilt)	1 arcmin
	DLT (thickness delta)	0.02 mm
	DLF (test plate fit-power)	2 fringes (WL: 546.1 nm)
All lenses in the relay optics	DSX (group x decenter)	0.02 mm
	DSY (group y decenter)	0.02 mm
	BTX (barrel β-tilt)	2 arcmin
	BTY (barrel α-tilt)	2 arcmin
	DLN (refractive index delta)	0.001
	DLV ( $v$ -number delta)	0.004

Table 2. The Tolerance value

View all Tables

5. Conclusions

In this Letter, we proposed a design of a CS imager using freeform optics. To the best of the authors’ knowledge, this is for the first time that freeform surfaces are introduced in the hardware design of computational imaging and CS systems. The system works in the MIR band. A 16 times compression and an FOV of $7.5 ° \times 6 °$ are realized. Detailed design processes of the individual parts and the integrated system are demonstrated. Good imaging performance is achieved in both the entire system and the freeform objective optics. Compared with the system using all spherical lenses, the volume of the freeform system is about 1/3 smaller, and the total transmittance is about 56% higher. The design results show the benefits of freeform surfaces and open a new and promising way to achieve high-performance and compact computational imaging systems.

References

[1]

[2] G. Barbastathis, A. Ozcan, G. Situ. On the use of deep learning for computational imaging. Optica, 6, 921(2019).

[3] R. Willett, R. Marcia, J. Nichols. Compressed sensing for practical optical imaging systems: a tutorial. Opt. Eng., 50, 072601(2011).

[4] A. Mahalanobis, R. Shilling, R. Murphy, R. Muise. Recent results of medium wave infrared compressive sensing. Appl. Opt., 53, 8060(2014).

[5] H. Chen, M. S. Asif, A. C. Sankaranarayanan, A. Veeraraghavan. FPA-CS: focal plane array-based compressive imaging in short-wave infrared. 2015 IEEE Conference on Computer Vision and Pattern Recoginition, 2358(2015).

[6] K. P. Thompson, J. P. Rolland. Freeform optical surfaces: a revolution in imaging optical design. Opt. Photon. News, 23, 30(2012).

[7] S. Wills. Freeform optics: notes from the revolution. Opt. Photon. News, 28, 34(2017).

[8] D. Cheng, Y. Wang, H. Hua, M. M. Talha. Design of an optical see-through head-mounted display with a low f-number and large field of view using a freeform prism. Appl. Opt., 48, 2655(2009).

[9] Y. Deng, G. Jin, J. Zhu. Design method for freeform reflective-imaging systems with low surface-figure-error sensitivity. Chin. Opt. Lett., 17, 092201(2019).

[10] A. Bauer, E. M. Schiesser, J. P. Rolland. Starting geometry creation and design method for freeform optics. Nat. Commun., 9, 1756(2018).

[11] A. Bauer, M. Pesch, J. Muschaweck, F. Leupelt, J. P. Rolland. All-reflective electronic viewfinder enabled by freeform optics. Opt. Express, 27, 30597(2019).

[12] T. P. Johnson, J. Sasian. Zernike monomials in wide field of view optical designs. Appl. Opt., 59, G146(2020).

[13] Y. Nie, R. Mohedano, P. Benítez, J. Chaves, J. C. Miñano, H. Thienpont, F. Duerr. Multifield direct design method for ultrashort throw ratio projection optics with two tailored mirrors. Appl. Opt., 55, 3794(2016).

[14] R. Wu, Z. Feng, Z. Zheng, R. Liang, P. Benítez, J. C. Minano, F. Duerr. Design of freeform illumination optics. Laser Photon. Rev., 12, 1700310(2018).

[15] R. Wu, L. Yang, Z. Ding, L. Zhao, D. Wang, K. Li, F. Wu, Y. Li, Z. Zheng, X. Liu. Precise light control in highly tilted geometry by freeform illumination optics. Opt. Lett., 44, 2887(2019).

[16] Y. Zhong, H. Gross. Improvement of Scheimpflug systems with freeform surfaces. Appl. Opt., 57, 1482(2018).

[17] M. Beier, J. Hartung, T. Peschel, C. Damm, A. Gebhardt, S. Scheiding, D. Stumpf, U. D. Zeitner, S. Risse, R. Eberhardt, A. Tünnermann. Development, fabrication, and testing of an anamorphic imaging snap-together freeform telescope. Appl. Opt., 54, 3530(2015).

[18] A. Wilson, H. Hua. Design and demonstration of a vari-focal optical see-through head-mounted display using freeform Alvarez lenses. Opt. Express, 27, 15627(2019).

[19] P. Benitez, J. C. Miñano, P. Zamora, D. Grabovičkić, M. Buljan, B. Narasimhan, J. Gorospe, J. López, M. Nikolić, E. Sánchez, C. Lastres, R. Mohedano. Advanced freeform optics enabling ultra-compact VR headsets. Proc. SPIE, 10335, 103350I(2017).

[20] L. Feng, J. Zhou, L. Wei, X. He, Y. Li, J. Jing, B. Xiangli. Design of a compact wide-spectrum double-channel prism imaging spectrometer with freeform surface. Appl. Opt., 57, 9512(2018).

[21] J. Ke, E. Y. Lam. Object reconstruction in block-based compressive imaging. Opt. Express, 20, 22102(2012).

[22] L. Gan. Block-compressed sensing of natural images. IEEE 15th International Conference on Digital Signal Processing, 403(2007).

[23] P. L. McCarley, M. A. Massie, J. P. Curzan. Foveating infrared image sensors. Proc. SPIE, 6660, 666002(2007).

[24] J. P. Dumas, M. A. Lodhi, W. U. Bajwa, M. C. Pierce. Computational imaging with a highly parallel image-plane-coded architecture: challenges and solutions. Opt. Express, 24, 6145(2016).

[25] E. J. Candes. The restricted isometry property and its implications for compressed sensing. Cr. Math., 346, 589(2008).

[26] (2018).