Measurable speckle gradation Hadamard single-pixel imaging

Liyu Zhou; Yanfeng Bai; Qin Fu; Xiaohui Zhu; Xianwei Huang; Xuanpengfan Zou; Xiquan Fu

doi:10.3788/COL202422.031104

Abstract

The array spatial light field is an effective means for improving imaging speed in single-pixel imaging. However, distinguishing the intensity values of each sub-light field in the array spatial light field requires the help of the array detector or the time-consuming deep-learning algorithm. Aiming at this problem, we propose measurable speckle gradation Hadamard single-pixel imaging (MSG-HSI), which makes most of the refresh mechanism of the device generate the Hadamard speckle patterns and the high sampling rate of the bucket detector and is capable of measuring the light intensity fluctuation of the array spatial light field only by a simple bucket detector. The numerical and experimental results indicate that data acquisition in MSG-HSI is 4 times faster than in traditional Hadamard single-pixel imaging. Moreover, imaging quality in MSG-HSI can be further improved by image stitching technology. Our approach may open a new perspective for single-pixel imaging to improve imaging speed.

Keywords

array spatial light field measurable speckle gradation single-pixel imaging

1. Introduction

As a novel imaging technique, ghost imaging (GI) achieves image reconstruction by calculating the intensity correlation between two spatially correlated beams. The first GI demonstration was reported with entangled photon pairs in 1995^[1,2]. Later, the thermal source and pseudo-thermal source were successively used to implement GI^[3–6]. Since 2008, researchers have demonstrated that the reference beam without interaction with the object could be omitted by using a digital micromirror device (DMD) or spatial light modulator (SLM) to produce the prefabricated light source^[7–9]. Thus, GI could be implemented by utilizing a set of spatially modulated speckle patterns and a bucket detector (BD) without spatial resolution, which was also called single-pixel imaging (SPI). Initially, random speckles with the properties of a pseudo-thermal source were used as the modulated speckle patterns. Then, researchers proposed Fourier basis patterns^[10] and Hadamard basis patterns^[11] with orthogonal properties, which could obtain better imaging effects. In the last decade, SPI has drawn much attention and been successfully applied in image encryption^[12,13], biological imaging^[14,15], terahertz imaging^[16–18], and fast-moving object tracking^[19–21].

An unavoidable problem in SPI is that imaging requires multiple measurements, which restricts its imaging speed. In recent years, the array spatial light field has gradually demonstrated its particular advantage in improving imaging speed^[22–24]. In 2019, Sun et al. utilized a quadrant photodiode detector to measure the light intensity of four light fields, which could improve imaging speed by 4 times when compared with traditional Hadamard single-pixel imaging (THSI)^[22]. Then, a method combining array detector measurement and deep learning was proposed, which could obtain high imaging quality even at a low sampling rate of 0.03^[23]. By using the array spatial light field, Zhou et al. proposed real-time computational GI by using only one single-pixel detector and deep learning^[24]. It can be found from the previous work that either the array detector for detection or the deep-learning algorithm is needed to distinguish the intensity values of each sub-light field in the array spatial light field during data processing. Therefore, is there a method that requires neither an array detector nor a time-consuming deep-learning algorithm for the array spatial light field?

For this problem, one can turn one’s attention to the generation mechanism of the modulated speckle patterns that are usually generated by the DMD or the projector in SPI. It is well known that the refresh mechanism of DMD or the projector determines that the display of one projected speckle pattern is a gradual process, which results in time-consuming data acquisition process, though the BD has high sampling rate. In this paper, we propose measurable speckle gradation Hadamard single-pixel imaging (MSG-HSI), taking advantage of the high sampling rate of the BD and the refresh mechanism of the device generating the Hadamard speckle patterns. Through the reasonable design of a synchronous system, MSG-HSI is capable of obtaining the light intensity fluctuation of the array spatial light field only by using a BD, which is beneficial for distinguishing the intensity values of each sub-light field. The simulation and experiment results demonstrate that MSG-HSI can reconstruct images 4 times faster than THSI without array detector and deep learning. In addition, image stitching technology can be effectively used to eliminate image aliasing which is produced in practical MSG-HSI experiment and further improve the SNR.

Sign up for Chinese Optics Letters TOC. Get the latest issue of Chinese Optics Letters delivered right to you！Sign up now

2. Methods

Figure 1 shows the experimental setup of the THSI. A thin-film transistor liquid crystal display (TFT-LCD) is controlled by a microcontroller unit (MCU, STM32F407ZGT6) to generate a set of Hadamard speckle patterns. Then, the modulated light illuminates the transmission object $O (x, y)$ , which is tightly attached to the TFT-LCD, and all of the light transmitted by the object is collected by a lens into the BD. The MCU controls the BD and the TFT-LCD to achieve sampling synchronization. When all of the Hadamard speckle patterns are projected once, the object image can be reconstructed by applying Hadamard transform.

Figure 1.Setup of THSI.

In THSI, the Hadamard matrix ( $H_{2^{k}}$ ) is a square and symmetric matrix composed of only the values of $+ 1$ and $- 1$ and can be computed as $H_{2^{k}} = H_{2} \otimes H_{2^{k - 1}} = [\begin{matrix} H_{2^{k - 1}} & H_{2^{k - 1}} \\ H_{2^{k - 1}} & - H_{2^{k - 1}} \end{matrix}],$ with $H_{2} = [\begin{matrix} 1 & 1 \\ 1 & - 1 \end{matrix}],$ where $\otimes$ denotes the Kronecker product. $2^{k}$ is the size of the row and column of the Hadamard matrix ( $k$ is a positive integer and $k \geq 2$ ). The Hadamard speckle pattern $P_{i} (x, y)$ can be obtained by reshaping the $i$ th row/column of the Hadamard matrix into a square matrix. Considering that the Hadamard speckle patterns with the value of $- 1$ cannot be generated in the application, $P_{i} (x, y)$ is transformed into two speckle patterns [ $P_{i}^{+} (x, y)$ and $P_{i}^{-} (x, y)$ ], which can be expressed as $P_{i}^{+} (x, y) = \frac{1 + P_{i} (x, y)}{2}, P_{i}^{-} (x, y) = \frac{1 - P_{i} (x, y)}{2} .$ (1)

The corresponding sampling values of the BD can be rewritten as $b_{i}^{+} = \sum_{x} \sum_{y} P_{i}^{+} (x, y) O (x, y), b_{i}^{-} = \sum_{x} \sum_{y} P_{i}^{-} (x, y) O (x, y) .$ (2)

Through Hadamard transform, the information of the target can be obtained: $O_{THSI} = \sum_{i} P_{i} (x, y) b_{i}, i = 1, 2, 3, \dots, 2^{k},$ (3)where $b_{i} = b_{i}^{+} - b_{i}^{-}$ . From Eqs. (1 )–(3), it can be found that for the Hadamard matrix ( $H_{2^{k}}$ ), the number of speckle patterns required for THSI is $2 \times 2^{k}$ under the limit of the Nyquist measurement.

As we all know, DMD and the projector controlled by electronic chips are usually used to generate the Hadamard speckle patterns in THSI. Their maximum refresh rate is fixed and cannot be changed. Based on this case, improving their imaging speed needs to be discussed from another perspective such as their refresh mechanism. Take an LCD projector as an example^[25–27]. Figure 2(a) depicts a raster scan refresh mechanism of the LCD with $5 \times 5$ pixels for demonstration purposes. When LCD loads a speckle pattern, it will load the speckle pattern information from the left of the row to the right and then switch to the next row to load this row’s speckle pattern information. This process will be repeated by the LCD until the entire speckle pattern is loaded. For ease of understanding, Fig. 2(b) shows the display process of the letter “X” with $5 \times 5$ pixels over time. It is known that the sampling rate of the BD can be much faster than the refresh rate of the LCD since the BD does not have spatial resolution. Therefore, why not utilize the BD to measure the light intensity fluctuation while the LCD loads one speckle pattern? In this way, not one sampling value but many data about speckle gradation can be acquired for one speckle pattern, which provides an excellent condition for the use of the array spatial light field.

Figure 2.(a) Raster scan refresh mechanism of the LCD. The direction of the blue solid line and the black dashed line represents the loading order of the speckle pattern information. (b) The display process of the letter “X” with 5 × 5 pixels on the LCD over time.

The array spatial light field, as the name suggests, is composed of many sub-light fields arranged in an array. Here, its main problem is how to distinguish the intensity values of each sub-light field in the array spatial light field with one BD. To address this problem, the layout of the sub-light field in the array spatial light field needs to be designed according to the loading order of the speckle pattern information. For a Hadamard matrix ( $H_{4096}$ ), its number of speckle patterns is 8192, and the size of the speckle pattern is $64 \times 64$ . Similarly, these are, respectively, 2048 and $16 \times 64$ for a Hadamard matrix ( $H_{1024}$ ). Here, the speckle pattern with $16 \times 64$ pixels represents the sub-light field [ $P_{i}^{sub} (x, y)$ ]. Repeating the sub-light field 4 times, the array spatial light field [ $P_{i}^{array} (x, y)$ ] with $64 \times 64$ pixels can be obtained, which is detailed in Fig. 3(a). Four sub-light fields illuminate different positions of the target when the array spatial light field is projected onto the target. The light intensity fluctuation detected by the BD is presented in Fig. 3(b). The curves of the four colors correspond to the four sub-light fields, respectively. From Fig. 3(b), it is easy to get the corresponding light intensity values of each sub-light field. Moreover, the light intensity values corresponding to each row of the array spatial light field can be obtained by performing differential operations on the results in Fig. 3(b). Therefore, it can be seen that designing the array spatial light field in this way is conducive to avoiding the mutual interference between four sub-light fields during detection. Then, the partial image of the target will be reconstructed by applying the Hadamard transform on the sub-light field and the sampling values ( $b_{i}^{16 th}$ , $b_{i}^{32 nd}$ , $b_{i}^{48 th}$ , and $b_{i}^{64 th}$ ) of the 16th, 32nd, 48th, and 64th rows in Fig. 3(b). For example, imaging information of the first sub-light field can be written as $\sum_{i} P_{i}^{sub} (x, y) b_{i}^{16 th}$ . Similarly, integrating the imaging information of four sub-light fields together can obtain a complete imaging result of the target. We name this imaging scheme MSG-HSI. For MSG-HSI, there are only 2048 speckle patterns, not 8192 speckle patterns required by $H_{4096}$ to reconstruct the target with $64 \times 64$ pixels. To highlight the performance of MSG-HSI, THSI achieving imaging with $64 \times 64$ pixels by using the Hadamard matrix ( $H_{4096}$ ) will also be presented and compared. In addition, the image reconstruction results will be presented when 2048 speckle patterns of the above type and the sampling value of the 64th row ( $b_{i}^{64 th}$ ) are used (Scheme I), and it can be expressed as $\sum_{i} P_{i}^{array} (x, y) b_{i}^{64 th}$ .

Figure 3.Schematic diagram of MSG-HSI. (a) The flow chart of speckle pattern repeating. The four sub-light fields are marked with four colors (red, green, blue, and yellow). (b) The light intensity fluctuation detected by the BD during the projection of one speckle pattern. (c) The difference result of (b).

3. Numerical and Experimental Results

In this section, numerical simulations and experiments are carried out to verify the proposed scheme. To evaluate the reconstruction results qualitatively, the SNR is defined as $SNR = 10 \log_{10} \frac{\sum {(O - \bar{O})}^{2}}{\sum {(O - R)}^{2}},$ (4)where $O$ and $R$ correspond to the original object image and the reconstruction image, respectively. $\bar{O}$ is the mean of $O$ .

First, the imaging object is shown in Fig. 4(a), and its imaging resolution is $64 \times 64$ . The number of speckle patterns in Scheme I, MSG-HSI, and THSI is 2048, 2048, and 8192, respectively. Figures 4(b)–4(d) present the simulation results in three schemes. It can be seen that the object information is divided into four parts and aliased together in the reconstruction image of each sub-light field when there is no intensity values corresponding to each row of the array spatial light field [see Fig. 4(b)]. On the contrary, MSG-HSI can distinguish the details of the target from aliasing after the light intensity fluctuation is known [see Fig. 4(c)]. It is also noted that imaging results in MSG-HSI and THSI are the same as the original object due to the fact that the noise interference is not considered in the simulation, but the number of speckle patterns required in MSG-HSI is only one-fourth of that in THSI, which proves the feasibility of MSG-HSI theoretically.

Figure 4.(a) The binary image “ghost-I.” (b)–(d) The reconstruction results based on Scheme I, MSG-HSI, and THSI, respectively.

Next, the performance of three schemes is further compared through experiments, and the experimental setup is shown in Fig. 1. Here, we focus on how to implement MSG-HSI in our experiment. A programmable MCU is used to control the TFT-LCD, the BD, and a secure digital memory card (SD card) storing the speckle patterns. The refresh rate of the TFT-LCD is 6 Hz and the sampling rate of the BD is 28 kHz. During the experiment, the MCU reads a speckle pattern from SD card and then sends it to the TFT-LCD. The MCU will control the BD to measure the light intensity once whenever the TFT-LCD finishes loading a row of the speckle pattern information. Moreover, the reason why the BD with high sampling rate is needed is that it takes time for the TFT-LCD to switch rows, and the BD sampling must be completed within this time. Therefore, the time consumption for the BD to measure the light intensity 64 times is the same as that for only once. In addition, it needs to be emphasized that although our experiment is demonstrated by TFT-LCD, our scheme can also be used for other spatial light modulation devices with similar refresh mechanisms, such as SLM and DMD.

Without loss of generality, the binary image “concentric ring,” the binary image “ghost-I,” the Chinese character “学,” and the grayscale image “ghost-II” are used as the objects imaged, which are $2.3 cm \times 2.3 cm$ in size. Their imaging resolution is also $64 \times 64$ and the corresponding experimental results are shown in Fig. 5. Compared with the simulation, it can be observed that there is more imaging noise in the reconstruction images of Scheme I and THSI. Note that there is a little aliasing in the experimental results of the MSG-HSI [see red dashed box in Fig. 5(c)]. A reasonable explanation is that non-synchronization between the BD and the TFT-LCD may give rise to this problem when the circuitry is unstable. As a result, the SNRs in the MSG-HSI are not as good as that in the THSI. To solve this problem, we introduce a characteristic of the array spatial light field, i.e., any continuous speckle pattern with $16 \times 64$ pixels in the array spatial light field can be used to reconstruct the object image through Hadamard transform. This is due to the fact that the light intensity values corresponding to each row of the speckle pattern are known in the MSG-HSI, as shown in Fig. 3(c). As mentioned above, imaging needs the sampling values of the 16th, 32nd, 48th, and 64th rows, which respectively correspond to the red (1–16 rows), green (17–32 rows), blue (33–48 rows), and yellow (49–64 rows) speckle patterns [see Fig. 6(a1)]. Now, the sampling values of the 8th, 24th, 40th, and 56th rows are applied to the imaging again for the red (9–24 rows), green (25–40 rows), and blue (41–56 rows) speckle patterns [see Fig. 6(a2)]. The corresponding reconstruction images are shown in Fig. 6(b). By stitching the imaging results without aliasing in these images, new stitching images are obtained, which are highlighted in Figs. 6(c)–6(f). Comparing Figs. 5(c) and 6(c)–6(f), it is apparent that the image stitching technology can effectively avoid the influence of non-synchronization and improve the SNR. Although the SNR in MSG-HSI drops by $\sim 8 %$ compared to THSI, the details of the targets are still well reconstructed. This experimentally verifies that our scheme can improve imaging speed by 4 times with only one BD when compared with THSI.

Figure 5.Experimental results for (a1)–(a4) binary image “concentric ring,” binary image “ghost-I,” Chinese character “学,” and grayscale image “ghost-II,” respectively. (b)–(d) The reconstruction results based on Scheme I, MSG-HSI, and THSI, respectively.

Figure 6.The flow chart of the image stitching. (a) Different imaging areas in the array spatial light field. (b1), (b2) Imaging results corresponding to (a1) and (a2), respectively. (c)–(f) Image stitching results.

Finally, it needs to be stressed that our scheme has the potential to improve imaging speed by $M$ times for imaging with $M \times N$ pixels in principle ( $M$ denotes the number of rows). Because the array spatial light field is divided into $M$ sub-light fields, the number of the speckle patterns for the MSG-HSI is reduced by $M$ times. In our experiment, the imaging resolution is $64 \times 64$ , and the experimental results only demonstrate that our scheme improves imaging speed by 4 times instead of 64 times. The reason is that the impact of non-synchronization makes it more difficult to solve the problem of image aliasing if the array spatial light field is divided into smaller sub-light fields in our experiment. Therefore, further experimental investigations are needed to approach the limit of theoretical imaging speed improvement.

4. Conclusion

In conclusion, we have developed MSG-HSI where the intensity values of each sub-light field in the array spatial light field can be measured with only one BD. Experiment and simulation results show that MSG-HSI is capable of reconstructing images 4 times faster than THSI by means of the refresh mechanism of the device generating the Hadamard speckle patterns. Furthermore, imaging results of different areas in the array spatial light field can be obtained in MSG-HSI, which is beneficial for eliminating image aliasing with the help of image stitching technology.

References

[1] T. B. Pittman, Y. H. Shih, D. V. Strekalov et al. Optical imaging by means of two-photon quantum entanglement. Phys. Rev. A, 52, R3429(1995).

[2] D. V. Strekalov, A. V. Sergienko, D. N. Klyshko et al. Observation of two-photon ghost interference and diffraction. Phys. Rev. Lett., 74, 3600(1995).

[3] R. S. Bennink, S. J. Bentley, R. W. Boyd. ‘Two-photon’ coincidence imaging with a classical source. Phys. Rev. Lett., 89, 113601(2002).

[4] J. Cheng, S. S. Han. Incoherent coincidence imaging and its applicability in X-ray diffraction. Phys. Rev. Lett., 92, 093903(2004).

[5] F. Ferri, D. Magatti, A. Gatti et al. High-resolution ghost image and ghost diffraction experiments with thermal light. Phys. Rev. Lett., 94, 183602(2005).

[6] A. Valencia, G. Scarcelli, M. D’Angelo et al. Two-photon imaging with thermal light. Phys. Rev. Lett., 94, 063601(2005).

[7] J. H. Shapiro. Computational ghost imaging. Phys. Rev. A, 78, 061802(2008).

[8] Y. Bromberg, O. Katz, Y. Silberberg. Ghost imaging with a single detector. Phys. Rev. A, 79, 053840(2009).

[9] B. Sun, M. P. Edgar, R. Bowman et al. 3D computational imaging with single-pixel detectors. Science, 340, 844(2013).

[10] Z. B. Zhang, X. Ma, J. G. Zhong. Single-pixel imaging by means of Fourier spectrum acquisition. Nat. Commun., 6, 6225(2015).

[11] L. Wang, S. M. Zhao. Fast reconstructed and high-quality ghost imaging with fast Walsh-Hadamard transform. Photonics Res., 4, 240(2016).