Imaging objects hidden behind opaque barriers is meaningful in many fields, ranging from biomedical to defense applications. Several approaches have been proposed in the past few decades, such as wavefront shaping^[1,2], deconvolution imaging^[3–5], and speckle-correlation imaging^[6,7]. Especially, the speckle-correlation imaging, which has the potential to image in currently inaccessible scenarios even with single-shot capture, is considered as one of the most promising scattering imaging techniques^[7,8]. The speckle-correlation technique has been extended for imaging moving targets hidden behind an optically turbid medium even in the bright-field scenario^[9], multispectral imaging through a thin scatterer based on spectral coding and compressed sending^[10], scattering imaging under strong ambient light interference by refining the speckles^[11], and noninvasive superresolution imaging through scattering media at the nanometer level^[12].

However, the above techniques have to contain an iris, which is necessary to enlarge the size of speckle grains, as well as to obtain high-contrast speckles. According to statistical optics theories^[13], the mean radius δ¯ of speckle grains generated by a circular diffuser is determined by δ¯=1.22λv/D (λ is the wavelength, D is the diameter of the diffuser, v is the distance between the diffuser and the detector). Generally, the mean size of the speckle grains δ¯ is smaller than the pixel size of the detector, leading to reconstruction distortion or even reconstruction failure. It should be pointed out that most current literature attributes the reconstruction failure in this case to the violation of the Shannon–Nyquist sampling theorem in the process of capturing the speckle images^[7,9,10,14]. Therefore, to solve this problem, in the image acquisition process of pseudothermal-light-illumination speckle-correlation imaging systems, an additional iris with a diameter of D′ (D′ is typically less than 1 cm, and much smaller than the diameter of the diffuser D) is placed between the diffuser and the detector to enlarge the mean radius of speckle grains and yields high-contrast speckles^{[7–10,12,14]}. As for the LED-illumination system, a small enough aperture, lying between the LED and object, is used to limit the size of a light spot projected onto the diffuser, which acts as an iris to reduce the effective area of the diffuser^[11,15–17]. Basically, the above speckle-correlation imaging methods are both essentially cooperative because the position and size of the iris require it to be adjusted with the object’s position and size to obtain a high-quality speckle autocorrelation.

Here, we propose a non-iris speckle-correlation imaging technique that can achieve a high-fidelity object reconstruction from a low-quality non-iris speckle image by applying an image-resizing or pixel-binning procedure before the conventional phase-retrieval reconstruction algorithm. An empirical formula was presented to estimate the suitable scale factor based on the specific experimental parameters. The proposed method is robust and universal and can relieve the experiment complexity of speckle-correlation imaging. Furthermore, the proposed method implemented by on-chip pixel binning will remarkably increase the frame rates, which will further benefit speckle-correlation imaging for dynamic objects, as well as high-frame-rate video technique of objects hidden behind opaque barriers.

A non-iris speckle-correlation imaging system is shown in Fig. 1(a). A transmissive object was illuminated by a narrowband spatially incoherent pseudothermal source that was generated by combining a continuous-wave laser (λ=532nm) and a rotating diffuser (LSR-3005, Edmund Optics); then the light was scattered by a diffuser (DG20-220-MD, Thorlabs) and imaged by a lensless monochrome camera (QHY600, pixel size 3.76 µm, 9600pixels×6422pixels) directly. The size of the object is limited by optical memory effect^[18,19], i.e., the object’s size should not extend greatly beyond uλ/πL, where u is the distance between the object and the diffuser, λ is the wavelength, and L is the scattering media thickness^[7,20]. The key feature here compared to the traditional single-shot speckle-correlation imaging (SSCI) systems mentioned previously is the absence of the iris next to the diffuser.

Actually, the absence of an iris will lead to a smaller mean size of the speckles in the captured speckle image and result in reconstruction failure, as analyzed in Refs. [7,9,10,14]. Herein, with respect to these poor-quality non-iris speckle images, we propose a processing approach to achieve successful reconstructions. As shown in Fig. 1(b), first, the speckle image acquired without the iris is resized to a lower pixel resolution by the scale factor s, whose estimate method will be discussed in detail in Section 3; second, the autocorrelation operation and Fourier transform are successively performed on the resized speckle image to obtain the power spectrum of the object; third, the object can be reconstructed through phase-retrieval algorithms^[7].

The autocorrelation of the speckle image is essentially identical to the object’s autocorrelation^[7], namely, I⋆I=(O⋆O)+C, where the symbol ⋆ denotes the autocorrelation operation and C is a constant background term. Therefore, the object can be retrieved from the autocorrelation of the speckle image by a phase-retrieval algorithm. Herein, the autocorrelation of the speckle image I is calculated according to the Wiener–Khinchin theorem, i.e., I⋆I=F−1{|F{I}|2}, where the symbols F and F−1 denote the Fourier transform and the inverse Fourier transform, respectively. After that, the power spectrum of the object can be obtained by |F{O}|2=F{O⋆O}=F{I⋆I−C}. To reconstruct the object from this power spectrum, we used the Ping-pong phase-retrieval algorithm^[21], which combines two typical Fienup-type algorithms [error reduction (ER) algorithm^[22] and hybrid input-output (HIO) algorithm^[23]]. In the iteration process, the feedback parameter β in HIO is changed stepwise. The β value decreases from 3 to 1 in steps of 0.1, with each step iterated 30 times. The ER algorithm is performed after each HIO algorithm. The alternation between the HIO and the ER strategies prevents the algorithm from being stuck in local minima.

One group of reconstructed results by the Ping-pong algorithm is shown in Fig. 2. It can be observed that our non-iris SSCI system, combined with the proposed processing method, can achieve high-quality reconstruction, and even small structural details can be faithfully recovered, such as the font features of the English letters [Figs. 2(d2) and 2(d3)] and the Chinese character [Fig. 2(d4)].

The resizing process is performed by resizing the original speckle image to a lower pixel resolution by a proper scale factor. For a scale factor s (0<s≤1), the resizing of an original speckle image I0 of M×N pixels (M and N denote the numbers of rows and columns) returns an image that is s times the size of I0, i.e., sM×sN. The interpolation method in the resizing process is not unique. Generally, bilinear interpolation (triangular kernel)^[24], cubic interpolation (cubic kernel)^[25], and box interpolation (box-shaped kernel)^[26] all work well in our tests. Herein, the box interpolation was used.

It is worth noting that the resizing processing plays a pivotal role in successful reconstructions, which can significantly improve the signal-to-noise ratio (SNR) of the speckle pattern’s autocorrelation and the corresponding power spectrum for the subsequent reconstruction process. As shown in Fig. 2, the raw speckle images of 6000pixels×6000pixels [Fig. 2(a1)–2(a4)] were all shrunk to a size of 1800pixels×1800pixels [Fig. 2(b1)–2(b4)] by a scale factor of s=0.3. Visually, the size and the SNR of the speckle grains are enhanced after resizing because the raw little speckle grains in a spatial neighborhood merge into one in the process of image resizing.

To analyze this problem quantitatively, we compare the autocorrelation of the speckle images at different scale factors with the autocorrelation of the original object (symbol “5” of Fig. 2). Furthermore, the peak signal-to-noise ratio (PSNR) is introduced to evaluate the quality of the autocorrelation at different scale factors. The object’s autocorrelation [Fig. 3(a1)], i.e., the ideal autocorrelation, serves as the reference image for this full-reference assessment approach.

As shown in Fig. 3, the autocorrelation of the raw non-iris speckle image [see Fig. 3(a5)] in fact shows poor quality and structural similarity visually and quantitatively compared with the ideal autocorrelation [Fig. 3(a1)]. As a result, its corresponding power spectrum [Fig. 3(b5)] is of poor quality as well, whose low spatial frequency component (i.e., the central region of the power spectrum) is blurred compared with that of the ideal power spectrum [Fig. 3(b1)]. Since the autocorrelation of the raw speckle image (or the corresponding power spectrum) is the input of the subsequent reconstruction algorithm, the reconstruction distortion or even failure [see Fig. 3(c5)] can be expected.

Though the resizing process further enlarges the size of the merged pixel, surprisingly, the autocorrelation’s PSNR indices of the resized speckle images turn out to be higher and vary with the scale factor, as shown in Fig. 3(e). This implies that the resizing process indeed improves the structural similarity between the autocorrelation of the speckle image and the object. Furthermore, a high-fidelity reconstruction requires a power spectrum with sharp low-frequency components and enough high-frequency components. At this point, compared with the ideal power spectrum, an excessively low scale factor will cause loss of a high spatial frequency component (outer region of the power spectrum), as shown in Figs. 3(b2) and 3(d). Meanwhile, an excessively high scale factor will cause distortion of low spatial frequency component, as shown in Figs. 3(b4), 3(b5), and 3(d). As shown in Fig. 3(b3), a suitable scale factor will lead to a high-quality power spectrum that shows great structural similarity with the ideal one for both low and high spatial frequency components.

In more detail, the autocorrelation’s PSNR indices of the resized speckle image by different scale factors are shown in Fig. 3(e), in which the scale factor of 0.3 leads to the highest PSNR index. Consequently, experimental results with a scale factor of 0.3 achieve high-quality reconstructions, as shown in Fig. 2. In fact, in our experiments the reconstructed results at a scale factor of 0.2 or 0.4 are satisfactory as well, which implies that the proposed method is robust for the scale factor and has great flexibility.

To estimate the suitable scale factor s for a non-iris SSCI system, the relationship between the pixel size and the mean radius of the speckle grains for successful reconstructions was analyzed. The equivalent pixel size after resizing by s can be represented by Δx′=Δx/s, in which Δx is the original size of the detector pixel, while the speckle grains’ mean radius δ¯ is determined by δ¯=1.22λv/D, according to the statistical optics theories^[13]. Rather than satisfying the condition that the mean diameter of speckle grains needs to be at least twice as large as the detector’s pixel size, i.e., Δx/δ¯≤1, as mentioned in Ref. [7], herein, we found that in the case of Δx/δ¯>1, the SNR of the speckle autocorrelation can be significantly improved and lead to successful reconstruction (as shown in Figs. 2 and 3) by introducing an additional resizing procedure at a suitable scale factor s.

To provide an appropriate scale factor s for an arbitrary non-iris SSCI system, we conducted a series of non-iris SSCI experiments with various detectors of different pixel sizes and various diffusers, and analyzed the statistics law of the relationship between the pixel size and the mean radius of the speckle grains for successful reconstructions. We found that the speckle autocorrelation’s improvement depends on the ratio of the equivalent pixel size after resizing and the mean speckle radius, i.e., C=Δx′/δ¯. Furthermore, when C was set to 6 by adjusting s, the speckle autocorrelation’s improvement turned out to be the most significant and led to the best reconstructions. When C was in the range of 5 to 7, the reconstructed results were satisfactory as well, which implies that the proposed method shows a good robustness for s or C.

In a word, for an arbitrary non-iris SSCI system, one can estimate the appropriate scale factor s as follows: s=1CΔx1.22λv/D.(1)C represents the ratio of the resized pixel size and the mean speckle radius, whose best empirical value is 6; due to its robustness, when C ranges from 5 to 7, successful reconstructions can be realized as well.

To verify this conclusion, we performed a series of experiments with detectors of different pixel sizes, including Dhyana 400BSI (pixel size 6.5 µm, 2048pixels×2048pixels) and Dhyana 95V2 (pixel size 11 µm, 2048pixels×2048pixels). The experimental results are shown in Fig. 4, where the diffuser is Thorlabs DG20-220-MD, with a diameter of 5 cm. In Fig. 4, all the scale factors s were calculated by Eq. (1) under the condition of C=6 and then rounded to multiples of 0.1 due to its robustness. Despite the poor quality of all the raw non-iris speckle images, by estimating the scale factor s by Eq. (1) and then resizing the speckle images by this s, high-quality reconstructions can be achieved under the conditions of different detector pixel sizes and various image distances v, as shown in Fig. 4. Note that in our calculations, s is simply set to 1 when its calculated value is larger than 1.

In addition, we tested the influence of diffuser surface roughness on the appropriate scale factor s. As described in Ref. [13], when the characteristic size of the diffuser’s surface roughness is much larger than the wavelength, the speckle size will be independent of the diffuser’s surface roughness. Here, we conducted a series of experiments with diffusers of different surface roughnesses, including 45 µm (DG20-120-MD, Thorlabs), 36 µm (DG20-220-MD, Thorlabs), and 16 µm (DG20-600-MD, Thorlabs)^[20], and the results demonstrate that the appropriate scale factor is invariant with the surface roughness of the diffuser.

More importantly, instead of digital resizing after image capture, the resizing procedure actually can be performed in the capture and storage process by on-chip pixel binning, which will further reduce the read-out noise and increase the frame rate significantly. Pixel binning is a technique that combines data from nearby sensor photo sites prior to analog-to-digital conversion and read-out in the imaging and storage process. The function of pixel binning is similar to digital resizing but can realize better SNR and a higher frame rate^[27]. Herein, we conducted a series of experiments to compare the performance of pixel binning with that of digital resizing. The experiment setup is the same as in Fig. 1(a).

First, the scale factor s was calculated by Eq. (1) under the condition of C=6; then, instead of digital resizing after capture, pixel binning of [1s]×[1s] was performed in the capture and storage process, i.e., [1s]×[1s] individual pixels nearby were binned and read out as one large pixel, in which [·] rounds a number to the nearest integer. In these experiments, the distance u was set to 35 cm to satisfy the memory-effect range, and the scale factors were 0.5, 0.3, and 0.25, respectively; their corresponding binned pixel numbers were thereby 2×2, 3×3, and 4×4. Because the resizing process had been completed in the acquisition process, the objects were reconstructed directly from the autocorrelation of the binned speckle image by phase-retrieval algorithm, as shown in Figs. 5(e1)–5(e3). Pixel binning can remarkably improve the frame rate of SSCI, as expected. For comparison, the results of digital resizing are listed in Figs. 5(c1)–5(c3).

It is worth noting that the most suitable scale factor for a non-iris SSCI system depends on the system parameters, such as the wavelength, the size of the pixels, the distance between the camera and the diffuser, and the diameter of illuminated area of the diffuser. Thus, one can estimate the appropriate scale factor for resizing by Eq. (1) based on the system parameters above; or more simply, one can perform resizing on the raw speckle image by decreasing the scale factor from 1 to 0.1 in a specific step to find the suitable value that achieves the best reconstruction. In addition, the experimental results imply that the object information is contained in the spatial distribution of the speckle grains rather than in each single speckle grain; therefore, to achieve successful SSCI reconstruction, the sizes of speckle grains in the captured speckle images in fact do not need to satisfy the Shannon–Nyquist sampling.

In summary, we propose a non-iris speckle-correlation imaging method. The absence of the iris can increase light flux, improve noncooperation and flexibility, and reduce hardware requirements for an SSCI system; however, it will lead to reconstruction distortion or even failure. By introducing a special resizing process on the non-iris speckle images, the SNR and the structural similarity of the speckle pattern’s autocorrelation compared with the ideal autocorrelation are significantly improved, which can help to achieve successful reconstructions. In addition, we put forward the approach to estimate the most appropriate scale factor for a non-iris SSCI system. The experimental results demonstrate that our method can achieve high-quality noncooperative speckle-correlation imaging by digital resizing or on-chip pixel binning; the latter can remarkably improve the frame rate of SSCI. The proposed method will benefit the noncooperative high-frame-rate speckle-correlation imaging, as well as scattering imaging for dynamic objects hidden behind opaque barriers.