Visualizing objects that are obstructed by intervening obstacles can significantly enhance situational awareness and facilitate critical decision-making processes in fields like surveillance, search and rescue, and autonomous navigation. Non-line-of-sight (NLOS) imaging analyzes multi-bounced photons to estimate scenes outside the direct line of sight [1]. The indirectly scattered photons can be analyzed in transient or steady-state scenarios [2]. Transient NLOS imaging employs ultrafast pulsed lasers to provide controlled illumination of hidden scenes and samples hidden objects by measuring the returning photon arrival rate as a function of time [3]. The reconstruction process primarily relies on the time-of-flight measurements of the indirectly scattered photons. Various techniques are employed, including non-confocal approaches of filtering for back-projection [4], confocal approaches like light-cone transform [5] and f–k migration [6], as well as other methods such as Fermat paths of light [7] and phasor-field virtual wave optics [8,9]. The transient method has made significant progress in long-distance imaging [10] and improving detection efficiency [11]. Nonetheless, the implementation of transient NLOS imaging necessitates the deployment of pulsed laser apparatuses and single-photon avalanche diode detectors for meticulous scanning and sampling of concealed objects. This requirement unavoidably escalates the expenses associated with equipment acquisition. Moreover, photons that undergo multiple bounces diminish substantially by several orders of magnitude, thereby demanding a sophisticated and iterative sampling process.

The hardware implementation of steady-state NLOS imaging is simple and efficient, which only requires a low-cost continuous-wave (CW) laser and a conventional camera [12]. Intensity-based steady-state NLOS imaging methods achieve hidden object reconstruction by solving an optimization problem that incorporates physical modeling to establish a relationship between camera measurements and the hidden object. A computational periscopy is proposed for object reconstruction, which uses an occluder for light transfer matrix improvement [13]. Nevertheless, this method is contingent upon the presence of an occluder, and the imaging quality is heavily influenced by the precise estimation of the occluder. A computational framework is proposed for steady-state NLOS localization via establishing a mapping relationship between camera measurements and the hidden object [14]. While achieving high reconstruction accuracy, this approach necessitates a substantial amount of training data, and further improvements are required to enhance generalization and robustness for unknown scenes.

By leveraging the speckles, coherence-based steady-state NLOS imaging methods can extract valuable information about the hidden object without an occluder and generalize to unknown scenarios. In coherence-based methods, the wall is commonly treated as a scattering medium [2], where incident light generates a speckle. Previous holographic imaging experiments have demonstrated that a stationary medium does not eliminate the spatial information within the light field. As a result, seemingly random speckles encode information about the hidden object [15]. Numerous studies have focused on reconstructing hidden objects using speckles [16,17]. The optical memory effect (OME), which depends on the angular dependence of speckles, is a widely explored physical phenomenon [18]. Based on the OME, non-invasive imaging through scattering media can be achieved by employing autocorrelation of speckles and phase retrieval algorithms [19]. However, it is essential to note that the range of the OME constrains the imaging field of view (FOV), with the detectable hidden scene typically confined to a few millimeters [20,21]. The generation of speckles in this approach relies on the narrow-band illumination from the hidden object. It does not fully use the broad-band spectrum characteristics of visible light [22]. The wavefront shaping method is also an important idea to solve the NLOS imaging problem [23,24]. By treating the hidden object as a guide star, the speckles illuminated on the hidden object can be focused into a sharp point [25]. The hidden scene can be reconstructed through raster scanning. Tens of minutes are needed for iteration and scanning [26], causing high requirements on the stability of the system.

A novel coherence-based steady-state NLOS method called indirect imaging correlography (IIC) [27] has been shown to have the potential to make full use of the speckles and simplify the sampling process. In the IIC, the intermediary wall serves as a virtual light source and detector, allowing the reconstruction of the hidden object using intensity measurements of light [28,29]. Studies have revealed that the Gaussian beam illuminating the intermediary wall scatters and indirectly illuminates the hidden object. The speckles generated by the object reflect a portion of the incident light onto the wall within the camera FOV, encoding the motion of the object [30]. A clustering algorithm developed from the statistical and geometric characteristics of speckles enables the tracking of multiple independently moving hidden objects by modeling the formation and motion of speckles [31]. A back-projection method is developed to build a 3D confidence map for the voxelized object space to find the voxel with the maximum confidence as the object position in the reconstructed trajectory at the corresponding video time [32]. Utilizing the neuromorphic event sensor, which captures binary information of the intensity changes with a much higher temporal resolution, fast-moving hidden object motion tracking is accurately achieved [10,33]. Although the above methods illustrate the usability of the system, it is still limited to motion tracking and does not realize the imaging of the hidden object. To achieve the reconstruction of the hidden object, the intensity correlation of multiple speckles on the wall in spatial mode is found and exploited [34–36]. The light source used in the approaches is still a single narrow-spectrum laser, and at least dozens of camera images need to be collected, which cannot resist the interference caused by system vibration and colored stains on the wall. By building a noise model of the IIC, the deep inverse correlography is proposed [37]. With the trained deep convolutional neural network noisy phase retrieval method, only two images with an exposure time of 0.125 s are needed, and the hidden object can be reconstructed. This approach needs to acquire at least two camera images and perform mechanical scanning on a virtual source. The mechanical scanning process necessitates multiple shot acquisitions, resulting in low imaging efficiency and susceptibility to vibration perturbation. There is still room for simplification in the sampling process. Imaging with full use of multi-wavelength lasers proved to allow fast and accurate reconstruction of objects [38–40]. The chromato-axial memory effect of laser speckles provides novel tools for controlling wavefields blindly around corners using multiple wavelengths [41].

This paper proposes a single-shot NLOS imaging method based on chromato-axial differential correlography. By leveraging the correlation distribution of laser speckles on the chromato-axis, the proposed method allows for encoding and sampling of the hidden object within a single exposure time. The autocorrelation estimate of the hidden object can be calculated from a single-shot image using spatial ensemble averaged correlography. The single-shot acquisition ensures the stability of the wall background, while the speckle illumination with chromato-axial difference provides redundant information across different spectral bands, enabling accurate reconstruction of the hidden object. Consequently, the autocorrelation estimate remains robust to vibration perturbation and colored stains. To assess the stability achieved through the single-shot acquisition, imaging experiments were conducted under vibration perturbation applied to the wall. Furthermore, imaging experiments involving four types of colored stains on the wall were performed to validate the redundancy of the reconstruction information provided by the speckle illumination with the chromato-axial difference. The simulation and practical experiments illustrated the capability of the proposed system to reconstruct both structurally simple and complex objects, achieving a structural similarity (SSIM) index of about 0.6. The proposed single-shot NLOS imaging method based on chromato-axial differential correlography contributes valuable insights towards advancing real-time and practical scenarios for steady-state NLOS imaging.

The principle diagram of the proposed single-shot NLOS imaging method based on chromato-axial differential correlography is illustrated in Fig. 1(a). The CW laser sources with wavelengths of 450 nm, 520 nm, and 640 nm are combined and indirectly illuminated on the hidden object through the intermediary wall, regarded as a virtual source (VS). The hidden object reflects part of the incident light to another part of the wall, and the camera with the lens is focused on the wall, regarded as a virtual detector (VD) to collect the secondary speckles.

The schematic sketch of the system is shown in Fig. 1(b). First, the forward model of the system under a single laser source is constructed. The laser source with wavelength λn is expanded by the optical path and used to illuminate the VS. At this point, the coherent beam is diffused by the wall, creating a random phase difference [37]. The resulting optical field EVS output by the VS is a laser speckle [15], which can be represented as EVS(zVS,λn)=ejΦVS(zVS,λn),(1)where ΦVS is a random phase factor generated by the wall diffusion, and zVS is the position of the VS.

The light field generated by the VS propagates over a certain distance and illuminates onto the hidden object, producing a laser speckle field EO that encodes the albedo r(zO) of the hidden object: EO(zO,λn)=EVS(zO,λn)r(zO),(2)where zO is the position of the hidden object, and EVS(zO,λn)=ejkzjλz∬∞EVS(zVS,λn)ejπλz(z−zVS)2 is the laser speckle field propagated from the VS to the hidden object.

EO continues to propagate to the VD, captured by the camera. Since the average interval of laser speckle spots is small enough, the propagation process between the hidden object and the VD can be regarded as far-field propagation [42]. The optical field EVD at the VD can be expressed as EVD(λn)=c1F[EO(zO,λn)].(3)

Intensity image I captured by the camera can be expressed as I(λn)=c2|F[EO(zO,λn)]|2+η.(4)

In the above two equations, c1 and c2 are constant terms, and η is the noise brought by the wall background. As the wall approximates a Lambertian surface, the noise at different wavelengths can be approximately equal.

The above model can be improved by combining the chromato-axial difference shown in Fig. 1(c). Equation (1) represents the speckle produced by the laser diffused through the wall. According to the memory effect of the scattering medium [18,41,43], the correlation coefficient between different speckles produced by diffusion can be expressed as C=[2πθLλn·sinh⁡(2πθLλn)]2,(5)where L is the effective thickness of the diffuse medium, θ is the angle of the incident light, and λn is the wavelength of the incident light wave. By constantly varying λn, a sufficiently wide wavelength interval is achieved, ensuring that the laser speckles created by the VS can be regarded as completely uncorrelated.

This work utilizes CW laser sources with wavelengths of 450 nm, 520 nm, and 640 nm. The resulting laser speckles, displaying the chromato-axial difference, are depicted in the middle of Fig. 1(c). The correlation coefficients between the chromato-axial differential speckles are computed, resulting in values of 0.0095 for 450 nm and 520 nm, 0.0039 for 520 nm and 640 nm, and 0.0014 for 450 nm and 640 nm. Considering a threshold value of 0.5, it can be concluded that these correlation coefficients indicate a complete lack of correlation between the speckles. The uncorrelation of the speckles ensures that each speckle pattern carries independent information, maximizing the information content obtained from a single-shot acquisition. It enables the encoding of unique information about the hidden object in each speckle pattern, which can be later extracted and utilized for the reconstruction process.

Under the same system, the spot size of the laser speckle produced by the wall changes as the wavelength of the laser source changes. The spot size of the laser speckle can be modeled using the Fresnel diffraction equation [44] dspeckle=2.44λnLDs,(6)where dspeckle represents the diameter of the speckle spots, and Ds represents the aperture size of the scattering system. By adjusting the size of the illuminated area on the VS, Ds can be changed to offset the effect of the wavelength change, to maintain the size of speckle spots constant.

Therefore, the size of the illumination area of the different wavelength lasers on the wall is adjusted by combining Eq. (6). As shown in Fig. 1(c), the hidden object is encoded by laser speckles with different wavelengths: {I(λ1)=c2|F[EVS(zO,λ1)r(zO)]|2+η,I(λ2)=c2|F[EVS(zO,λ2)r(zO)]|2+η,……I(λN)=c2|F[EVS(zO,λN)r(zO)]|2+η.(7)

Each chromato-axial differential speckle encodes the hidden object in a different random encoding and contains approximately the same wall background noise η.

This section introduces a single-shot hidden object reconstruction method utilizing the camera image containing chromato-axial differential information of the hidden objects. The reconstruction process comprises two steps: hidden object autocorrelation estimation and phase retrieval [45,46]. The autocorrelation estimation method aims to obtain the Fourier amplitude of the hidden object. By employing algorithms such as the Gerchberg–Saxton algorithm [47], the hybrid input–output (HIO) algorithm [48], the alternating direction method of multipliers based method [49], or deep learning network models [50] for retrieving the Fourier phase, the hidden object can be reconstructed. This work employs the widely used HIO algorithm in the reconstruction process. The detailed process of the reconstruction method is illustrated in Fig. 2(a).

The autocorrelation estimate method is based on the following equation from the deep inverse correlography method [37]: $$\begin{gathered} limN→∞1N∑n=1N|F−1[I(λn)]|2 \\ =r⋆r(ΔzO)+δ(ΔzO)[∫z1∞r(z1)dz1]2, \end{gathered}$$(8)and $$\begin{gathered} limN→∞|1N∑n=1NF−1[I(λn)]|2 \\ =δ(ΔzO)[∫z1∞r(z1)dz1]2, \end{gathered}$$(9)where N denotes the number of chromato-axial differential speckles utilized in the single-shot NLOS imaging system instead of the number of periodograms of the speckle images [37]. Letting Eq. (8) subtract Eq. (9), a preliminary autocorrelation value can be estimated.

After observing the preliminary autocorrelation value, it can be noticed that the autocorrelation contains a large number of noisy points similar to salt and pepper noise. According to Eq. (8), as N, the number of differential speckles used, increases, the estimated autocorrelation becomes more precise. To mitigate the negative impact of noise on the phase retrieval results, the captured camera image is segmented into several patches, the preliminary autocorrelation values in the patches are calculated, and ensemble averaging is performed. This approach is to sacrifice the spatial resolution of the image to improve the accuracy of the phase retrieval algorithm to reconstruct the hidden object.

As shown in Fig. 2(b), a series of segment experiments demonstrate the effect of the number of patches on the reconstruction results of the hidden object. When the number of patches is less than 16, the calculated autocorrelation estimate contains much noise and cannot reconstruct the hidden object. When the number of patches exceeds 64, obtaining a high-quality reconstruction result through the phase retrieval algorithm is difficult because the spatial resolution of a single patch is too low. Considering the reconstruction effect and spatial resolution, we finally segment the camera image into 64 patches for calculation.

To validate the capability of the proposed method in reconstructing hidden objects, a numerical simulation of the system for the single-shot NLOS imaging method is conducted based on the established system model in Section 2. Near-field propagation is modeled using the Fresnel diffraction formulation combined with angular spectrum theory. Far-field propagation is modeled using the Fraunhofer diffraction formula. It covers the entire process from the initial interaction with the hidden object to the outgoing process. Simulated camera images are generated, and autocorrelations of the objects are compared with autocorrelation estimates obtained from the simulated camera images. The numerical simulation results showcasing character and fashion objects are presented in Fig. 3.

Observing Fig. 3(a), we can easily find that for a character object with a simple structure, the single-shot NLOS imaging method can accurately estimate the autocorrelation of the object. With the HIO algorithm, the hidden object can be reconstructed with high precision. The complex objects used in the simulation are fashion images with complex structures and grayscale distributions. The result of the simulation experiment is presented in Fig. 3(b). For complex objects, since the single-shot NLOS imaging method is based on the estimation of object autocorrelation, it may result in the loss of high-frequency information, making it challenging to reconstruct the details of the hidden object. Nevertheless, the overall structure of the object can still be restored, and even a portion of the grayscale distribution can be recovered. This restoration serves as a valuable physical prior, which can be utilized for subsequent reconstructions using other methods to achieve higher accuracy.

After conducting numerical simulations, an experimental system is constructed to validate the practical imaging performance of the proposed single-shot NLOS imaging method. The system setup is depicted in Fig. 4. Three 300 mW, CW laser sources of 450 nm, 520 nm, and 640 nm are combined and directed towards the VS, generating chromato-axial differential speckles that illuminate the hidden object. A portion of the speckles is reflected from the hidden object to the VD. A camera (MV-CA050-12UC, 2056×2464 pixels Sony IMX264 RGB) with a 50 mm focal length lens focused on the VD captures the single-shot image. The size of the hidden object used in the experiment is approximately 10 mm, with a distance of about 300 mm from the VS and approximately 400 mm from the VD. The distance between the camera and the VD is roughly 500 mm. The difference in incident power of chromato-axial differential speckles causes a decrease in autocorrelation contrast; the intensity is normalized according to the spectral response curve of the used CMOS. The size of the VS is expressed by the aperture diameter Ds in Eq. (6) and affects the quality of the estimated autocorrelation. The size of the VD is the size of the camera’s FOV area on the intermediary wall, which affects the spatial resolution of the system [27]. The experiment demonstrates the effectiveness of the proposed single-shot NLOS imaging method for autocorrelation estimation and reconstruction of various hidden objects measuring 10 mm in size. A comparison between the experimental results obtained using the multi-shot method [37], which involves mechanical scanning, and the proposed single-shot method based on chromato-differential correlography is presented in Figs. 5(b) and 5(c).

In the experiment, each image has an exposure time of 1 s, and the camera FOV covers an area of approximately 20mm×20mm. SSIM is used to evaluate the imaging quality of the proposed method. It is not difficult to find that the proposed single-shot NLOS imaging method can reconstruct the hidden object with an SSIM average of about 0.6, which can achieve imaging quality similar to that of the compared multi-shot method. In terms of reconstruction details, the proposed method is slightly weaker than the multi-shot method in reconstructing the curved parts of “3” and “5,” and the overhead part of the vertical line in “5” has not been reconstructed. The reason for the difference may be that the speckle correction process changes the shape of the curved portion of the estimated autocorrelation. The proposed method is better than the multi-shot method in reconstructing the sharp corners and straight continuity of “N” and “F” due to the better estimated autocorrelation. This approach eliminates the need for multiple exposure measurements and mechanical scanning, offers substantial simplification of the sample steps, and has notable savings in data acquisition time. The improved efficiency of the single-shot method enables rapid and accurate reconstruction of hidden objects, enhancing the overall practicality and usability of NLOS imaging systems.

In practical observation scenarios, the observation environment is often dynamic, and various factors such as wall vibrations and active light source disturbances can introduce disturbances into the imaging process. An experiment is conducted to assess the robustness of the proposed single-shot NLOS method under system vibration perturbation.

Figure 6(a) illustrates the introduction of system vibration perturbation into both the multi-shot method and the proposed single-shot method based on chromato-axial differential correlography. Specifically, the wall is subjected to periodic vibrations with a period of 0.5 s and an amplitude of 0.1 mm. As shown in Fig. 6(b), the estimated autocorrelation of the hidden object in the multi-shot method is completely distorted due to the changes in the background caused by the vibrating wall. However, due to the stable background noise, the proposed method can more accurately calculate the estimated autocorrelation of the hidden object, resulting in improved reconstruction performance.

The system vibration perturbation experiment demonstrates the superior performance of the proposed single-shot NLOS method in the presence of system vibration perturbation. By leveraging the stability of background noise in one exposure measurement, the method can effectively mitigate the adverse effects caused by wall vibrations, leading to more accurate autocorrelation estimation and improved reconstruction results.

Walls in practical scenes often deviate from ideal Lambertian surfaces, as they may contain small areas of colored stains or even large areas with colored surfaces. The chromato-perturbation poses a challenge for NLOS imaging methods that rely on the assumption of Lambertian properties of the intermediary wall. A series of relevant experiments is conducted to assess the robustness of the proposed single-shot NLOS imaging based on chromato-axial differential correlography under wall chromato-perturbation.

In the wall chromato-perturbation experiment, watercolor paints are used to introduce four types of perturbations onto the intermediary wall. The first three types of chromato-perturbation correspond to the laser sources employed in this work. The fourth chromato-perturbation involves using a dark pigment to simulate more challenging scenarios. The spectral curve of the perturbed wall is illustrated in Fig. 7(b), while the experimental results under different chromato-perturbations are presented in Fig. 7(c).

It is observed that under perturbations correlated to the laser sources used in the first three experiments, the estimated autocorrelation and object reconstruction results remain stable. The stable results can be attributed to the fact that while some chromato-axial differential speckles may be influenced by the reflectivity and absorptivity of the intermediary wall, the remaining chromato-axial differential speckles still provide sufficient information for accurate autocorrelation estimation of the hidden object. In the fourth case, where all the chromato-axial differential speckles are affected, the estimated autocorrelations are disturbed. However, due to the redundancy of the collected information, the reconstructed result can still be obtained, albeit with some degradation.

The wall chromato-perturbation experiment demonstrates the robustness of the proposed method for colored stains on the wall. Even when facing challenging scenarios, where the intermediary wall deviates from Lambertian properties, the method can still provide stable estimation and reconstruction results by leveraging the redundant information encoded in the chromato-axial differential speckles.

In this section, we discuss the cons and pros of the proposed method and give outlooks on its further applications. (i)Our method is proposed to tackle the challenges of active steady-state NLOS imaging. With the chromato-axial differential scattering prior, the method achieves precise reconstruction of the hidden object with one exposure measurement. Single-shot acquisition greatly simplifies the sampling process of the system; combined with the hardware advantages of a steady-state system, it makes it possible to miniaturize NLOS imaging devices.(ii)The background stability caused by single-shot acquisition and the information redundancy caused by chromato-axial differential speckle illumination provide our method with robustness to vibration and colored stains. However, deficiencies still exist. The aliasing of the different channels of the camera causes a reduction in the contrast of the autocorrelation estimate. The encoding method based on chromato-axial differential correlography makes it difficult to image a color object. Ambient light noise and more complex wall chromato-perturbation can also impact the imaging capabilities of the system.(iii)To further enhance the proposed method, several improvements can be considered. Applying the FastICA method [51] with appropriate constraint values can separate the signals from different camera channels, enhancing the signal-to-noise ratio and thereby improving the overall imaging quality of the system. By replacing the encoding method of the illumination, it becomes feasible to reconstruct hidden objects with complex color information [52,53]. In addition, the application of white-light illumination can be explored for ambient light interference and more complex wall chromato-perturbation. Adaptive sampling and improved reconstruction accuracy can be achieved by utilizing illumination with a more comprehensive wavelength range [54], and dynamic band selection [55]. By achieving these enhancements, the proposed method can provide solutions for breaking through the limitation of the visible line of sight in biomedical microscopy imaging [56,57], detection in semiconductor material production [58], etc.

In this paper, a single-shot NLOS imaging method based on chromato-axial differential correlography is proposed. The proposed method can estimate the autocorrelation of the hidden object and reconstruct it with a structural similarity of about 0.6 using only one exposure measurement. By adequately segmenting the camera image, the accuracy of the hidden object autocorrelation estimates can be improved for the phase retrieval algorithm. Simulating character objects and fashion objects of different complexities verifies the generality of the system. Furthermore, the proposed method has been compared with the multi-shot method in the practical system. Distinct advantages such as shorter acquisition time and more vital anti-perturbation ability have been achieved. The proposed method provides valuable insights for the further development of NLOS imaging.

Acknowledgment. The authors thank Chenyin Zhou, Yi Wei, Jinye Miao, and Yingjie Shi for technical support and experiment discussion.