Depth map resolution improvement for 3D range-intensity correlation laser imaging

Xiaoquan Liu; Xinwei Wang; Yinan Cao; Songtao Fan; Yan Zhou; Yuliang Liu

doi:10.3788/COL201513.071102

Abstract

This Letter proposes a high bit-depth coding method to improve depth map resolution and render it suitable to human-eye observation in 3D range-intensity correlation laser imaging. In this method, a high bit-depth CCD camera with a nanosecond-scaled gated intensifier is used as an image sensor; subsequently two high bit-depth gate images with specific range-intensity profiles are obtained to establish the gray depth map and finally the gray depth map is encoded by an equidensity pseudocolor. With this method, a color depth map is generated with higher range resolution. In our experimental work, the range resolution of the depth map is improved by a factor of 1.67.

3D imaging, such as integral imaging^[1], digital holography^[2], ghost imaging^[3], and range-gated imaging^[4–17] have made impressive progress in recent years. Among them, 3D range-gated laser imaging (3DRGLI) has been proven to have great potential in various applications, such as underwater imaging^[4–7], remote sensing^[8], and long-range detection^[9]. Up until now, three approaches of depth scanning^[9,10], range-intensity correlation^[11–13] (or super-resolution depth mapping), and gain modulation^[14,15] have been reported to improve real-time performance and range precision and accuracy. Compared with the other two 3DRGLI approaches, 3D range-intensity correlation laser imaging not only can reduce raw data volume and simplify data processing with great real-time performance, but also has higher range precision, larger depth-to-resolution ratio, and lower signal-to-noise ratio (SNR)^[8,16,17].

In practical applications of 3D range-intensity correlation laser imaging, the gray depth map is always low-contrast, blurring, and degenerate due to backscattering and a hostile environment, such as underwater, fog, and rain. Equidensity pseudocolor encoding of the gray depth map is a good choice as the human eye is much more sensitive to color than to gray. However, to our knowledge, most established equidensity pseudocolor methods can only be applied to images with a bit-depth no more than 8^[18]. At the same time, the gray value of each pixel in the gray depth map should be expressed by a float with 32 bits depth or a double with 64 bits depth. This is because the gray value in the gray depth map represents distance information. As distance of 3DRGLI varies from a few meters to several thousand meters; the gray value in gray depth map varies over the same range. To equidensity pseudocolor encode a high bit-depth depth map, the general approach is simply to compress the high bit-depth depth map to 256 gray levels before equidensity pseudocolor encoding, which decreases range precision and accuracy.

How to improve depth map resolution, or render the depth map more suitable to human-eye observation and improve range precision and accuracy at the same time, has not yet been presented in previous works. We investigated the formation process of a depth map in 3D range-intensity correlation laser imaging as well as the means by which to overcome this drawback. We propose a high bit-depth coding method. With this method, a color depth map is generated and the depth map resolution is improved as well. In this Letter, depth map resolution includes range precision and accuracy.

Sign up for Chinese Optics Letters TOC. Get the latest issue of Chinese Optics Letters delivered right to you！Sign up now

In range-intensity correlation 3D laser imaging, the first step is to obtain gate images with specific shape range-intensity profiles (RIPs). Two algorithms based on the 3D range-intensity correlation laser imaging have been developed based on different RIPs of gate images. One is a trapezoid algorithm^[11] with trapezoid RIPs of gate images, and the other is a triangular algorithm^[12] with triangular RIPs of gate images. When the laser pulse width equals the gate time, gate images with triangular RIPs are obtained. When the gate time is 2 times larger than the laser pulse width, gate images with trapezoid RIPs are obtained. The second step is to establish a depth map utilizing spatial range-intensity correlation of two gate images, which can refer to previous reports of Wang et al.^[12] and Laurenzis et al.^[11,13]. Taking the triangular algorithm as an example, the target range can be denoted as $r_{i, x, y} = \frac{τ_{i} c}{2} + \frac{I_{i + 1, x, y}}{I_{i + 1, x, y} + I_{i, x, y}} \frac{t_{L} c}{2},$ (1)where $r_{i, x, y}$ is the gray value of pixel $(x, y)$ in the gray depth map and it equals the target range numerically, $τ_{i}$ is the time delay between the laser pulse and the gate pulse in the $i$ th gate image, c is the light speed in vacuum, $t_{L}$ is the laser pulse width which equals the gate time $t_{g}$ , $I_{i, x, y}$ is the gray value of pixel $(x, y)$ in the $i$ th gate image, and $I_{i + 1, x, y}$ is gray value of pixel $(x, y)$ in the $(i + 1)$ th gate image with a time delay of $(τ_{i} + t_{L})$ .

Based on the rules of error propagation and considering time jitter and noise disturbance, from Eq. (1) the range precision under the range-intensity correlation method with triangular RIPs is $σ_{r_{i, x, y}} = \frac{c}{2} [σ_{τ_{i}}^{2} + \frac{I_{i + 1, x, y}^{2}}{{(I_{i + 1, x, y} + I_{i, x, y})}^{2}} σ_{t_{L}}^{2} + \frac{t_{L}^{2} I_{i, x, y}^{2}}{{(I_{i, x, y} + I_{i + 1, x, y})}^{2}} σ_{I_{i + 1, x, y}}^{2} {+ \frac{t_{L}^{2} I_{i + 1, x, y}^{2}}{{(I_{i, x, y} + I_{i + 1, x, y})}^{2}} σ_{I_{i, x, y}}^{2}]}^{1 / 2},$ (2)where $σ_{τ_{i}}^{2}$ is the variance of the delay time $τ_{i}$ , $σ_{t_{L}}^{2}$ is the variance of the laser pulse width $t_{L}$ , and $σ_{I_{i + 1, x, y}}^{2}$ and $σ_{I_{i, x, y}}^{2}$ are the variance of $I_{i + 1, x, y}$ and $I_{i, x, y}$ , respectively. Terms $σ_{τ_{i}}$ and $σ_{t_{L}}$ are both induced by time jitter, and $σ_{I_{i + 1, x, y}}^{2}$ and $σ_{I_{i, x, y}}^{2}$ are caused by noise disturbance from atmospheric optical turbulence, ambient background, and sensor noise.

Apparently from Eq. (1), depth information is derived from $I_{i + 1, x, y}$ and $I_{i, x, y}$ in gate images, and precise $I_{i + 1, x, y}$ and $I_{i, x, y}$ contribute to precise depth information and improvement of range precision and accuracy. A high bit-depth CCD camera samples deeper in the depth domain of signal intensity, which results more precise $I_{i + 1, x, y}$ and $I_{i, x, y}$ in gate images. Consequently, we propose a high bit-depth coding method by which the range precision and accuracy can be improved. In this method, a high bit-depth CCD camera with a nanosecond (ns)-scaled gated intensifier is used as an image sensor; subsequently two high bit-depth gate images with specific RIPs are obtained to establish the gray depth map, and finally a gray depth map with high range precision and accuracy is encoded by equidensity pseudocolor. Given that an equidensity pseudocolor encoding method applicable to high bit-depth images has not been rigorously established, one can compress it to 256-level gray (8 bit-depth) linearly first. For the compressed 8-bit gray depth map, there are various established methods to equidensity pseudocolor encoding, such as continuous pseudocolor coding^[19], pixel self-transformations^[20], and the gray level transformation to color method^[21]. We obtained a color depth map which is more suitable to human-eye observation, and range precision and accuracy of depth map are improved as well.

For our work, we developed a 3DRGI system that consists of a pulsed laser as an illuminator, a gated camera, a timing control unit, a PC, and software as illustrated in Fig. 1. The illuminator is realized by a laser diode with a center wavelength of 808 nm, and its laser pulse width can be changed from 10 ns to several μs under the trigger of the timing control unit. The gated camera is an intensified CCD (ICCD) that consists of a gated GEN II image intensifier acting as a gate with a minimal gate time of 30 ns, and a CCD chip with $1392 pixels \times 1040 pixels$ . The CCD chip has two bit-depth modes of 8 and 12 bit, and can output 8 bit gate images and 12 bit gate images, respectively. The image intensifier is coupled to the CCD by a fiber. The timing control unit realized by field-programmable gate array can provide the desired time sequence for the pulsed laser and the ICCD. The software is realized by C++ programing language and consists of not only a control console for the entire system, but also a 3D reconstruction algorithm and an equidensity pseudocolor encoding algorithm.

Figure 1.Schematic diagram of the experiment setup.

In our work, the illuminator has a pulse width of 40 ns, a pulse repetition frequency (PRF) of 50000, and the average power is 8 mW. The gated image intensifier has the same repetition frequency as the pulsed laser, i.e., 50000, and the gate time is 40 ns, which is the same as the pulse width. Two time delays corresponding to two gate images are set at 100 and 140 ns. The CCD has a frame rate of 15 frames/s and an exposure time of 40 ms. The 3D reconstruction algorithm is a triangular algorithm in the range-intensity correlation method and the equidensity pseudocolor encoding method for the compressed gray depth map is the gray level transformation to color method. The targets are three white flat cardboards at distance of 16, 17, and 19 m from the imaging system in sequence, which can be distinguished from gate images illustrated in Figs. 2 and 3. The work was carried out in an indoor hallway to eliminate the disturbance of atmospheric turbulence.

Figure 2.Experiments results under 12 bit-depth mode: (a) and (b) gate images with different time delay; (c) color depth map; (d) trace across Line A–B of (c).

Figure 3.Experiments results under 8 bit-depth mode: (a) and (b) gate images with different time delay; (c) color depth map; (d) trace across Line A–B of (c).

The work under 12 bit-depth mode was performed as shown in Fig. 2. Figures 2(a) and 2(b) are 12-bit gate images and captured with camera working in 12 bit-depth mode. To decrease the impact of noise, we introduce a threshold pixel value in the gate images below which the range cannot be reliably determined, and those pixels are simply removed from the depth map. The color depth map in Fig. 2(c) is obtained by an equidensity pseudocolor encoding compressed gray depth map using the gray level transformation to color method, and the compressed gray depth map is deduced from gate images in Figs. 2(a) and 2(b). Figure 2(d) is the trace across line A–B corresponding to the 520th row in Fig. 2(c).

Figure 3 is a comparison experiment with a camera working in 8 bit-depth mode. Except the camera working in 8 bit-depth mode, 8 bit gate images in Figs. 3(a) and 3(b) are captured under the same condition as Figs. 2(a) and 2(b). Using the same equidensity pseudocolor encoding method and preprocessing method as in Fig. 2(c), Fig. 3(c) is deduced from the gate images in Figs. 3(a) and 3(b). Figure 3(d) is the trace across line A–B corresponding to the 520th row in Fig. 3(c).

The trace across lines in Figs. 2(c) and 3(c) are corresponding and should cover all three targets. Under these conditions, the present lines are randomly chosen. In fact, there are no significant differences in performance of the corresponding lines that cover three targets.

Color depth maps in Figs. 2(c) and 3(c) are both high contrast and suitable to human-eye observation. However, comparing targets designated by Ellipse 1 in Figs. 2(c) and 3(c), it can be seen that Fig. 2(c) has more smooth targets with more detail. This phenomenon demonstrates that 12 bit gate images have more detailed samples of the echo signal in the depth domain. More detailed samples in the depth domain contribute to an improvement in the range accuracy and precision, which can be proved by tracing across Line A–B in Figs. 2(c) and 3(c). Apparently Fig. 2(d) is very close to the actual range of 16, 17, and 19 m, and Fig. 3(d) has a range deviation from reality and a large range fluctuation due to the impact of noise. This phenomenon proves that high bit-depth gate images improve range accuracy in 3D range-intensity correlation laser imaging.

To compare range precision directly under two bit-depth modes, the range standard deviation is taken as an evaluation and trace across Lines A–B of Figs. 2(c) and 3(c) are analyzed. First we analyze the range standard deviation in theory. Figure 4(a) is predicted by Eq. (2) under two bit-depth modes, showing that the range precision (or range standard deviation) of the 12 bit-depth mode and 8 bit-depth mode are about 0.3 and 0.5 m, respectively. In the calculation of the range precision using Eq. (2), $σ_{τ_{i}}$ and $σ_{t_{L}}$ are both 1 ns and the other parameters are calculated from the gate images. Second, we prove an improvement of range precision under the 12 bit-depth mode experimentally. The experiments of Figs. 2 and 3 are performed another 5 times and another five depth maps are obtained. Standard deviations of $r_{i, x, y}$ in two bit-depth modes are obtained from the six depth maps as illustrated in Fig. 4(b), which also demonstrates that the range precision of 12 bit-depth mode and 8 bit-depth mode are respectively about 0.3 and 0.5 m. In accordance with Fig. 4, Eq. (2) predicts well compared with experiment results. The range precisions under 12 bit-depth mode and 8 bit-depth mode are about 0.3 and 0.5 m, respectively, and the former is about 1.67 times better than the latter.

Figure 4.Comparison of range precision under two bit-depth modes: (a) range precision predicted in theory; (b) range precision obtained from experiment.

In conclusion, we investigate the formation process of the depth map in 3D range-intensity correlation laser imaging, and propose a high bit-depth coding method to improve the depth map resolution, rendering it suitable to human-eye observation. In this method, a high bit-depth CCD camera with a ns-scaled gated intensifier is used as an image sensor; subsequently two high bit-depth gate images with specific RIPs are obtained to establish the gray depth map, and finally a gray depth map with high range precision and accuracy is encode by an equidensity pseudocolor. With this method, the depth map is more suitable to human-eye observation, and range precision and accuracy in the depth map are improved at the same time. Our work demonstrates that not only is the depth map high-contrast and suitable to human-eye observation, but also the range precision of the depth map is improved by a factor of 1.67. These results are of crucial importance for practical applications in 3D range-intensity correlation laser imaging.

References

[1] W. Xie, Y. Wang, H. Deng, Q. Wang. Chin. Opt. Lett., 12, 011101(2014).

[2] X. Cao, X. Sang, Z. Chen, Y. Zhang, J. Leng, N. Guo, B. Yan, J. Yuan, K. Wang, C. Yu. Chin. Opt. Lett., 12, 080901(2014).

[3] Y. Zhu, J. Shi, H. Li, G. Zeng. Chin. Opt. Lett., 12, 071101(2014).

[4] E. McLean, H. Burris, M. Strand. Appl. Opt., 34, 4343(1995).

[5] J. Busck. Opt. Eng., 44, 116001(2005).

[6] F. Christnacher, M. Laurenzis, D. Monnin, G. Schmitt, N. Metzger, S. Schertzer, T. Scholz. Proc. SPIE, 9250, 92500F(2014).

[7] X. Wang, Y. Cao, C. Liu, Q. Kong, W. Cui, Y. Zhou, Y. Li. Infrared Laser Eng., 43, 2584(2014).

[8] X. Wang, Y. Cao, W. Cui, X. Liu, S. Fan, Y. Zhou, Y. Li. Proc. SPIE, 9260, 9260L(2014).

[9] P. Andersson. Opt. Eng., 45, 034301(2006).

[10] J. Busck, H. Heiselberg. Appl. Opt., 43, 4705(2004).

[11] M. Laurenzis, F. Christnacher, D. Monnin. Opt. Lett., 32, 3146(2007).

[12] X. Wang, Y. Li, Y. Zhou. Appl. Opt., 52, 7399(2013).

[13] M. Laurenzis, E. Bacher. Appl. Opt., 50, 3824(2011).

[14] X. Zhang, H. Yan, Y. Jiang. Opt. Lett., 33, 1219(2008).

[15] C. Jin, X. Sun, Y. Zhao, Y. Zhang, L. Liu. Opt. Lett., 34, 3550(2009).

[16] X. Zhang, H. Yan, Q. Zhou. Chin. Opt. Lett., 9, 061101(2011).

[17] M. Laurenzis, A. Woiselle. Opt. Eng., 53, 053106(2014).

[18] M. Cao, D. Yu. Opt. Technol., 34, 115(2002).

[19] W. Li. J. Beijing Inst. Technol. (Eng. Ed.), 6, 37(1997).

[20] M. Cao, D. Yu, N. Sun. Proceedings of Tenth National Conference on Image and Graphics for China Society of Image and Graphics and First National Seminar on Virtual Reality, 5(2011).

[21] C. G. Rafacel, E. W. Richard. Digital Image Processing(2007).