Abstract
Keywords
Introduction
Prior knowledge of the shape, size, and texture of dim small target is almost non-existent which limits the development of infrared searching and tracking systems [
Based on the above analysis, this study involved the design of a horizontal-vertical multi-scale grayscale difference (HV-MSGD) weighted bilateral filtering (BF) method to detect dim small targets using infrared imaging. The method has the following advantages: (1) the method measures the discontinuity between the target area and background area by comparing the regional standard deviation to determine the size of the window; (2) the HV-MSGD weighted operator is used to realize the expansion of the difference between the target and background area by using the size of the window to achieve target enhancement; (3) background edge information is suppressed in consideration of the difference between the distance and grayscale value of each pixel in the image; (4) the combination of global and local threshold segmentation (GLTS) prevents extremely strong signals such as those of noise and clutter to influence the target itself, which eliminates noise elements and detects the real target signal. In other words, HV-MSGD is combined with BF to improve the separability of the target and background gradient, thereby increasing the energy of the target while suppressing the edge information in the image. Experiments show that this method is superior to other algorithms in terms of its ability to detect weak targets.
1 Single frame detection of dim targets
1.1 Target signal enhancement—HV-MSGD
In an infrared image, the grayscale of the target pixels differs greatly from that of the surrounding pixels with a large discontinuity in the luminance. This discontinuity fundamentally determines that the nature of the average grayscale can be obtained from the pixels adjacent to the target. [
Small and weak targets are concentrated in a small, uniform and compact area, which has discontinuity with the surroundings. Therefore, this paper designs a method to calculate this discontinuity and achieve enhancement of weak targets. The standard deviation of the image reflects the clutter fluctuations of the images, so the appropriate window size can be selected to obtain the MSGD of the image by comparing the LG-SGD.
The image is traversed from top to bottom and left to right, and the area is divided into neighborhoods surrounding the pixels. As shown in
Figure 1.
where is the number of differential regions with values k=1,2,3, ..., k, the set represents the differential region, represents the number of pixels in region , and represents the grayscale of pixels in the region . In actual operation, the sizes of windows 3, 5, 7, 9, and 11 are primarily used for calculations and selections. Then the local standard deviation in the kth region is as follows:
According to Eq. 2, for a heterogeneous region, the local grayscale standard deviation is large when the window is small, whereas the local grayscale standard deviation is small for a homogeneous region in the same window.
Figure 2.The response of different boundaries at different scales
The determined window size is used to obtain the grayscale value of the predicted point by using the HV-MSGD, and enhancement of the target area is achieved by the difference in response between the horizontal and vertical boundaries. The specific implementation is shown in
Figure 3.calculate the HV-MSGD weighting operator
where is the corresponding value of the region, the horizontal grayscale value is obtained after the vertical gradient difference is calculated, then the final target enhancement value is obtained by calculating the horizontal grayscale gradient difference. In summary, the specific processing of this algorithm is shown in the following module:
Image | Input | BF | TDLMS | PM | ||||||
---|---|---|---|---|---|---|---|---|---|---|
BSF | GSNR | BSF | GSNR | BSF | GSNR | |||||
Image 1 | 2.78 | 30.05 | 2.15 | 6.56 | 8.19 | 8.5 | 1.73 | 3.26 | ||
Image 2 | 1.77 | 42.87 | 4.08 | 13.12 | 18.72 | 7.98 | 1.33 | 3.12 | ||
Image 3 | 1.79 | 59.95 | 1.82 | 4.33 | 6.37 | 4.57 | 1.77 | 4.33 | ||
Image 4 | 1.13 | 36.73 | 1.38 | 8.19 | 11.77 | 4.92 | 0.69 | 1.5 | ||
Image 5 | 1.16 | 29.26 | 7.1 | 20.89 | 31.46 | 14.24 | 2.56 | 7.12 | ||
Image | LCM | NWIE | Our method | |||||||
BSF | GSNR | BSF | GSNR | BSF | GSNR | |||||
Image 1 | 2.63 | 7.7 | 4.55 | 1.59 | 13.48 | 40.99 | ||||
Image 2 | 2.63 | 8.89 | 4.43 | 18.94 | 21.33 | 71.37 | ||||
Image 3 | 1.61 | 2.98 | 2.72 | 16.98 | 11.73 | 27.53 | ||||
Image 4 | 1.09 | 1.92 | 8.46 | 7.82 | 20.63 | 12.65 | ||||
Image 5 | 4.41 | 4.59 | 21.52 | 21.32 | 121.92 | 131 |
Table 1. BSF and GSNR of different five images processed by different methods
1.2 Background estimation—BF
Bilateral filtering can smooth the image while estimating the background of the image. The filter employs two weights: the filter coefficient determined by the geometric spatial distance and the filter coefficient determined by the difference in grayscale similarity. In the sampling procedure, which considers the relationship between pixels by using the spatial distance and grayscale similarity difference, the two coefficients are expressed as follows:
The weight coefficient is composed of the two:
where and are the bandwidth coefficients of the two filter coefficients, is grayscale gray of the region of , then the background estimation can be obtained:
Subtracting the result of the background estimation from the original image, the background suppression image is obtained as follows:
where the is the final background suppression image. It is worth noting that the image needs to be normalized during the calculation process. Then the candidate target is extracted by GLTS.
1.3 Threshold segmentation
The target and its background are discontinuous[
Figure 4.The detection process of the entire dim small target
The threshold segmentation process adopts a method that combines global and local threshold segmentation to determine dim small targets. The global threshold segmentation is an adaptive threshold segmentation for the entire image and is obtained as follows:
where is the standard deviation of the image, m is the average of the image, and is an odd number greater than 3. Then we can get the image of global threshold segmentation by Eq. 11. is the grayscale value of the image processed by HV-MSGD weighted BF at point . Local threshold segmentation divides the image into regions and calculates the th segmentation threshold value for different regions respectively, using Eq. 12:
where is the standard deviation of the th partition region, =1,2,3, …, , and is the average of the th partition region. Then we can get the image of local threshold segmentation by Eq. 13. The algorithm is designed to detect an entire single-frame containing a small target is described in the following module. The two equations use the same threshold .
IMAGE | Frame number | Input | BF | TDLMS | PM | ||||||
---|---|---|---|---|---|---|---|---|---|---|---|
Seq 1 | 75 | 25.72 | 2.23 | 3.13 | 4.36 | 1.01 | 3.58 | 3.1 | 2.31 | ||
Seq 2 | 90 | 8.36 | 0.56 | 1.22 | 5.72 | 2.42 | 5.12 | 1.28 | 4.56 | ||
Seq 3 | 65 | 6 | 1.21 | 9.3 | 10.68 | 2.55 | 8.24 | 6.16 | 7.37 | ||
Seq 4 | 90 | 32.78 | 2.09 | 2.64 | 9.49 | 5.71 | 7.61 | 1.19 | 4.01 | ||
Seq 5 | 90 | 11.53 | 1.17 | 1.07 | 6.77 | 1.2 | 4.86 | 1.95 | 3.89 | ||
Average | - | 17.22 | 1.44 | 3.12 | 7.32 | 2.64 | 5.82 | 2.51 | 4.33 | ||
IMAGE | Frame number | LCM | NWIE | Our method | |||||||
Seq 1 | 75 | 3.1 | 1.27 | 1.98 | 6.04 | 5.38 | 33.97 | ||||
Seq 2 | 90 | 1.01 | 2.02 | 2.40 | 9.42 | 5.46 | 15.64 | ||||
Seq 3 | 65 | 4.76 | 3.13 | 1.34 | 13.73 | 14.71 | 55.18 | ||||
Seq 4 | 90 | 1.17 | 7.94 | 3.17 | 10.25 | 23.26 | 81.8 | ||||
Seq 5 | 90 | 2.94 | 1.61 | 1.74 | 13.1 | 7.47 | 47.51 | ||||
Average | - | 2.45 | 3.27 | 2.13 | 10.51 | 11.26 | 36.47 |
Table 2. the average of BSF and GSNR for five sequences
2 Trajectory detection
On the basis of the procedure for single-frame detection described in Sec. 2, further verification of the effect of the proposed method, combined with the improved UPF, is conducted by predicting the target position by the probability data association (PDA) and to finally obtain the motion trajectory. According to the actual moving speed of the target, the frame rate and other factors, setting the range of the suspected position to 10 pixels, to solve the tracking accuracy. For the improved UPF, we detect the single-frame according to Sect. 1, to obtain the positions of suspected target points, and then estimate the suspected target points probabilistically to obtain the predictive value, and according to literature [
where represents all valid measurement sets that fall within the target tracking gate at time , represents the number of effective measurement at time , is covariance matrix, is the detection probability with a value of 1, is the threshold probability with a value of 0.97. The steps of the process are shown in
Figure 5.Flow chart of trajectory detection
Seq 1 | Seq2 | Seq 3 | Seq 4 | Seq 5 | Average | |
---|---|---|---|---|---|---|
BF | 0.9200 | 0.9470 | 0.8704 | 0.9253 | 0.9525 | 0.9230 |
TDLMS | 0.7202 | 0.7427 | 0.4342 | 0.4765 | 0.9334 | 0.6614 |
LCM | 0.6374 | 0.8439 | 0.2871 | 0.9056 | 0.6738 | 0.6696 |
PM | 0.5551 | 0.7645 | 0.3061 | 0.8577 | 0.6338 | 0.6234 |
NWIE | 0.8477 | 0.9167 | 0.5578 | 0.8614 | 0.9507 | 0.8269 |
Our method | 0.9320 | 0.9577 | 0.9349 | 0.9847 | 0.9761 | 0.9571 |
Table 3. AUC of five sequences in different algorithms
3 Analysis of experimental results
3.1 Experimental environment and images
The effectiveness of the infrared dim small target detection algorithm based on HV-MSGD weighted BF was verified by using five sets of real infrared image sequences for experimental comparison. The operating environment of this experiment is MATLAB2014b on a Windows 10 (64-bit) system with 2.5 GHz CPU, an Intel Core i5 processor, and 8 GB memory. The parameters used in the experiment were =0.8, =0.3, . The choice of these three parameters needs to be determined according to the images in different scenes. controls the spatial distance, when the value is large, the effective spatial range is larger, and the edge points can also obtain larger weights, and the denoising effect is obvious. controls the grayscale change in the image. The larger grayscale difference, the higher the weight value can be obtained, but the small edge will be affected, and there is no good edge-preserving effect. mainly affects the result of threshold segmentation. When is too small, the false alarm rate will increase. When the value is too large, the true target may be removed, resulting in a missed alarm. Therefore, it is necessary to select according to actual conditions. In this manuscript, these three values are obtained by experiment based on the specific image.
Figure 6.the results of five images that processed by our method, (a) is the input image, (b) is the 3D view of the input image, (c) is the image processed by HV-MSGD, (d) is a 3D image processed by HV-MSGD, (e) is a threshold segmentation image processed by (c), (f) is a 3D threshold segmentation image processed by (c)
Figure 7.Background suppression results of different algorithms in different scenarios (a) original image, (b) BF filtering result, (c) TDLMS filtering result in literature 15, (d) PM filtering result, (e) LCM filtering result of in literature 24, (f) NWIE, (g) filtering result of our method
3.2 Evaluation method
The performance of these methods was compared by using three indicators, i.e., the gain of the signal-to-noise ratio (GSNR), background suppression factor (BSF), and receiver operating characteristic (ROC) curve for the quantitative evaluation of the background suppression and target detection performance. The indicators are defined as follows:
where and are the local SNR of the target before and after background suppression, and are the standard deviations of the local background area of the target before and after background suppression, and are grayscale peak value of the target and background areas, and is the standard deviation of the local background area.
The evaluation index (Eqs. 18-19) was used to compute BSF and GSNR of the image shown in
The ROC curve describes the mutual constraint relationship between the probability of detection (Pd) and the false alarm rate (FAR). The value of Pd for different values of FAR can be obtained by changing the detection threshold in Eqs. 10 and 12, where and are the values of Pd and FAR, calculated as follows:
where is the number of detected targets, is the number of actual targets, is the number of pixels of the false target, and is the number of all pixel points in the detection images.
Based on the description of ROC, the ROC values of the five sequences are plotted in
Figure 8.ROC of the five sequences
Table Infomation Is Not EnableTable Infomation Is Not EnableFigure 9.Track of five sequences
Figure 10.Histograms of detected bias pixels obtained by using our method (a) histograms of horizontal detected bias pixels of five sequences, (b) histograms of vertical detected bias pixels of five sequences
4 Conclusions
This study investigated the problem associated with detecting small targets that are weak infrared emitters. Our efforts mainly focused on increasing the discontinuity between the target region and the background region by using HV-MSGD, which improved the separability of the target from the background. In addition, background edge information was suppressed by BF, and the target was finally extracted by the GLTS algorithm. A comparison of the results with those of other methods on five images showed that our method is superior to other methods in both GSNR and BSF, and the enhancement effect is 6∼30 times that of the other methods. In addition, comparison of the average and of five sequences that include a total of 410 images also confirmed our method to be 5-12 times more effective than other methods. Counting the probability of detection of the five sequences also indicated our method to be significantly more accurate than other methods, with and the average detection probability of our algorithms of 95.71%. The method proposed in this paper has a good effect on the image with obvious grayscale fluctuation in adjacent regions. For images with small fluctuations, the method of this paper needs further improvement. In future research, we plan to develop a more effective detection algorithm for multi-target complex background and occlusion based on the single-frame detection method proposed in this paper.
References
[1] C Q Gao, D Y Meng, Y Yang. Infrared patch-image model for small target detection in a single image. IEEE Transactions on Image Processing, 22, 4996-5009(2013).
[2] K P Luo. Space-based infrared sensor scheduling with high uncertainly: Issues and challenges. Syst. Eng., 18, 102-113(2015).
[3] F Gao, H Li, T Li. Infrared small target detection in compressive domain. Electron. Lett., 50, 510-512(2014).
[4] C Q Gao, T Q Zhang, Q Li. Small infrared target detection using sparse ring representation. IEEE Aerospace and Electronic Systems Magazine, 27, 21-30(2012).
[5] X Yang, Y P Zhou, D K Zhou. A new infrared small and dim target detection algorithm based on multi-directional composite window. Infrared Phys. Technol., 71, 402-407(2015).
[6] X P Shao, H Fan, G X Lu. An improved infrared dim and small target detection algorithm based on the contrast mechanism of human visual system. Infrared Phys. Technol., 55, 403-408(2012).
[7] P Wang, J W Tian, C Q Gao. Infrared small target detection using directional high pass filters based on LS-SVM. Electron. Lett., 45, 156-158(2009).
[10] Y Li, Y Song, Y F Zhao. An infrared target detection algorithm based on lateral inhibition and singular value decomposition. Infrared Physics & Technology, 85, 238-245(2017).
[11] Z M Chen, M C Tian, Y M Bo. Improved infrared small target detection and tracking method based on new intelligence particle filter. Computational Intelligence, 34, 917-938(2018).
[12] F Zhao, H Z Lu, Z Y Zhang. Complex background suppression based on fusion of morphological Open filter and nucleus similar pixels bilateral filter. Infrared Physics & Technology, 55, 454-461(2012).
[13] Spatial and temporal bilateral filter for infrared small target enhancement. Infrared Physics & Technology, 63, 42-53(2014).
[14] H B Tian, E E Department, H V Amp. Infrared small target detection based on bilateral filter and bhattacharyya distance. Nuclear Electronics & Detection Technology, 34, 1159-1163(2014).
[15] H Deng, Y T Wei, M W Tong. Background suppression of small target image based on fast local reverse entropy operator. IET Computer Vision, 7, 405-413(2013).
[16] H Deng, J G Liu, Z Chen. Infrared small target detection based on modified local entropy and EMD. Chinese Optical Letters, 8, 24-28(2010).
[17] C J Li, Y Wei, Z L Shi. A small target detection algorithm based on multi-scale energy cross. IEEE Int. Conf. Robotics Intell. Syst. Signal Process, 2, 1191-1196(2003).
[18] X Z Bai, Y G Bi. PP(. IEEE Transactions on Geoscience & Remote Sensing, 1-15(2018).
[19] K Shang, X Sun, J W Tian. Infrared small target detection via line-based reconstruction and entropy-induced suppression. Infrared Physics & Technology, 76, 75-81(2016).
[20] Z Chen, S Luo, T Xie. A novel infrared small target detection method based on BEMD and local inverse entropy. Infrared Physics & Technology, 66, 114-124(2014).
[21] Y Mao, M Zheng, W Jia. Analysis of small target detection algorithm based on image gray entropy(2016).
[22] G H Peng, H Chen, Q Wu. Infrared small target detection under complex background. Advanced Materials Research, 346, 615-619(2011).
[23] X J Qu, H Chen, G H Peng. Novel detection method for infrared small targets using weighted information entropy. Journal of Systems Engineering and Electronics, 23, 838-842(2012).
[24] C L Philip, H Li, Y T Wei. A local contrast method for small infrared target detection. IEEE Trans. on Geoscience & Remote Sensing, 52, 574-581(2013).
[25] H Deng, X P Sun, M L Liu. Entropy-based window selection for detecting dim and small infrared targets. Pattern Recognition, 61, 66-77(2017).
[26] H Deng, X P Sun, M L Liu. Infrared small-target detection using multiscale gray difference weighted image entropy. IEEE Transactions on Aerospace & Electronic Systems, 52, 60-72(2016).
[27] G Y Wang. Efficient method for multiscale small target detection from a natural scene. Opt. Eng, 35, 761-768(1996).
[28] X P Huang, Y Wang.
Set citation alerts for the article
Please enter your email address