• Infrared and Laser Engineering
  • Vol. 51, Issue 4, 20210393 (2022)
Jinhui Han1, Yantao Wei2、*, Zhenming Peng3, Qian Zhao1, Yaohong Chen4, Yao Qin5, and Nan Li1
Author Affiliations
  • 1College of Physics and Telecommunication Engineering, Zhoukou Normal University, Zhoukou 466001, China
  • 2Faculty of Artificial Intelligence in Education, Central China Normal University, Wuhan 430079, China
  • 3School of Information and Communication Engineering, University of Electronic Science and Technology of China, Chengdu 611731, China
  • 4Xi’an Institute of Optics and Precision Mechanics, Chinese Academy of Sciences, Xi’an 710119, China
  • 5Northwest Institute of Nuclear Technology, Xi’an 710024, China
  • show less
    DOI: 10.3788/IRLA20210393 Cite this Article
    Jinhui Han, Yantao Wei, Zhenming Peng, Qian Zhao, Yaohong Chen, Yao Qin, Nan Li. Infrared dim and small target detection: A review[J]. Infrared and Laser Engineering, 2022, 51(4): 20210393 Copy Citation Text show less
    (a) Single-frame infrared image containing small target; (b) 3D mesh of different positions
    Fig. 1. (a) Single-frame infrared image containing small target; (b) 3D mesh of different positions
    Classify of infrared dim and small target detection methods
    Fig. 2. Classify of infrared dim and small target detection methods
    General framework of background estimation method
    Fig. 3. General framework of background estimation method
    Structure window of new Top-hat operator
    Fig. 4. Structure window of new Top-hat operator
    Gradient distribution of different locations. (a) Normal background; (b) Background edge; (c) Real target
    Fig. 5. Gradient distribution of different locations. (a) Normal background; (b) Background edge; (c) Real target
    Contrast mechanisms in the human visual system
    Fig. 6. Contrast mechanisms in the human visual system
    Construction of DoG filter template. (a) Sharp Gaussian kernel function; (b) Flat Gaussian kernel function; (c) Difference of them, i.e., DoG filtering template
    Fig. 7. Construction of DoG filter template. (a) Sharp Gaussian kernel function; (b) Flat Gaussian kernel function; (c) Difference of them, i.e., DoG filtering template
    Double-layer window used in LCM algorithm
    Fig. 8. Double-layer window used in LCM algorithm
    (a) Infrared image of real scene; (b) Result image computed by local contrast method; (c) 3D show of contrast map
    Fig. 9. (a) Infrared image of real scene; (b) Result image computed by local contrast method; (c) 3D show of contrast map
    Two kinds of common frequency-domain detection methods
    Fig. 10. Two kinds of common frequency-domain detection methods
    (a) Clustering; (b) SVM; (c) Neural network
    Fig. 11. (a) Clustering; (b) SVM; (c) Neural network
    Some dictionary atoms obtained using 2D Gaussian function
    Fig. 12. Some dictionary atoms obtained using 2D Gaussian function
    (a) Original infrared image; (b) Low-rank background obtained by RPCA; (c) Sparse foreground obtained by RPCA
    Fig. 13. (a) Original infrared image; (b) Low-rank background obtained by RPCA; (c) Sparse foreground obtained by RPCA
    Construction and decomposition of image patch matrix
    Fig. 14. Construction and decomposition of image patch matrix
    MethodsTypical algorithmsResearch trends
    Single-frame methodsMethods on based local informationBackground estimationMedian filtering; Max-mean filtering; Max-median filtering; TDLMS adaptive filtering, etc From simple traditional filtering methods to adaptive filtering methods (such as TDLMS, etc.)
    Morphological methodsNew tophat, etc.1. Design more appropriate structural windows; 2. Use other information (such as local significance, local entropy, etc.) for weighting
    Directional derivative/ gradient methodsDirectional derivative; High order directional derivative; Gradient, etc. 1. Obtain the directional derivative using facet kernel; 2. Introduce higher derivative; 3. Combined with other methods (such as contrast), etc.
    Local contrastDifference-form LCM; Ratio-form LCM; Ratio-difference LCM, etc. 1. Transition from difference type or ratio type to ratio-difference joint type; 2. Improve the calculation window; 3. Use more information (such as local entropy, local signal to clutter ratio, etc.) as the weighting function; 4. Combined with other types of algorithms (such as frequency domain algorithm, background estimation algorithm, etc.)
    Methods based on nonlocal informationFrequency-domain methodsHigh pass filter; Low pass filter; Fourier transform; Wavelet transform, etc. 1. Transition from simple Fourier transform and high/low pass filter to wavelet analysis; 2. Combined with other types of algorithms (such as morphological methods)
    ClassifierClustering; Support vector machine; Neural network, etc. 1. Optimize classification parameters (such as distance threshold in clustering algorithm, mapping method in support vector machine, etc.); 2. Design a more suitable classification network (mainly for neural network method); 3. Combined with other types of algorithms (such as the combination of depth network and contrast algorithm)
    Over complete sparse representationSparse representation; Group sparse representation, etc. 1. Construct an appropriate over complete target dictionary; 2. Construct the over complete dictionary for the target and background respectively; 3. Combined with other types of algorithms (such as frequency domain algorithm, etc.)
    Sparse and low-rank decomposition (mainly refers to RPCA) RPCA for image matrix; RPCA for image patch; RPCA for tensor, etc. 1. Improve the data organization method (from image matrix to image patch or tensor); 2. Improve the relaxation conditions (from simple 1-norm, nuclear norm to more complex relaxation conditions); 3. Introduce regularization factors (such as total variation); 4. Improve the iterative solution process (such as protecting large singular values, or introducing weighting factors, etc.) 5. Combined with other types of algorithms (such as contrast algorithm, etc.)
    Multi-frame methodsAssociated checkingSingle-frame detection first, then correlation check between multi framesPipeline filtering; Energy accumulation; Markov theory; Kalman filtering; Particle filter, etc. From simple energy accumulation and time-domain verification to more complex verification theories and methods, such as Markov theory, Kalman filter, particle filter, and so on
    Directed calculation3D search3D matched filter; Dynamic programming, etc. From simple time-domain calculation (such as 3D matched filtering, dynamic programming, temporal variance, temporal difference, etc.) to spatial-temporal joint calculation (such as spatial-temporal contrast, spatial-temporal tensor, etc.)
    1D processingTemporal variance; Temporal power law; Temporal inner product; Temporal profile, etc.
    Inter-frame differenceDifference between two adjacent frames; Difference between multi adjacent frames, etc.
    Spatial-temporal methodsSpatial-temporal local contrast; Spatial-temporal tensor, etc.
    Table 1. Typical examples of different types of algorithms and main improvements of existing research
    MethodsAdvantagesDisadvantages
    Single-frame methodsMethods based on local informationBackground estimation1. The principle and the calculation are both simple; 2. It is in good agreement with imaging theory When the background is very complex, it is easy to get an inaccurate estimation
    Morphological methodsThe structural window can be designed according to the characteristics of the target to improve the detection performanceWhen the target is very weak, it is easy to erase the target by mistake
    Directional derivative/ gradient methods It expands the characteristics of target and background, and makes it easier to distinguish themThe mathematical definition is strict, so it is difficult to design the formula
    Local contrast1. The definition is flexible and the formula can be designed pertinently; 2. Target enhancement and complex background suppression can be considered simultaneously; 3. The principle is simple and easy to implement It is required that the target must be the most prominent locally, otherwise it cannot be detected (for example, when the target is close to the highlighted background)
    Methods based on nonlocal informationFrequency-domain methodsIt is a transform domain algorithm with complete theory and clear physical meaningWhen the background is very complex, the complex background also contains high-frequency information, which is easy to interfere with target detection
    Classifier1. It belongs to machine learning algorithm, and has strong adaptability; 2. The target is not required to be the most prominent locally. 1. Prior knowledge is generally required (training samples are required); 2. It is required that the samples and the data to be tested are distributed in the same way
    Over complete sparse representation 1. It belongs to machine learning algorithm, and has strong adaptability; 2. The target is not required to be the most prominent locally. 1. Prior knowledge is generally required; 2. It is impossible to build a dictionary that covers all situations
    Sparse and low-rank decomposition (mainly refers to RPCA) 1. It belongs to machine learning algorithm, and has strong adaptability; 2. The target is not required to be the most prominent locally. 3. No prior knowledge is required When the background is very complex, some background edges and noise are easily incorrectly decomposed into sparse matrix to interfere with target detection
    Multi-frame methodsAssociated checkingSingle-frame detection first, then correlation check between multi framesMulti frame information can be used to further eliminate false targetsThe detection performance largely depends on the detection results of single-frame algorithm
    Directed calculation3D searchThe target position and target trajectory can be determined at the same timeThe motion information of the target should be taken as a priori knowledge
    1D processing1. The principle is simple and easy to implement 2. Good parallel ability Only applicable to the scenes of fixed background and moving target
    Inter-frame differenceThe principle is simple and easy to implementOnly applicable to the scenes of fixed background and moving target
    Spatial-temporal methodsUsing multi frame information to achieve good detection performance1. Only applicable to the scenes of fixed background and moving target 2. Large amount of calculation and output lag
    Table 2. Advantages and disadvantages of different types of algorithms
    Jinhui Han, Yantao Wei, Zhenming Peng, Qian Zhao, Yaohong Chen, Yao Qin, Nan Li. Infrared dim and small target detection: A review[J]. Infrared and Laser Engineering, 2022, 51(4): 20210393
    Download Citation