• Acta Photonica Sinica
  • Vol. 50, Issue 2, 76 (2021)
Yongping HAO1, Zhaorui CAO1、*, Fan BAI1, Haoyang SUN1, Xing WANG2, and Jie QIN1
Author Affiliations
  • 1College of Equipment Engineering, Shenyang Ligong University, Shenyang059, China
  • 2College of Mechanical Engineering, Shenyang Ligong University, Shenyang110159, China
  • show less
    DOI: 10.3788/gzxb20215002.0210002 Cite this Article
    Yongping HAO, Zhaorui CAO, Fan BAI, Haoyang SUN, Xing WANG, Jie QIN. Research on Infrared Visible Image Fusion and Target Recognition Algorithm Based on Region of Interest Mask Convolution Neural Network[J]. Acta Photonica Sinica, 2021, 50(2): 76 Copy Citation Text show less
    Network structure of IVFNN
    Fig. 1. Network structure of IVFNN
    Feature extraction network structure of IVFNN
    Fig. 2. Feature extraction network structure of IVFNN
    Input image changes in the process of dual channel adaptive fusion and attention logic code calculation
    Fig. 3. Input image changes in the process of dual channel adaptive fusion and attention logic code calculation
    Comparison of imaging contribution of infrared and visible light targets in different environments
    Fig. 4. Comparison of imaging contribution of infrared and visible light targets in different environments
    Adaptive fusion process of infrared and visible feature maps group
    Fig. 5. Adaptive fusion process of infrared and visible feature maps group
    Feature maps of each channel before and after fusion
    Fig. 6. Feature maps of each channel before and after fusion
    The process of calculating the thermal radiation type type of image point
    Fig. 7. The process of calculating the thermal radiation type type of image point
    Attention network in visible channel
    Fig. 8. Attention network in visible channel
    Feature maps of each channel before and after adding the logical mask of ROI
    Fig. 9. Feature maps of each channel before and after adding the logical mask of ROI
    Schematic diagram of CIOU in IVFNN
    Fig. 10. Schematic diagram of CIOU in IVFNN
    Cross loss calculation of dual channels and fused images
    Fig. 11. Cross loss calculation of dual channels and fused images
    Loss function curve of IVFNN
    Fig. 12. Loss function curve of IVFNN
    Recognition effect of IVFNN and each frequency band
    Fig. 13. Recognition effect of IVFNN and each frequency band
    Recognition effect of low-heat radiation target
    Fig. 14. Recognition effect of low-heat radiation target
    Imaging frequency bandMinimum pixel of recognizableMinimum identifiable target pixel ratioLimit accuracy of minimum target recognition
    Infrared370×232 pixels6.98%65.3%
    Visible light452×292 pixels10.7%69.5%
    Dual channels adaptive fusion of IVFNN248×179 pixels3.61%60.7%
    Table 1. Multi-scale target recognition ability of IVFNN and each frequency band

    Algorithm used

    Processes

    Average recognition rate

    Average calculation speed

    Ref.[2]

    Feature matching and fusion

    70.6%

    35 fps

    Ref.[3]

    Filtering fusion

    77.9%

    37 fps

    Ref.[6]

    Filtering fusion

    76.1%

    32 fps

    Ref.[7]

    Neural network fusion

    80.5%

    22 fps

    Ref.[10]

    Neural network fusion

    81.3%

    23 fps

    Proposed algorithm

    Global IVFNN

    83.2%

    25 fps

    Table 2. Comparison of test results of each algorithm
    Yongping HAO, Zhaorui CAO, Fan BAI, Haoyang SUN, Xing WANG, Jie QIN. Research on Infrared Visible Image Fusion and Target Recognition Algorithm Based on Region of Interest Mask Convolution Neural Network[J]. Acta Photonica Sinica, 2021, 50(2): 76
    Download Citation