• Infrared and Laser Engineering
  • Vol. 51, Issue 3, 20210139 (2022)
Kewei Huan, Xiangyang Li, Yutong Cao, and Xiao Chen
Author Affiliations
  • College of Physics, Changchun University of Science and Technology, Changchun 130022, China
  • show less
    DOI: 10.3788/IRLA20210139 Cite this Article
    Kewei Huan, Xiangyang Li, Yutong Cao, Xiao Chen. Infrared and visible image fusion of convolutional neural network and NSST[J]. Infrared and Laser Engineering, 2022, 51(3): 20210139 Copy Citation Text show less
    Convolutional neural network structure
    Fig. 1. Convolutional neural network structure
    Cross entropy loss function
    Fig. 2. Cross entropy loss function
    Infrared image ''UN Camp''and images after saliency extraction by various methods. (a) Infrared image "UN Camp"; (b) Standard segmentation of image "UN Camp"; (c) AC method; (d) SR method; (e) LC method; (f) FT method; (g) CNN method; (h) CNN+FT method
    Fig. 3. Infrared image ''UN Camp''and images after saliency extraction by various methods. (a) Infrared image "UN Camp"; (b) Standard segmentation of image "UN Camp"; (c) AC method; (d) SR method; (e) LC method; (f) FT method; (g) CNN method; (h) CNN+FT method
    Infrared image ''dune'' and images after saliency extraction by various methods. (a) Infrared image "dune"; (b) Standard segmentation of image "dune"; (c) AC method; (d) SR method; (e) LC method; (f) FT method; (g) CNN method; (h) CNN+FT method
    Fig. 4. Infrared image ''dune'' and images after saliency extraction by various methods. (a) Infrared image "dune"; (b) Standard segmentation of image "dune"; (c) AC method; (d) SR method; (e) LC method; (f) FT method; (g) CNN method; (h) CNN+FT method
    Image fusion model based on convolutional neural network and NSST
    Fig. 5. Image fusion model based on convolutional neural network and NSST
    ''UN Camp'' infrared and visible images and fusion results. (a) Infrared image; (b) Visible image; (c) DWT method; (d) CS method; (e) BEMD method; (f) NSCT+FL method; (g) NSST+FL method; (h) Proposed method; (i) Significant area fusion image
    Fig. 6. ''UN Camp'' infrared and visible images and fusion results. (a) Infrared image; (b) Visible image; (c) DWT method; (d) CS method; (e) BEMD method; (f) NSCT+FL method; (g) NSST+FL method; (h) Proposed method; (i) Significant area fusion image
    ''dune'' infrared and visible images and fusion results. (a) Infrared image; (b) Visible image; (c) DWT method; (d) CS method; (e) BEMD method; (f) NSCT+FL method; (g) NSST+FL method; (h) Proposed method; (i) Significant area fusion image
    Fig. 7. ''dune'' infrared and visible images and fusion results. (a) Infrared image; (b) Visible image; (c) DWT method; (d) CS method; (e) BEMD method; (f) NSCT+FL method; (g) NSST+FL method; (h) Proposed method; (i) Significant area fusion image
    ''iron'' infrared and visible images and fusion results. (a) Infrared image; (b)Visible image; (c) DWT method; (d) CS method; (e) BEMD method; (f) NSCT+FL method; (g) NSST+FL method; (h) Proposed method; (i) Significant area fusion image
    Fig. 8. ''iron'' infrared and visible images and fusion results. (a) Infrared image; (b)Visible image; (c) DWT method; (d) CS method; (e) BEMD method; (f) NSCT+FL method; (g) NSST+FL method; (h) Proposed method; (i) Significant area fusion image
    MethodACSRLCFTCNNCNN+FT
    MAE115.641.008.599.340.500.27
    MAE28.450.552.912.910.630.35
    Table 1. Target significance extraction evaluation index MAE
    ImageMethodEAGSFMICE
    UNDWT6.934 27.049 213.921 72.668 30.356 3
    CampCS6.252 74.965 910.300 51.593 30.599 9
    BEMD6.603 86.146 212.007 71.573 80.575 9
    NSCT+FL6.810 57.110 914.126 52.432 80.326 4
    NSST+FL6.855 57.051 614.475 82.355 00.279 8
    Proposed method7.116 38.008 216.106 92.991 90.264 8
    DuneDWT6.657 46.507 512.371 82.594 30.307 2
    CS5.903 84.694 89.790 91.188 40.654 5
    BEMD6.156 65.426 710.391 41.196 50.576 0
    NSCT+FL6.675 86.578 315.540 92.222 00.292 5
    NSST+FL6.666 77.365 313.954 22.637 00.301 4
    Proposed method6.701 17.386 814.153 92.815 70.289 2
    IronDWT6.677 812.647 633.730 43.617 30.550 3
    CS6.537 27.647 520.025 43.177 30.498 7
    BEMD6.677 79.090 623.349 13.392 50.539 7
    NSCT+FL6.767 714.998 138.969 73.397 50.474 5
    NSST+FL6.751 115.645 440.410 43.177 40.427 9
    Proposed method6.768 716.282 241.718 83.701 30.409 3
    Table 2. Infrared and visible image fusion effect evaluation
    Kewei Huan, Xiangyang Li, Yutong Cao, Xiao Chen. Infrared and visible image fusion of convolutional neural network and NSST[J]. Infrared and Laser Engineering, 2022, 51(3): 20210139
    Download Citation