Fig. 1. Convolutional neural network structure
Fig. 2. Cross entropy loss function
Fig. 3. Infrared image ''UN Camp''and images after saliency extraction by various methods. (a) Infrared image "UN Camp"; (b) Standard segmentation of image "UN Camp"; (c) AC method; (d) SR method; (e) LC method; (f) FT method; (g) CNN method; (h) CNN+FT method
Fig. 4. Infrared image ''dune'' and images after saliency extraction by various methods. (a) Infrared image "dune"; (b) Standard segmentation of image "dune"; (c) AC method; (d) SR method; (e) LC method; (f) FT method; (g) CNN method; (h) CNN+FT method
Fig. 5. Image fusion model based on convolutional neural network and NSST
Fig. 6. ''UN Camp'' infrared and visible images and fusion results. (a) Infrared image; (b) Visible image; (c) DWT method; (d) CS method; (e) BEMD method; (f) NSCT+FL method; (g) NSST+FL method; (h) Proposed method; (i) Significant area fusion image
Fig. 7. ''dune'' infrared and visible images and fusion results. (a) Infrared image; (b) Visible image; (c) DWT method; (d) CS method; (e) BEMD method; (f) NSCT+FL method; (g) NSST+FL method; (h) Proposed method; (i) Significant area fusion image
Fig. 8. ''iron'' infrared and visible images and fusion results. (a) Infrared image; (b)Visible image; (c) DWT method; (d) CS method; (e) BEMD method; (f) NSCT+FL method; (g) NSST+FL method; (h) Proposed method; (i) Significant area fusion image
Method | AC | SR | LC | FT | CNN | CNN+FT | MAE1 | 15.64 | 1.00 | 8.59 | 9.34 | 0.50 | 0.27 | MAE2 | 8.45 | 0.55 | 2.91 | 2.91 | 0.63 | 0.35 |
|
Table 1. Target significance extraction evaluation index MAE
Image | Method | E | AG | SF | MI | CE | UN | DWT | 6.934 2 | 7.049 2 | 13.921 7 | 2.668 3 | 0.356 3 | Camp | CS | 6.252 7 | 4.965 9 | 10.300 5 | 1.593 3 | 0.599 9 | | BEMD | 6.603 8 | 6.146 2 | 12.007 7 | 1.573 8 | 0.575 9 | | NSCT+FL | 6.810 5 | 7.110 9 | 14.126 5 | 2.432 8 | 0.326 4 | | NSST+FL | 6.855 5 | 7.051 6 | 14.475 8 | 2.355 0 | 0.279 8 | | Proposed method | 7.116 3 | 8.008 2 | 16.106 9 | 2.991 9 | 0.264 8 | Dune | DWT | 6.657 4 | 6.507 5 | 12.371 8 | 2.594 3 | 0.307 2 | | CS | 5.903 8 | 4.694 8 | 9.790 9 | 1.188 4 | 0.654 5 | | BEMD | 6.156 6 | 5.426 7 | 10.391 4 | 1.196 5 | 0.576 0 | | NSCT+FL | 6.675 8 | 6.578 3 | 15.540 9 | 2.222 0 | 0.292 5 | | NSST+FL | 6.666 7 | 7.365 3 | 13.954 2 | 2.637 0 | 0.301 4 | | Proposed method | 6.701 1 | 7.386 8 | 14.153 9 | 2.815 7 | 0.289 2 | Iron | DWT | 6.677 8 | 12.647 6 | 33.730 4 | 3.617 3 | 0.550 3 | | CS | 6.537 2 | 7.647 5 | 20.025 4 | 3.177 3 | 0.498 7 | | BEMD | 6.677 7 | 9.090 6 | 23.349 1 | 3.392 5 | 0.539 7 | | NSCT+FL | 6.767 7 | 14.998 1 | 38.969 7 | 3.397 5 | 0.474 5 | | NSST+FL | 6.751 1 | 15.645 4 | 40.410 4 | 3.177 4 | 0.427 9 | | Proposed method | 6.768 7 | 16.282 2 | 41.718 8 | 3.701 3 | 0.409 3 |
|
Table 2. Infrared and visible image fusion effect evaluation