Fig. 1. Network model of visible-polarized image fusion algorithm
Fig. 2. Structure of Sobel Gradient Dense Block （SGDB）
Fig. 3. Structure of attention mechanism model
Fig. 4. Targets for mines used in dataset
Fig. 5. Multi-angle polarization observation devices
Fig. 6. Low-light camera
Fig. 7. Targets for mines used in datasets
Fig. 8. Fused images of different methods in sandy soil background
Fig. 9. Fused images from different methods in complex environment with multiple deciduous leaves
Fig. 10. Fused images with different equilibrium coefficients α
Fig. 11. Fused images generated by models with different attention mechanisms
Fig. 12. Effect of visible intensity images and fused images on target detection performance
Fig. 13. Detection effect of visible intensity images and fused images in part of the test set
| Methods | SSIM | MS-SSIM | PSNR | MSE | EN | SD | AG | SCD | VIF | Qabf | MI | SF |
|---|
| Our method | 0.758 5 | 0.781 3 | 27.84 | 53.21 | 5.707 | 19.68 | 3.819 | 1.597 | 0.837 3 | 0.606 5 | 2.689 | 10.69 | | CVT | 0.737 4 | 0.779 3 | 25.97 | 68.19 | 5.205 | 13.44 | 3.803 | 1.536 | 0.709 6 | 0.545 8 | 1.444 | 10.46 | | NSCT | 0.742 4 | 0.780 7 | 26.01 | 67.73 | 5.203 | 13.54 | 3.719 | 1.571 | 0.778 9 | 0.584 6 | 1.549 | 10.50 | | SR | 0.723 7 | 0.739 2 | 28.92 | 46.26 | 5.556 | 19.56 | 3.337 | 1.360 | 0.710 9 | 0.473 0 | 2.801 | 10.05 | | NSCT-ST | 0.731 2 | 0.742 6 | 28.96 | 46.21 | 5.605 | 19.57 | 3.593 | 1.411 | 0.739 2 | 0.512 6 | 2.811 | 10.49 | | FusionGAN | 0.608 0 | 0.702 3 | 24.51 | 91.86 | 5.544 | 17.92 | 2.327 | 1.175 | 0.500 6 | 0.205 3 | 1.375 | 5.499 | | DenseFusion | 0.677 2 | 0.746 3 | 24.98 | 86.82 | 5.703 | 21.95 | 3.804 | 1.461 | 0.819 3 | 0.485 2 | 2.623 | 11.14 | | PFNet | 0.746 8 | 0.770 5 | 27.38 | 57.55 | 5.381 | 19.49 | 3.711 | 1.501 | 0.746 5 | 0.565 3 | 2.544 | 14.55 |
|
Table 1. Objective evaluation results of fused images by different methods
| α | SSIM | MS-SSIM | PSNR | MSE | EN | SD | AG | SCD | VIF | Qabf | MI | SF |
|---|
| 0 | 0.755 6 | 0.784 0 | 27.18 | 52.67 | 5.317 | 18.44 | 3.346 | 1.573 | 0.799 8 | 0.545 7 | 3.082 | 9.868 | | 0.5 | 0.746 6 | 0.775 2 | 27.86 | 48.62 | 5.503 | 19.61 | 3.854 | 1.610 | 0.910 1 | 0.645 0 | 3.004 | 10.71 | | 1 | 0.745 1 | 0.775 4 | 27.99 | 47.50 | 5.512 | 19.49 | 3.858 | 1.640 | 0.910 4 | 0.645 7 | 2.948 | 10.72 | | 5 | 0.743 9 | 0.775 3 | 27.60 | 50.59 | 5.495 | 19.04 | 3.838 | 1.668 | 0.913 8 | 0.646 6 | 3.061 | 10.71 | | 10 | 0.739 7 | 0.774 5 | 26.78 | 57.95 | 5.465 | 19.03 | 3.829 | 1.660 | 0.909 5 | 0.646 7 | 3.003 | 10.68 | | 100 | 0.083 9 | 0.477 6 | 23.61 | 109.3 | 1.810 | 2.470 | 0.812 | 0.410 4 | 0.030 1 | 0.066 3 | 0.493 2 | 2.579 | | 1000 | 0.608 2 | 0.644 5 | 24.01 | 101.9 | 5.334 | 17.44 | 3.956 | 0.532 7 | 0.758 2 | 0.631 8 | 2.712 | 11.48 |
|
Table 2. Effect of balancing coefficient α on quality of fused images
| λ | SSIM | MS-SSIM | PSNR | MSE | EN | SD | AG | SCD | VIF | Qabf | MI | SF |
|---|
| 0 | 0.745 1 | 0.775 2 | 27.99 | 47.50 | 5.512 | 19.49 | 3.858 | 1.640 | 0.910 4 | 0.644 7 | 2.948 | 10.71 | | 0.1 | 0.758 5 | 0.779 9 | 27.82 | 53.55 | 5.448 | 19.67 | 3.812 | 1.592 | 0.844 4 | 0.600 3 | 2.689 | 10.72 | | 0.5 | 0.759 5 | 0.781 3 | 27.60 | 56.74 | 5.375 | 19.19 | 3.770 | 1.611 | 0.819 0 | 0.585 1 | 2.561 | 10.63 | | 1 | 0.759 1 | 0.779 1 | 27.75 | 56.14 | 5.389 | 19.66 | 3.758 | 1.554 | 0.813 1 | 0.576 7 | 2.515 | 10.64 | | 10 | 0.760 4 | 0.781 5 | 27.70 | 57.41 | 5.329 | 18.87 | 3.712 | 1.574 | 0.793 1 | 0.568 7 | 2.440 | 10.56 | | 100 | 0.761 6 | 0.783 1 | 27.54 | 58.34 | 5.302 | 18.67 | 3.695 | 1.586 | 0.788 4 | 0.565 9 | 2.431 | 10.51 | | 1 000 | 0.761 1 | 0.781 5 | 27.73 | 57.32 | 5.335 | 19.09 | 3.707 | 1.574 | 0.791 9 | 0.566 4 | 2.444 | 10.54 |
|
Table 3. Effect of different balance coefficients λ on quality of fused images
| Fusion module | SSIM | MS-SSIM | PSNR | MSE | EN | SD | AG | SCD | VIF | Qabf | MI | SF |
|---|
| Lack of attention | 0.516 2 | 0.738 6 | 24.43 | 95.86 | 3.104 | 4.154 | 0.662 4 | 0.965 2 | 0.181 0 | 0.025 1 | 1.392 | 2.067 | | Spatial attention | 0.752 4 | 0.798 6 | 25.85 | 70.37 | 4.863 | 11.47 | 2.736 0 | 1.387 0 | 0.507 1 | 0.378 0 | 1.978 | 8.059 | | Channel attention | 0.754 4 | 0.776 6 | 27.25 | 57.53 | 5.408 | 18.83 | 3.819 0 | 1.553 0 | 0.842 8 | 0.597 8 | 2.608 | 10.66 | | Mixed attention | 0.758 5 | 0.779 9 | 27.82 | 53.55 | 5.448 | 19.67 | 3.812 0 | 1.592 0 | 0.844 3 | 0.600 3 | 2.689 | 10.71 |
|
Table 4. Objective evaluation of model-generated fused images under different attention mechanisms
