Author Affiliations
1National Center for Applied Mathematics in Chongqing, Chongqing 401331, China2Guangzhou Institute of Technology, Xidian University, Guangzhou 710068, China3Shenyang Institute of Automation, Chinese Academy of Sciences, Shenyang 110169, Chinashow less
Fig. 1. (a) Framework of proposed algorithm: It contains a frequency decoupling branch, a spatial decoupling branch and a multi-frequency convolution attention block; (b) Framework of multi-frequency convolution attention block
Fig. 2. Comparison results of RoadScene dataset
Fig. 3. Comparison results of MSRS dataset
Fig. 4. Comparison results of TNO datase
Fig. 5. Visualization results of frequency domain disentangling branches
Fig. 6. Visualization results of spatial domain disentangling branches
Fig. 7. Visualization results of ablation comparison of different modules
Fig. 8. The visualization analysis and comparison of image pair "00959 N" in the MSRS dataset
Fig. 9. The visualization analysis and comparison of image pair "FLIR_04688" in the RoadScene dataset
| Method | EN↑ | SF↑ | AG↑ | MI↑ | SCD↑ | VIF↑ | $ \mathrm{\mathit{Q}}^{{{{A}{B}}}/\mathrm{\mathit{F}}} $↑ | SSIM↑ | | RFN-Nest | 7.33 | 7.85 | 3.34 | 1.90 | 1.73 | 0.50 | 0.30 | 0.78 | | SwinFusion | 7.00 | 14.36 | 5.55 | 1.79 | 1.43 | 0.6 | 0.59 | 1.02 | | SDNet | 7.32 | 15.61 | 6.19 | 2.25 | 1.45 | 0.60 | 0.51 | 1.01 | | U2Fusion | 6.80 | 11.81 | 4.77 | 1.82 | 1.20 | 0.53 | 0.47 | 0.98 | | Densefuse | 6.84 | 9.50 | 3.69 | 1.88 | 1.36 | 0.51 | 0.37 | 0.93 | | FusionGAN | 7.07 | 8.73 | 3.37 | 1.91 | 1.13 | 0.37 | 0.26 | 0.59 | | DATFusion | 6.72 | 11.36 | 4.02 | 2.54 | 1.15 | 0.60 | 0.47 | 0.92 | | Ours | 7.55 | 16.88 | 6.33 | 2.08 | 1.77 | 0.62 | 0.49 | 0.96 |
|
Table 1. Objective evaluation results of RoadScene dataset comparison experiment (Bold, underlined, and italicized correspond to the first, second, and third places respectively)
| Method | EN↑ | SF↑ | AG↑ | MI↑ | SCD↑ | VIF↑ | $ {{Q}}^{{{{{A}{B}}}}/{{F}}} $↑ | SSIM↑ | | RFN-Nest | 6.20 | 6.16 | 2.11 | 1.70 | 1.47 | 0.66 | 0.39 | 0.77 | | SwinFusion | 6.07 | 10.79 | 3.46 | 1.58 | 1.32 | 0.72 | 0.60 | 0.95 | | SDNet | 5.25 | 8.67 | 2.67 | 1.18 | 0.99 | 0.50 | 0.38 | 0.72 | | U2Fusion | 4.95 | 6.71 | 2.09 | 1.35 | 1.01 | 0.47 | 0.32 | 0.61 | | Densefuse | 5.93 | 6.02 | 2.05 | 1.84 | 1.25 | 0.69 | 0.37 | 0.90 | | FusionGAN | 5.43 | 4.35 | 1.45 | 1.31 | 0.98 | 0.44 | 0.14 | 0.50 | | DATFusion | 6.48 | 10.93 | 3.56 | 2.70 | 1.41 | 0.91 | 0.62 | 0.91 | | Ours | 6.71 | 12.28 | 3.94 | 2.65 | 1.69 | 0.94 | 0.64 | 0.95 |
|
Table 2. Objective evaluation results of MSRS dataset comparison experiment (Bold, underlined, and italicized correspond to the first, second, and third places respectively)
| Method | EN↑ | SF↑ | AG↑ | MI↑ | SCD↑ | VIF↑ | $ {{Q}}^{{A}{B}/{F}} $↑ | SSIM | | RFN-Nest | 6.96 | 5.87 | 2.66 | 1.46 | 1.78 | 0.56 | 0.33 | 0.81 | | SwinFusion | 6.49 | 10.22 | 4.00 | 1.31 | 1.64 | 0.62 | 0.55 | 1.04 | | SDNet | 6.69 | 11.64 | 4.56 | 1.56 | 1.56 | 0.58 | 0.43 | 0.97 | | U2Fusion | 7.00 | 11.86 | 5.00 | 1.39 | 1.78 | 0.62 | 0.43 | 0.94 | | Densefuse | 6.82 | 8.99 | 3.54 | 1.59 | 1.78 | 0.66 | 0.45 | 1.02 | | FusionGAN | 6.56 | 6.28 | 2.41 | 1.62 | 1.38 | 0.42 | 0.23 | 0.66 | | DATFusion | 6.45 | 9.61 | 3.54 | 2.17 | 1.50 | 0.68 | 0.48 | 0.93 | | Ours | 7.31 | 13.14 | 5.19 | 1.84 | 1.79 | 0.75 | 0.50 | 0.99 |
|
Table 3. Objective evaluation results of TNO dataset comparison experiment (Bold, underlined, and italicized correspond to the first, second, and third places respectively)
| Dataset | n | EN$ \uparrow $ | SF$ \uparrow $ | AG$ \uparrow $ | MI$ \uparrow $ | SCD$ \uparrow $ | VIF$ \uparrow $ | $ {{Q}}^{{A}{B}/{F}}\uparrow $ | SSIM$ \uparrow $ | | TNO | n=2 | 6.99 | 13.4 | 4.88 | 1.42 | 0.74 | 0.32 | 0.40 | 0.09 | | n=4 | 7.09 | 15.33 | 5.93 | 1.41 | 0.37 | 0.24 | 0.42 | 0.15 | | n=8 | 7.10 | 13.52 | 5.13 | 1.54 | 0.85 | 0.36 | 0.46 | 0.11 | | n=16 | 7.31 | 13.14 | 5.19 | 1.84 | 1.79 | 0.75 | 0.50 | 0.99 | | n=32 | 7.29 | 12.35 | 5.34 | 1.68 | 1.53 | 0.47 | 0.39 | 0.72 |
|
Table 4. The impact of different spectral components on the fusion performance in MFCA (Bold, underlined, and italicized correspond to the first, second, and third places respectively)
| $ \mathrm{D}\mathrm{a}\mathrm{t}\mathrm{a}\mathrm{s}\mathrm{e}\mathrm{t} $ | $ \mathrm{M}\mathrm{e}\mathrm{t}\mathrm{h}\mathrm{o}\mathrm{d}\mathrm{s} $ | $ \mathrm{E}\mathrm{N}\uparrow $ | $ \mathrm{A}\mathrm{G}\uparrow $ | $ \mathrm{S}\mathrm{S}\mathrm{I}\mathrm{M}\uparrow $ | $ \mathrm{V}\mathrm{I}\mathrm{F}\uparrow $ | $ {{Q}}^{{A}{B}/{F}}\uparrow $ | | TNO | Spa+MFCA | 7.06 | 4.64 | 0.98 | 0.72 | 0.50 | | Fre+MFCA | 7.15 | 4.81 | 0.99 | 0.72 | 0.51 | | Fre+Spa | 7.18 | 5.04 | 1.01 | 0.76 | 0.53 | | Fre+Spa+MFCA | 7.31 | 5.19 | 1.02 | 0.78 | 0.55 | | RoadScene | Spa+MFCA | 7.34 | 5.69 | 0.96 | 0.66 | 0.52 | | Fre+MFCA | 7.36 | 4.94 | 0.97 | 0.69 | 0.50 | | Fre+Spa | 7.50 | 6.33 | 0.99 | 0.67 | 0.50 | | Fre+Spa+MFCA | 7.55 | 6.69 | 1.00 | 0.71 | 0.55 | | MSRS | Spa+MFCA | 6.67 | 3.74 | 0.97 | 0.97 | 0.65 | | Fre+MFCA | 6.73 | 3.66 | 0.99 | 1.04 | 0.66 | | Fre+Spa | 6.71 | 3.91 | 0.96 | 1.01 | 0.65 | | Fre+Spa+MFCA | 6.80 | 3.92 | 1.00 | 1.04 | 0.70 |
|
Table 5. Evaluation results of ablation experiments for each component. (Bold, underlined, and italicized correspond to the first, second, and third places respectively)
| $ \mathrm{D}\mathrm{a}\mathrm{t}\mathrm{a}\mathrm{s}\mathrm{e}\mathrm{t} $ | $ \mathrm{L}\mathrm{o}\mathrm{s}\mathrm{s} $ | $ \mathrm{E}\mathrm{N}\uparrow $ | $ \mathrm{M}\mathrm{I}\uparrow $ | $ \mathrm{S}\mathrm{S}\mathrm{I}\mathrm{M}\uparrow $ | $ \mathrm{V}\mathrm{I}\mathrm{F}\uparrow $ | $ {{Q}}^{{A}{B}/{F}}\uparrow $ | | TNO | $ {\mathcal{L}}_{{\mathrm{spa}}}+{\mathcal{L}}_{{\mathrm{recon}}} $ | 7.19 | 1.36 | 0.15 | 0.37 | 0.39 | | $ {\mathcal{L}}_{{\mathrm{fre}}}+{\mathcal{L}}_{{\mathrm{recon}}} $ | 7.12 | 1.73 | 0.94 | 0.60 | 0.40 | | $ {\mathcal{L}}_{{\mathrm{fre}}}+{\mathcal{L}}_{{\mathrm{spa}}}+{\mathcal{L}}_{{\mathrm{recon}}} $ | 7.31 | 1.84 | 1.02 | 0.78 | 0.55 |
|
Table 6. Evaluation results of the ablation study on loss functions (Bold, underlined, and italicized correspond to the first, second, and third places respectively)
