Infrared and Visible Image Fusion Method via Interactive Attention-based Generative Adversarial Network

Zhishe WANG; Wenyu SHAO; Fengbao YANG; Yanlin CHEN

doi:10.3788/gzxb20225104.0410002

Journals >Acta Photonica Sinica >Volume 51 >Issue 4 >Page 0410002 > Article

Acta Photonica Sinica
Vol. 51, Issue 4, 0410002 (2022)

Infrared and Visible Image Fusion Method via Interactive Attention-based Generative Adversarial Network

Zhishe WANG^1、*, Wenyu SHAO¹, Fengbao YANG², and Yanlin CHEN¹

Author Affiliations

¹School of Applied Science，Taiyuan University of Science and Technology，Taiyuan 030024，China

²School of Information and Communication Engineering，North University of China，Taiyuan 030051，China

show less

DOI: 10.3788/gzxb20225104.0410002 Cite this Article

Zhishe WANG, Wenyu SHAO, Fengbao YANG, Yanlin CHEN. Infrared and Visible Image Fusion Method via Interactive Attention-based Generative Adversarial Network[J]. Acta Photonica Sinica, 2022, 51(4): 0410002 Copy Citation Text

show less

Fig. 1. The principle of interactive attention-based generative adversarial network

Download full size | View in the Article

Fig. 2. Interactive attention fusion model

Download full size | View in the Article

Fig. 3. The subjective comparison results of five fusion models

Download full size | View in the Article

Fig. 4. The subjective comparison results of different fusion methods on the TNO dataset

Download full size | View in the Article

Fig. 5. The subjective comparison results of different fusion methods for FLIR_06422

Download full size | View in the Article

Fig. 6. The subjective comparison results of different fusion methods for FLIR_07210

Download full size | View in the Article

Models	AG	SD	MI	PC	NCIE	SF	MS_SSIM	VIF
No_atten	5.464 33	29.894 71	1.942 38	0.237 18	0.804 27	10.010 08	0.872 68	0.395 57
Only_CA	5.153 56	36.361 13	2.657 49	0.308 27	0.806 42	9.620 20	0.892 71	0.409 20
Only_SA	4.649 28	37.827 80	2.968 02	0.316 78	0.807 41	8.807 06	0.879 70	0.403 27
SA_CA	5.547 57	37.047 94	2.958 20	0.315 43	0.807 46	10.554 43	0.875 66	0.426 09
Ours	5.572 04	36.768 19	3.047 12	0.309 84	0.807 91	10.588 10	0.863 14	0.426 42

Table 1. The objective comparison results of five fusion models on the TNO dataset

View in the Article

Methods	AG	SD	MI	PC	NCIE	SF	MS_SSIM	VIF
WLS	5.417 71	24.860 27	1.742 86	0.280 99	0.803 91	10.793 84	0.909 88	0.378 66
DenseFuse	3.193 31	22.857 69	2.035 89	0.290 28	0.804 51	6.094 43	0.876 96	0.330 90
IFCNN	5.467 53	24.067 12	1.801 91	0.278 68	0.804 04	10.131 41	0.904 74	0.389 20
SEDRFuse	3.544 11	40.793 02	2.110 14	0.172 27	0.804 62	6.794 46	0.895 05	0.316 82
U2Fusion	5.615 15	33.596 08	1.760 99	0.261 20	0.803 92	10.436 55	0.919 82	0.420 78
PMGI	4.695 36	34.768 16	2.108 01	0.243 11	0.804 82	8.999 78	0.887 96	0.390 59
FusionGAN	3.073 57	26.820 44	2.166 52	0.102 64	0.805 03	5.982 47	0.734 49	0.248 69
GANMcC	3.139 83	29.929 73	2.108 64	0.232 71	0.804 52	6.009 63	0.859 15	0.305 10
RFN-Nest	3.125 21	34.853 73	1.928 51	0.233 90	0.804 28	6.012 69	0.912 17	0.355 10
Ours	5.572 04	36.768 19	3.047 12	0.309 84	0.807 91	10.588 10	0.863 14	0.426 42
AMIR	34.548%	21.416%	54.385%	33.056%	0.436%	33.735%	-1.670%	22.384%

Table 2. The objective comparison results of different fusion methods on the TNO dataset

View in the Article

Methods	AG	SD	MI	PC	NCIE	SF	MS_SSIM	VIF
WLS	6.119 13	24.220 50	1.475 49	0.244 28	0.803 25	11.297 25	0.906 43	0.309 24
DenseFuse	3.814 31	22.715 99	1.579 94	0.258 93	0.803 41	7.130 29	0.873 69	0.286 36
IFCNN	6.651 62	23.563 42	1.500 51	0.237 27	0.803 25	12.168 38	0.902 03	0.315 65
SEDRFuse	4.993 49	38.476 49	2.059 20	0.252 53	0.804 31	9.803 58	0.934 91	0.316 76
U2Fusion	5.961 61	34.031 64	1.593 96	0.259 71	0.803 42	11.554 53	0.936 11	0.339 02
PMGI	5.274 55	34.230 81	2.080 61	0.214 98	0.804 35	9.987 53	0.876 01	0.318 94
FusionGAN	3.115 54	25.355 11	2.431 53	0.082 94	0.805 50	6.404 57	0.670 09	0.207 64
GANMcC	4.333 90	33.617 44	1.994 60	0.228 22	0.804 04	8.180 66	0.900 37	0.278 32
RFN-Nest	3.568 69	35.239 55	1.687 92	0.264 75	0.803 65	7.057 25	0.925 03	0.304 43
Ours	5.775 83	39.153 66	3.252 20	0.299 74	0.807 97	11.442 50	0.857 93	0.329 06
AMIR	18.593%	29.815%	78.433%	32.005%	0.505%	23.208%	-2.565%	10.656%

Table 3. The objective comparison results of different fusion methods on the Nato-camp sequence

View in the Article

Methods	AG	SD	MI	PC	NCIE	SF	MS_SSIM	VIF
WLS	5.883 14	29.056 59	2.361 24	0.342 41	0.806 11	12.347 23	0.885 22	0.311 46
DenseFuse	3.652 56	27.115 48	2.776 46	0.350 99	0.807 31	7.679 22	0.831 36	0.266 34
IFCNN	5.964 53	28.818 46	2.483 70	0.334 82	0.806 36	12.521 83	0.873 55	0.293 26
SEDRFuse	5.337 39	46.910 59	3.017 21	0.280 03	0.807 34	11.115 36	0.895 57	0.276 69
U2Fusion	6.514 43	45.098 18	3.001 63	0.348 45	0.807 95	13.724 65	0.912 77	0.315 35
PMGI	5.027 04	45.020 31	2.681 32	0.304 32	0.808 45	10.472 62	0.894 09	0.286 20
FusionGAN	3.919 09	39.887 01	2.672 07	0.110 38	0.807 26	8.450 70	0.732 31	0.190 09
GANMcC	4.076 88	42.787 94	2.805 20	0.280 49	0.806 81	8.451 23	0.862 17	0.243 64
RFN-Nest	3.830 42	44.210 13	2.887 68	0.262 81	0.806 80	8.115 65	0.876 61	0.265 58
Ours	6.071 37	39.124 91	3.416 26	0.401 35	0.808 96	13.062 80	0.825 99	0.331 78
AMIR	23.610%	0.923%	24.547%	38.148%	0.224%	26.580%	-4.247%	21.948%

Table 4. The objective comparison results of different fusion methods on the Roadscene dataset

View in the Article

Dataset	WLS	Dense-Fuse	IFCNN	SEDR-Fuse	U2Fusion	PMGI	Fusion-GAN	GAN-McC	RFN-Nest	Ours
TNO	2.359	0.085	0.046	1.148	1.722	0.588	2.184	4.445	0.177	0.151
Sequence	0.455	0.029	0.015	1.158	0.659	0.181	0.678	1.329	0.068	0.059
Roadscene	1.087	0.048	0.024	1.137	0.992	0.293	1.135	2.271	0.096	0.087

Table 5. The comparison results of computation efficiency for different fusion methods（units：s）

Download Citation

Save the article for my favorites

Paper Information