Infrared and Visible Image Fusion Algorithm Based on Feature Optimization and GAN

Shuai HAO; Jiahao LI; Xu MA; Tian HE; Siyan SUN; Tong LI

doi:10.3788/gzxb20235212.1210004

Journals >Acta Photonica Sinica >Volume 52 >Issue 12 >Page 1210004 > Article

Acta Photonica Sinica
Vol. 52, Issue 12, 1210004 (2023)

Infrared and Visible Image Fusion Algorithm Based on Feature Optimization and GAN

Shuai HAO, Jiahao LI, Xu MA^*, Tian HE..., Siyan SUN and Tong LI|Show fewer author(s)

Author Affiliations

College of Electrical and Control Engineering，Xi'an University of Science and Technology，Xi'an 710054，China

show less

DOI: 10.3788/gzxb20235212.1210004 Cite this Article

Shuai HAO, Jiahao LI, Xu MA, Tian HE, Siyan SUN, Tong LI. Infrared and Visible Image Fusion Algorithm Based on Feature Optimization and GAN[J]. Acta Photonica Sinica, 2023, 52(12): 1210004 Copy Citation Text

show less

Fig. 1. Block diagram of the proposed algorithm

Download full size | View in the Article

Fig. 2. LatLRR decomposition results

Download full size | View in the Article

Fig. 3. Comparison chart before and after optimization

Download full size | View in the Article

Fig. 4. MSDC-Fem structure diagram

Download full size | View in the Article

Fig. 5. Attention fusion process

Download full size | View in the Article

Fig. 6. Feature reconstruction module

Download full size | View in the Article

Fig. 7. Discriminator structure

Download full size | View in the Article

Fig. 8. Subjective experimental results comparison

Download full size | View in the Article

Fig. 9. Objective experimental result comparison

Download full size | View in the Article

Fig. 10. Loss function curve

Download full size | View in the Article

n：number of chameleons in search space
d：spatial dimension
t：number of iterations，T：max-number of iterations
Build adaptive optimization image： $I_{o} = y I_{B} + I_{D}$
Design objective function for optimizing parameter y
$F = m i n (L_{A G} + L_{S D} + λ L_{C O N})$
Initialize a population of n chameleons within the search space
$y^{i} = l_{j} + r \times (u_{j} - l_{j})$
Evaluate the fitness of each chameleon using the objective function F
While（t < T）do
Step1：Search for prey
for i=1 to n do
for j=1 to d do
if $r_{i} \geq P_{p}$ then
$y_{t}^{i, j} + p_{1} (p_{t}^{i, j} - G_{_{t}}^{j}) r_{2} + p_{2} (G_{_{t}}^{j} - y_{t}^{i, j}) r_{1}$
else
$y_{t}^{i, j} + μ \cdot l_{b}^{j} s g n (r a n d - 0.5) + μ [(u_{j} - l_{j}) r_{3}] s g n (r a n d - 0.5)$
end if
end for
end for
Step2：Eyes’ rotation reveals prey
for i=1 to n do
$y_{t + 1}^{i} = m \times (y_{t}^{i} - {\bar{y}}_{t}^{i}) + {\bar{y}}_{t}^{i}$
end for
Step3：Hunting process for prey
for i=1 to n do
for j=1 to d do
$y_{t + 1}^{i, j} = y_{t}^{i, j} + [(v_{t}^{i, j})^{2} - (v_{t - 1}^{i, j})^{2}] / (2 a)$
end for
end for
Evaluate the new positions of the chameleons
Update the position of the chameleons
Evaluate the fitness of each chameleon
Return best solution y
t = t + 1
end while
Obtain the global optimal solution y，and then obtain the optimized image I_o

Table 1. Adaptive optimization of objective function based on CSA

View in the Article

step1：for M epochs do
step2：for p times do
step3：select b visible image samples： ${I_{v i s}^{1}, I_{v i s}^{2}, \dots, I_{v i s}^{b}}$ ；
step4：select b infrared image samples： ${I_{i r}^{1}, I_{i r}^{2}, \dots, I_{i r}^{b}}$ ；
step5：select b fusion image samples： ${I_{f u s e d}^{1}, I_{f u s e d}^{2}, \dots, I_{f u s e d}^{b}}$ ；
step6：Using the Adam optimizer to update discriminator parameters： $\nabla_{D} (L_{D} = L_{D - i r} + L_{D - v i s})$ ；
step7：end for
step8：select b visible image samples： ${I_{v i s}^{1}, I_{v i s}^{2}, \dots, I_{v i s}^{b}}$ ；
step9：select b infrared image samples： ${I_{i r}^{1}, I_{i r}^{2}, \dots, I_{i r}^{b}}$ ；
step10：Using the Adam optimizer to update generator parameters： $\nabla_{G} (L_{G} = L_{a d v} + λ_{1} L_{c o n t e n t})$ ；
step11：end for

Table 2. Training process of network model

View in the Article

Algorithms	DenseFuse	FusionGAN	ResNet-ZCA	MDLatLRR	PMGI	RFN-Nest	Ours
Time	0.124	1.846	1.719	5.846	0.637	0.284	0.451 9

Table 3. Average running time of different algorithms（units：s）

View in the Article

Module A	Module B	Module C	EN	SF	JE	VIF	SSIM	Q^AB/F
			6.389 5	5.864 2	11.956 1	0.664 2	0.813 4	0.307 0
√			6.415 8	6.468 5	11.990 7	0.693 4	0.824 6	0.350 9
	√		6.625 8	7.130 1	12.128 4	0.691 5	0.826 9	0.377 1
		√	6.489 6	6.256 1	12.034 2	0.689 8	0.819 8	0.315 9
√	√		6.842 9	7.956 0	12.362 9	0.715 3	0.863 7	0.388 0
√		√	6.559 3	6.594 7	12.186 3	0.708 8	0.820 4	0.375 9
	√	√	6.952 4	8.556 3	12.461 2	0.833 5	0.822 5	0.411 8
√	√	√	7.168 8	10.006 5	12.650 9	0.868 6	0.854 2	0.457 4

Table 4. Ablation experiments objectively results comparison

Shuai HAO, Jiahao LI, Xu MA, Tian HE, Siyan SUN, Tong LI. Infrared and Visible Image Fusion Algorithm Based on Feature Optimization and GAN[J]. Acta Photonica Sinica, 2023, 52(12): 1210004

Download Citation

Tools

Save the article for my favorites

Paper Information

微信扫一扫：分享

微信扫一扫：分享