Infrared and Visible Image Fusion Method Based on Dual-path Cascade Adversarial Mechanism

Lili TANG; Gang LIU; Gang XIAO

doi:10.3788/gzxb20215009.0910004

Journals >Acta Photonica Sinica >Volume 50 >Issue 9 >Page 0910004 > Article

Acta Photonica Sinica
Vol. 50, Issue 9, 0910004 (2021)

Infrared and Visible Image Fusion Method Based on Dual-path Cascade Adversarial Mechanism

Lili TANG^1、2, Gang LIU^1、*, and Gang XIAO²

Author Affiliations

¹College of Automation Engineering， Shanghai Electric Power University， Shanghai200090， China

²College of Aeronautics and Astronautics Aerospace， Shanghai Jiao Tong University， Shanghai0040， China

show less

DOI: 10.3788/gzxb20215009.0910004 Cite this Article

Lili TANG, Gang LIU, Gang XIAO. Infrared and Visible Image Fusion Method Based on Dual-path Cascade Adversarial Mechanism[J]. Acta Photonica Sinica, 2021, 50(9): 0910004 Copy Citation Text

show less

Fig. 1. DPCAM Network Framework

Download full size | View in the Article

Fig. 2. Network structure of the generator

Download full size | View in the Article

Fig. 3. Network structure of the discriminator

Download full size | View in the Article

Fig. 4. Subjective fusion results of infrared and visible images of four different scenes

Download full size | View in the Article

Fig. 5. Comparison of fusion results of “Two men in front of house”

Download full size | View in the Article

Fig. 6. Results of infrared and visible image fusion based on TNO and VIFB dataset with different algorithms

Download full size | View in the Article

Fig. 7. Comparison of six objective metrics for different algorithms

Download full size | View in the Article

Algorithm： Training process of DPCAM model
Input： Infrared image I_irand visible image I_vis
1： for number of training iteration do
2： for n steps do
3： Select m fusion patches $\{I_{f}^{1}, I_{f}^{1}, \cdot \cdot \cdot I_{f}^{m}\}$ from generator
4： Select m infrared patches $\{I_{i r}^{1}, I_{i r}^{1}, \cdot \cdot \cdot I_{i r}^{m}\}$ from generator
5： Update the parameters of the discriminator-ir by ADAM
6： Select m visible patches $\{I_{v i s}^{1}, I_{v i s}^{1}, \cdot \cdot \cdot I_{v i s}^{m}\}$ from generator
7： Update the parameters of the discriminator-vis by ADAM
8： end for
9： Select m infrared patches $\{I_{i r}^{1}, I_{i r}^{1}, \cdot \cdot \cdot I_{i r}^{m}\}$ and visible pathes $\{I_{v i s}^{1}, I_{v i s}^{1}, \cdot \cdot \cdot I_{v i s}^{m}\}$ from training data
10： Update generator by ADAM
11： end for
Output： Parameters of generator and discriminator

Table 1. Training process of DPCAM model

View in the Article

Metrics	Method
Metrics	CBF	ConvSR	FPDE	CNN	DenseFuse	FusionGAN	DDCGAN	MDLatLRR	DPCAM
EN	6.476 3	6.450 6	5.619 7	6.417 7	5.905 5	6.250 0	7.342 7	4.928 3	6.927 7
Q^AB/F	0.394 7	0.430 8	0.330 0	0.337 5	0.338 6	0.133 9	0.236 6	0.450 0	0.439 7
SCD	0.654 9	0.319 2	1.342 0	0.562 4	1.023 9	1.005 5	1.576 2	1.499 7	1.818 3
VIFF	0.430 6	0.895 8	0.564 8	0.722 9	0.719 5	0.449 3	0.921 2	0.469 8	0.752 5
MS-SSIM	0.555 8	0.826 2	0.928 5	0.909 7	0.894 1	0.783 0	0.618 3	0.923 6	0.955 7
SF	0.041 8	0.045 7	0.046 2	0.043 8	0.019 3	0.009 9	0.032 2	0.017 0	0.052 1

Table 2. “Two men in front of house” objective evaluation of comparative experiments

View in the Article

Metrics	Method
Metrics	CBF	ConvSR	FPDE	CNN	DenseFuse	FusionGAN	DDCGAN	MDLatLRR	DPCAM
EN	6.710 6	6.906 6	6.797 4	7.127 0	6.918 4	6.372 8	7.489 4	6.209 0	6.917 4
Q^AB/F	0.332 4	0.403 0	0.264 8	0.306 3	0.277 9	0.129 1	0.347 6	0.371 5	0.362 7
SCD	1.407 2	1.163 9	1.563 0	1.442 1	1.664 2	1.324 0	1.625 7	1.658 9	1.755 0
VIFF	0.530 5	0.707 5	0.704 1	0.781 0	0.740 9	0.364 0	0.720 6	0.652 1	0.759 8
MS-SSIM	0.699 3	0.783 1	0.875 5	0.890 3	0.828 8	0.587 0	0.766 5	0.896 2	0.898 2
SF	0.521 4	0.062 9	0.059 0	0.061 4	0.026 0	0.019 3	0.049 7	0.027 5	0.052 2

Table 3. The average quantitative value of the fusion results of 21 groups

Lili TANG, Gang LIU, Gang XIAO. Infrared and Visible Image Fusion Method Based on Dual-path Cascade Adversarial Mechanism[J]. Acta Photonica Sinica, 2021, 50(9): 0910004

Download Citation

Tools

Save the article for my favorites

Paper Information