Infrared Target Detection Algorithm Based on Pseudo Multimodal Images

Hao-nan AN; Ming ZHAO; Sheng-da PAN; Chang-qing LIN

doi:10.3788/gzxb20204908.0810002

Journals >Acta Photonica Sinica >Volume 49 >Issue 8 >Page 0810002 > Article

Acta Photonica Sinica
Vol. 49, Issue 8, 0810002 (2020)

Infrared Target Detection Algorithm Based on Pseudo Multimodal Images

Hao-nan AN¹, Ming ZHAO^1、2, Sheng-da PAN¹, and Chang-qing LIN²

Author Affiliations

¹College of Information Engineering, Shanghai Maritime University, Shanghai 201306, China

²Key Laboratory of Intelligent Infrared Perception, Chinese Academy of Sciences, Shanghai 200083, China

show less

DOI: 10.3788/gzxb20204908.0810002 Cite this Article

Hao-nan AN, Ming ZHAO, Sheng-da PAN, Chang-qing LIN. Infrared Target Detection Algorithm Based on Pseudo Multimodal Images[J]. Acta Photonica Sinica, 2020, 49(8): 0810002 Copy Citation Text

show less

Fig. 1. Structure diagram of the proposed algorithm

Download full size | View in the Article

Fig. 2. Process of generating pseudo visible light image using CycleGAN dual cycle countermeasure

Download full size | View in the Article

Fig. 3. Residual module in bimodal feature extraction network

Download full size | View in the Article

Fig. 4. Improved residual network of bimodal feature extraction

Download full size | View in the Article

Fig. 5. Feature vector of image obtained by residual network

Download full size | View in the Article

Fig. 6. Dataset sample images

Download full size | View in the Article

Fig. 7. Infrared image and its corresponding pseudo visible image

Download full size | View in the Article

Fig. 8. Average accuracy of each iteration of training vehicle and person on training set

Download full size | View in the Article

Fig. 9. Detection result on FLIR-ADAS data set

Download full size | View in the Article

Fig. 10. Detection result on SODA data set

Download full size | View in the Article

Algorithm:PMFD: Pse-model fused detection
Input:（1）Infrared image training set:{(x_i，y_i)}_i=1ⁿ
（2）Generator of I2I framework :W_I2R
（3）stage1:IR Pre-trained:W_IR
（4）stage2:FRGB Pre-trained:W_FRGB
（5）stage3:Fusion Pre-train:W_ADD
Output: Trained PMFD model, F (g)
for num_epoches do
for x_i，i = 1，…，n do
Through I2I framework generate a pseudo RGB $\hat{x}_{i}$ using W_I2R
Then the infrared image x_i and its corresponding pseudo RGB image $\hat{x}_{i}$ are input into the respective training channels
using W_IR and W_FRGB, generate fusion vector (Tensor in Fig. 1)
Pass the fusion vector to Fusion Pre-train network using W_ADD
Update W_I2R, W_IR, W_FRGB, W_ADD by minimizing Loss function of the PMFD model
end
end

Table 1. Algorithm detailed process

View in the Article

Data sets	Number of samples
Training set	8 140
Validation set	2 326
Testing set	1 163
Total	11 629

Table 2. Dataset distribution

View in the Article

Networks		Precision	Recall	F1-score	mAP
SSD	All	0.564	0.836	0.674	0.571
	Car	0.603	0.881	0.716	0.628
	Person	0.525	0.790	0.631	0.514
Faster-RCNN	All	0.581	0.841	0.687	0.612
	Car	0.625	0.880	0.731	0.676
	Person	0.536	0.801	0.642	0.547
Baseline	All	0.608	0.892	0.723	0.786
	Car	0.630	0.909	0.744	0.82
	Person	0.586	0.874	0.701	0.752
MMTOD	All	0.629	0.893	0.738	0.800
	Car	0.640	0.902	0.749	0.835
	Person	0.618	0.884	0.727	0.765
PMFD	All	0.625	0.909	0.741	0.813
	Car	0.638	0.923	0.754	0.839
	Person	0.611	0.894	0.726	0.786

Table 3. Experimental results on test set

Hao-nan AN, Ming ZHAO, Sheng-da PAN, Chang-qing LIN. Infrared Target Detection Algorithm Based on Pseudo Multimodal Images[J]. Acta Photonica Sinica, 2020, 49(8): 0810002

Download Citation

Tools

Save the article for my favorites

Paper Information