Small object detection based on multi-scale feature fusion using remote sensing images

Liang Ma; Yutao Gou; Tao Lei; Lei Jin; Yixuan Song

doi:10.12086/oee.2022.210363

Journals >Opto-Electronic Engineering >Volume 49 >Issue 4 >Page 210363 > Article

Opto-Electronic Engineering
Vol. 49, Issue 4, 210363 (2022)

Small object detection based on multi-scale feature fusion using remote sensing images

Liang Ma^1、2、3, Yutao Gou^1、2、3, Tao Lei^1、2、*, Lei Jin^1、2, and Yixuan Song^1、2、3

Author Affiliations

¹Photoelectric Detection Technology Laboratory, Chinese Academy of Sciences, Chengdu, Sichuan 610209, China

²Institute of Optics and Electronics, Chinese Academy of Sciences, Chengdu, Sichuan 610209, China

³University of Chinese Academy of Sciences, Beijing 100049, China

show less

DOI: 10.12086/oee.2022.210363 Cite this Article

Liang Ma, Yutao Gou, Tao Lei, Lei Jin, Yixuan Song. Small object detection based on multi-scale feature fusion using remote sensing images[J]. Opto-Electronic Engineering, 2022, 49(4): 210363 Copy Citation Text

show less

Fig. 1. Complex background in remote sensing images

Download full size | View in the Article

Fig. 2. Network framework

Download full size | View in the Article

Fig. 3. Network structure

Download full size | View in the Article

Fig. 4. Feature weighting method based on grouped convolution

Download full size | View in the Article

Fig. 5. (a) Schematic diagram of convolutional network receptive field; (b) Object classification strategy based on receptive field

Download full size | View in the Article

Fig. 6. Object scale distribution of the dataset

Download full size | View in the Article

Fig. 7. Sample of plane and small-vehicle image of DOTA dataset used in the experiment. (a) Training set; (b) Testing set

Download full size | View in the Article

Fig. 8. Objects cut and copy flow diagram

Download full size | View in the Article

Fig. 9. The loss curve of the network trained on the DOTA plane training set

Download full size | View in the Article

Fig. 10. The loss curve of the network trained on the DOTA small-vehicle training set

Download full size | View in the Article

Fig. 11. Partial plane test results. Yellow circles represent false alarms and green circles represent missed detection.

Download full size | View in the Article

Fig. 12. Partial small-vehicle test results. Yellow circles represent false alarms and green circles represent missed detection.

Download full size | View in the Article

Fig. 13. Model convergence under different initial values of fusion factors

Download full size | View in the Article

模型	参数量M
VGG16	138
ResNet50	25.6
ResNet101	44.6
Ours	0.49

Table 1. Parameters of different networks

View in the Article

金字塔层数		检测目标尺寸	下采样倍数	感受野	感受野步长	感受野/目标尺寸
两层	1	6~25	4	55	4	3.5
两层	2	25~50	8	95	8	2.5
三层	1	6~10	2	23	2	2.9
	2	10~20	4	47	4	3.1
	3	20~50	8	79	8	2.3

Table 2. Receptive field of feature map and corresponding object size parameters

View in the Article

Basic unit		FPN	mAP	Precision	Recall
两层	B_11	-	86.8	76.8	88.9
两层	B_11	√	87.2	82.6	88.8
三层	B_10	-	87.4	47.4	91.8
三层	B_10	√	88.5	83.8	90.4

Table 3. Detection results of different feature fusion schemes

View in the Article

B_13	FPN	分组数量(3)	特征图通道(channel)	常数融合因子[0.71,0.87]	mAP	Precision	Recall
√	-	-	-	-	80.5	63.4	82.8
√	√	-	-	-	82.0	81.4	85.1
√	√	√	-	-	82.3	85.1	84.5
√	√	-	√	-	83.6	85.5	87.0
√	√	-	-	√	82.5	82.3	85.6

Table 4. DOTA plane dataset test results under different network configurations

View in the Article

B_12	FPN	分组数量(3)	特征图通道(channel)	常数融合因子[0.63,1.28]	mAP	Precision	Recall
√	-	-	-	-	63.7	56.8	73.9
√	√	-	-	-	65.9	86.1	68.5
√	√	√	-	-	66.3	83.3	68.9
√	√	-	√	-	68.7	86.4	71.7
√	√	-	-	√	64.4	84.0	67.3

Table 5. DOTA small-vehicle dataset test results under different network configurations

View in the Article

B_10	FPN	分组数量(3)	特征图通道(channel)	常数融合因子[1.08,1.05]	mAP	Precision	Recall
√	-	-	-	-	89.9	44.2	93.7
√	√	-	-	-	90.2	83.6	91.4
√	√	√	-	-	90.6	84.8	92.0
√	√	-	√	-	91.0	87.7	92.4
√	√	-	-	√	未收敛	未收敛	未收敛

Table 6. Test results of our dataset under different network configurations

View in the Article

数据集	尺度	目标数量	常数融合因子 $(\frac{S_{i + 1}}{S_{i}})$
DOTA飞机训练集	$S_{1}$ [6-12]	5503	0.87
	$S_{2}$ [12-30]	4807
	$S_{3}$ [30-70]	3428	0.71
DOTA小汽车训练集	$S_{1}$ [6-15]	59875	1.28
	$S_{2}$ [15-25]	76615
	$S_{3}$ [25-60]	48203	0.63
自建数据训练集	$S_{1}$ [6-10]	4963	1.05
	$S_{2}$ [10-20]	5196
	$S_{3}$ [20-50]	5625	1.08

Table 7. Statistics of the distribution of each scale objects number

View in the Article

融合因子初始值	mAP
1	83.6
随机初始化	80.7

Table 8. Influence of initial value of fusion factor on detection performance

View in the Article

模型+数据集	mAP	Precision	Recall	推理速度/(s/张)
B_10+FPN+CBAM(自建数据集)	90.5	83.8	90.6	0.036
B_10+FPN+自适应融合模块(自建数据集)	91.0	87.7	92.4	0.027
B_13+FPN+CBAM(DOTA飞机数据集)	83.0	82.6	85.8	0.048
B_13+FPN+自适应融合模块(DOTA飞机数据集)	83.6	85.5	87.0	0.037
B_12+FPN+CBAM(DOTA小汽车数据集)	67.6	83.0	71.1	0.043
B_12+FPN+自适应融合模块(DOTA小汽车数据集)	68.7	83.3	71.7	0.034

Table 9. Influence of CBAM and adaptive fusion module on detection performance

View in the Article

方法	DOTA飞机数据集(mAP)	DOTA小汽车数据集(mAP)	自建数据集(mAP)
SSD	63.4	43.3	64.4
RetinaNet	55.2	45.1	62.7
Yolov3-tiny	70.8	58.3	74.3
Faster R-CNN	73.0	59.0	88.6
Ours	83.6	68.7	91.0

Table 10. Comparison of detection performance of different methods

View in the Article

Backbone+数据集	mAP
ResNet50+FPN(自建数据集)	88.6
ResNet50+自适应融合模块(自建数据集)	89.7
ResNet50+FPN(DOTA飞机数据集)	73.0
ResNet50+自适应融合模块(DOTA飞机数据集)	73.8
ResNet50+FPN(DOTA小汽车数据集)	59.0
ResNet50+自适应融合模块(DOTA小汽车数据集)	63.2

Table 11. Performance of FPN module based on adaptive feature weighted fusion on Faster R-CNN

Liang Ma, Yutao Gou, Tao Lei, Lei Jin, Yixuan Song. Small object detection based on multi-scale feature fusion using remote sensing images[J]. Opto-Electronic Engineering, 2022, 49(4): 210363

Download Citation

Tools

Save the article for my favorites

Paper Information