A multi-target semantic segmentation method for millimetre wave SAR images based on a dual-branch multi-scale fusion network

Junhua Ding; Minghui Yuan

doi:10.12086/oee.2023.230242

Journals >Opto-Electronic Engineering >Volume 50 >Issue 12 >Page 230242-1 > Article

Opto-Electronic Engineering
Vol. 50, Issue 12, 230242-1 (2023)

A multi-target semantic segmentation method for millimetre wave SAR images based on a dual-branch multi-scale fusion network

Junhua Ding^1,2 and Minghui Yuan^1,2,*

Author Affiliations

¹Terahertz Technology Innovation Research Institute, University of Shanghai for Science and Technology, Shanghai 200093, China

²School of Optical-Electrical and Computer Engineering, University of Shanghai for Science and Technology, Shanghai 200093, China

show less

DOI: 10.12086/oee.2023.230242 Cite this Article

Junhua Ding, Minghui Yuan. A multi-target semantic segmentation method for millimetre wave SAR images based on a dual-branch multi-scale fusion network[J]. Opto-Electronic Engineering, 2023, 50(12): 230242-1 Copy Citation Text

show less

Fig. 1. DBMFnet network structure diagram

Download full size | View in the Article

Fig. 2. Feature fusion process

Download full size | View in the Article

Fig. 3. Different feature fusion methods. (a) FCM; (b) FDM; (c) MSFM

Download full size | View in the Article

Fig. 4. HM-SAR security images. (a) Back scanning image of the human body; (b) Frontal scanning image of the human body

Download full size | View in the Article

Fig. 5. DBMFnet thermal diagram

Download full size | View in the Article

Fig. 6. Test results of each model. Each row represents the test results of the same picture, and each column represents the test results of the same model. Black denotes the background, green denotes the wrench, yellow denotes the pistol, red denotes the hammer, and blue denotes the knife

Download full size | View in the Article

Fig. 7. Baseline model

Download full size | View in the Article

Stage	Output	DBFEN	Stage	Output	DBFEN
Conv1	256×256	3×3, 64, stride 2	Conv6	64×64	$(\begin{array}{l} 3 \times 3,128 \\ 3 \times 3,128 \end{array}) \times 2$
Conv2	128×128	3×3, 64, stride 2	Conv7	16×16	$(\begin{array}{l} 3 \times 3,256 \\ 3 \times 3,512 \end{array}) \times 2$
Conv3	64×64	$(\begin{array}{l} 3 \times 3,64 \\ 3 \times 3,128 \end{array}) \times 2$	Conv8	64×64	$(\begin{array}{l} 3 \times 3,128 \\ 3 \times 3,256 \end{array}) \times 2$
Conv4	64×64	$(\begin{array}{l} 3 \times 3,128 \\ 3 \times 3,128 \end{array}) \times 2$	Conv9	8×8	$(\begin{array}{l} 3 \times 3,512 \\ 3 \times 3,1024 \end{array}) \times 2$
Conv5	32×32	$(\begin{array}{l} 3 \times 3,128 \\ 3 \times 3,256 \end{array}) \times 2$

Table 1. Architectures of DBFEN

View in the Article

Network model	MPA/%	mIoU/%	F1/%	Network model	MPA/%	mIoU/%	F1/%
U-net	80.29	70.35	81.87	Deeplabv3+	81.05	70.58	82.00
Pspnet	82.98	72.32	83.28	HRnet-v2	82.33	72.90	83.69
FCN-8s	81.29	72.11	83.11	DBMFnet (ours)	85.01	75.44	85.21

Table 2. Comparisons of the segmentation performance of each model in the HM-SAR dataset

View in the Article

Class	U-net		Pspnet		Deeplabv3+		HRnet-v2		FCN-8s		DBMFnet (ours)
Class	Pre	IoU	Pre	IoU	Pre	IoU	Pre	IoU	Pre	IoU	Pre	IoU
Hammer	80.74	61.98	76.49	63.7	80.15	63.99	79.93	67.35	79.16	65.17	81.91	69.33
Wrench	82.66	66.78	82.88	71.84	80.61	66.57	78.80	66.15	84.04	69.56	84.22	75.24
Pistol	75.63	63.77	77.3	64.21	75.45	62.65	85.71	69.47	81.07	65.81	87.89	70.56
Knife	78.59	59.4	81.36	62.01	78.82	59.84	81.67	61.68	80.06	60.16	82.55	66.15

Table 3. Comparisons of the objects segmentation performance of each model in the HM-SAR dataset

View in the Article

Network model	Params/M	GFLOPs	Speed/(f/s)
U-net	24.89	452.31	32
Pspnet	46.7	118.43	33.5
FCN-8s	32.95	277.74	16
Deeplabv3+	54.71	166.87	21
HRnet	29.55	80.18	11.5
DBMFnet(our)	19.54	47.36	26

Table 4. Calculation complexity and inference speed of each model

View in the Article

Network model	mIoU	Params/M	GFLOPs
Baseline	72.61	23.15	38.78
Deeplabv3+(FCM)	70.58	54.71	166.87
FCN-8s(FDM)	72.11	32.95	277.74
Baseline+FCM	74.1	22.44	100.8
Baseline+FDM	73.16	21.65	45.27
Baseline+MSFM	75.44	23.06	47.86

Table 5. Comparisons of models using different decoder modules

Download Citation

Save the article for my favorites

Paper Information

微信扫一扫：分享

微信扫一扫：分享