Real-Time Semantic Segmentation Algorithm Based on Feature Fusion Technology

Cai Yu; Huang Xuegong; Zhang Zhian; Zhu Xinnian; Ma Xiang

doi:10.3788/LOP57.021011

Journals >Laser & Optoelectronics Progress >Volume 57 >Issue 2 >Page 21011 > Article

Laser & Optoelectronics Progress
Vol. 57, Issue 2, 21011 (2020)

Real-Time Semantic Segmentation Algorithm Based on Feature Fusion Technology

Cai Yu^*, Huang Xuegong, Zhang Zhian, Zhu Xinnian, and Ma Xiang

Author Affiliations

College of Mechanical Engineering, Nanjing University of Science & Technology, Nanjing, Jiangsu 210094, China

show less

DOI: 10.3788/LOP57.021011 Cite this Article Set citation alerts

Cai Yu, Huang Xuegong, Zhang Zhian, Zhu Xinnian, Ma Xiang. Real-Time Semantic Segmentation Algorithm Based on Feature Fusion Technology[J]. Laser & Optoelectronics Progress, 2020, 57(2): 21011 Copy Citation Text

show less

Fig. 1. Framework of semantic segmentation network

Download full size

Fig. 2. Framework of context information network. (a) General framework; (b) attention module; (c) optimization module

Download full size

Fig. 3. Framework of separable convolution network

Download full size

Fig. 4. Framework of feature fusion network

Download full size

Fig. 5. Example results of proposed segmentation algorithm based on Cityscapes dataset. (a) Original image;(b) true segmentation result; (c) result of proposed algorithm

Download full size

Structure	Input	Convolution size	Mapping	Output
Conv1	W×H×3	3×7×7×64	ReLU	$\begin{matrix} \frac{1}{2} \end{matrix}$ W× $\begin{matrix} \frac{1}{2} \end{matrix}$ H×64
Conv2	$\begin{matrix} \frac{1}{2} \end{matrix}$ W× $\begin{matrix} \frac{1}{2} \end{matrix}$ H×64	64×3×3×64	ReLU	$\begin{matrix} \frac{1}{4} \end{matrix}$ W× $\begin{matrix} \frac{1}{4} \end{matrix}$ H×64
Conv3	$\begin{matrix} \frac{1}{4} \end{matrix}$ W× $\begin{matrix} \frac{1}{4} \end{matrix}$ H×64	64×3×3×64	ReLU	$\begin{matrix} \frac{1}{8} \end{matrix}$ W× $\begin{matrix} \frac{1}{8} \end{matrix}$ H×64
Conv4	$\begin{matrix} \frac{1}{8} \end{matrix}$ W× $\begin{matrix} \frac{1}{8} \end{matrix}$ H×64	64×1×1×128	ReLU	$\begin{matrix} \frac{1}{8} \end{matrix}$ W× $\begin{matrix} \frac{1}{8} \end{matrix}$ H×128

Table 1. Spatial information network structure

Experiment	Method	MIOU /%
Experiment 1	Spatial information network	60.56
Experiment 2	Experiment 1+context information network	63.47
Experiment 3	Experiment 2+attention model	68.43

Table 2. Module performance

Algorithm	Base model	Data set	Time /ms	Speed /(frame·s^-1)	MIOU /%
FCNN-8s	Paper source	Cityscapes	330.03	3.03	61.78
DeepLab-v3+CRF	Paper source	Cityscapes	1162.79	0.86	70.44
PSPNet	Paper source	Cityscapes	2380.95	0.42	75.84
ENet	Paper source	Cityscapes	73.42	13.62	52.34
Proposed algorithm	Xception	Cityscapes	70.72	14.14	68.43
Proposed algorithm	Xception	CamVid	70.72	14.14	67.28

Table 3. Comparison of algorithm performance

Cai Yu, Huang Xuegong, Zhang Zhian, Zhu Xinnian, Ma Xiang. Real-Time Semantic Segmentation Algorithm Based on Feature Fusion Technology[J]. Laser & Optoelectronics Progress, 2020, 57(2): 21011

Download Citation

Set citation alerts for the article

Tools

Set citation alerts for the article

Save the article for my favorites

Paper Information