Visual Simultaneous Localization and Mapping Method of Semantic Octree Map Toward Indoor Dynamic Scenes

Rongfen Zhang; Wenhao Yuan; Jin Lu; Yuhong Liu

doi:10.3788/LOP202259.1811003

Journals >Laser & Optoelectronics Progress >Volume 59 >Issue 18 >Page 1811003 > Article

Laser & Optoelectronics Progress
Vol. 59, Issue 18, 1811003 (2022)

Visual Simultaneous Localization and Mapping Method of Semantic Octree Map Toward Indoor Dynamic Scenes

Rongfen Zhang^*, Wenhao Yuan, Jin Lu, and Yuhong Liu

Author Affiliations

College of Big Data and Information Engineering, Guizhou University, Guiyang 550025, Guizhou , China

show less

DOI: 10.3788/LOP202259.1811003 Cite this Article Set citation alerts

Rongfen Zhang, Wenhao Yuan, Jin Lu, Yuhong Liu. Visual Simultaneous Localization and Mapping Method of Semantic Octree Map Toward Indoor Dynamic Scenes[J]. Laser & Optoelectronics Progress, 2022, 59(18): 1811003 Copy Citation Text

show less

Fig. 1. Structure of proposed vSLAM system

Download full size

Fig. 2. Network structure of FAST-SCNN^[10]

Download full size

Fig. 3. Result of semantic segmentation. (a) Original drawing; (b) segmented output

Download full size

Fig. 4. Outliers removing. (a) All feature points; (b) effect after removing moving points

Download full size

Fig. 5. Diagram of octree map

Download full size

Fig. 6. Semantic octree map building process

Download full size

Fig. 7. Dining Rooms image sequence and its semantic segmentation results

Download full size

Fig. 8. Point cloud. (a) Original point cloud; (b) filtered point cloud

Download full size

Fig. 9. Semantic point cloud and semantic octree map. (a) Semantic point cloud; (b) semantic octree map

Download full size

Fig. 10. Comparison of ATE. (a) ORB-SLAM2; (b) proposed system

Download full size

Fig. 11. Comparison of relative translation error. (a) ORB-SLAM2; (b) proposed system

Download full size

Fig. 12. Semantic octree map building effect. (a) Original frame sequence; (b) semantic octree map

Download full size

Fig. 13. Feature points extraction of proposed system in two scenes

Download full size

Fig. 14. Semantic segmentation results of proposed system in two scenes

Download full size

Fig. 15. Moving point removing effect of proposed system in two scenes

Download full size

Fig. 16. Map building effect of proposed system

Download full size

Algorithm Multi-stage RANSAC Algorithm
Input： Previous Frame， F₁； KeyPoints of F₁， K； Current Frame， F₂；The minimum number of points required， m
Output： The outliers， O； The inliers， I
1： assign K to I
2： for the current stage number i is less than n
3： compute the maximum number required for iterations， N
4： select randomly m points from I
5： while the number of iterations is less than N
6： solve for F
7： determine the portion of inliers among K using F
8： if the portion of inliers is larger than τ_i
9： leave the loop
10： end if
11： end while
12： assign new inliers to I
13： end for

Table 1. Multi-stage RANSAC algorithm

τ	Ins.	Outs.	Time /ms
0.1	168	6	8.22814
0.2	228	12	1.02447
0.3	260	13	0.60222

Table 2. Test results of RANSAC algorithm

τ₁	τ₂	Ins.₁	Ins.₂	Outs.	Time /ms
1	0.1	294	159	5	4.99953
1	0.2	294	210	4	0.83729
1	0.3	294	238	11	0.48140
2	0.1	327	159	7	9.83135
2	0.2	327	211	9	1.63077
2	0.3	327	238	11	0.69094

Table 3. Test results of Multi-stage RANSAC algorithm

Dynamic Points Removing Algorithm
Input： Dynamic Points， O； Semantic mask of most likely moving objects， M； KeyPoints， K
Output： The set of inliers， I
1： if M not empty then
2： for point o， m in O， M do
3： if o = m then
4： remove M from K
5： leave the loop
6： end if
7： end for
8： end if

Table 4. Moving point removing algorithm

Seq.	ORB-SLAM2		Proposed system		Improvement
Seq.	RMSE /m	S.D /m	RMSE /m	S.D /m	RMSE /%	S.D /%
WX	0.825981	0.449478	0.019991	0.010374	97.58	97.69
WH	0.502363	0.299615	0.025955	0.012036	94.83	95.98
WR	1.212279	0.676205	0.062167	0.041967	94.87	93.79
WS	0.585281	0.423743	0.009044	0.004097	98.45	99.03
SX	0.013602	0.006688	0.012649	0.006140	7.01	8.19
SH	0.040732	0.023205	0.017660	0.008521	56.64	63.28
SR	0.030898	0.018059	0.021174	0.011048	31.47	38.82
SS	0.012007	0.005570	0.007237	0.003720	39.73	33.21

Table 5. Typical value of translation

Seq.	ORB-SLAM2		Proposed system		Improvement
Seq.	RMSE /deg	S.D /deg	RMSE /deg	S.D /deg	RMSE /%	S.D /%
WX	14.812930	8.086117	0.618651	0.376314	95.82	95.35
WH	13.379170	7.277847	0.749063	0.354548	94.40	95.13
WR	22.021472	12.858064	1.201734	0.787774	94.54	93.87
WS	10.334787	7.523754	0.256180	0.110816	97.52	98.53
SX	0.578052	0.299133	0.503881	0.274636	12.83	8.19
SH	1.030726	0.456638	0.652451	0.330930	36.70	27.53
SR	0.882169	0.434188	0.748790	0.362180	15.12	16.58
SS	0.336292	0.144834	0.259707	0.116066	22.77	19.86

Table 6. Typical value of rotation

Seq.	ORB-SLAM2		Proposed system		Improvement
Seq.	RMSE /m	S.D /m	RMSE /m	S.D /m	RMSE /%	S.D /%
WX	0.565505	0.200691	0.014932	0.007969	97.36	96.03
WH	0.327989	0.177225	0.025357	0.013169	92.27	92.57
WR	0.817879	0.430206	0.047672	0.033180	94.17	92.29
WS	0.409268	0.175114	0.006957	0.003244	98.30	98.15
SX	0.009275	0.004796	0.010674	0.004922	-15.08	-2.63
SH	0.027882	0.013692	0.014091	0.006767	49.46	50.58
SR	0.021513	0.014181	0.016052	0.009607	25.38	32.25
SS	0.007698	0.003655	0.005568	0.003047	27.67	16.63

Table 7. Typical value of ATE

Seq.	ORB-SLAM2		Proposed system		Improvement
Seq.	RMSE /m	S.D /m	RMSE /m	S.D /m	RMSE /%	S.D /%
WX	0.030283	0.019977	0.019991	0.010374	33.99	48.07
WH	0.031893	0.016458	0.025955	0.012036	18.62	26.87
WR	0.152189	0.116961	0.062167	0.041967	59.15	64.12
WS	0.009347	0.004358	0.009044	0.004097	3.24	5.99
SX	0.012692	0.006664	0.012649	0.006140	0.34	7.86
SH	0.017403	0.007808	0.017660	0.008221	-1.48	-5.29
SR	0.025757	0.014738	0.021174	0.011048	17.79	25.04
SS	0.007327	0.003564	0.007237	0.003520	1.29	1.23

Table 8. Typical value of translation

Seq.	ORB-SLAM2		Proposed system		Improvement
Seq.	RMSE /deg	S.D /deg	RMSE /deg	S.D /deg	RMSE /%	S.D /%
WX	0.766283	0.529380	0.618651	0.376314	19.27	28.91
WH	0.846296	0.422084	0.749063	0.354548	11.49	16.00
WR	3.043619	2.335232	1.201734	0.787774	60.52	66.27
WS	0.255013	0.106953	0.256180	0.110816	-0.46	-3.61
SX	0.495392	0.270148	0.493881	0.274636	0.31	-1.66
SH	0.630755	0.300657	0.628451	0.290930	0.37	3.24
SR	0.843054	0.468068	0.748790	0.362180	11.18	22.62
SS	0.262147	0.117104	0.259707	0.116066	0.93	0.89

Table 9. Typical value of rotation

Seq.	ORB-SLAM2		Proposed system		Improvement
Seq.	RMSE /m	S.D /m	RMSE /m	S.D /m	RMSE /%	S.D /%
WX	0.022180	0.014402	0.014932	0.007969	32.68	44.67
WH	0.032083	0.017715	0.025357	0.013169	20.96	25.66
WR	0.433820	0.228252	0.047672	0.033180	89.01	85.46
WS	0.007709	0.003275	0.006957	0.003244	9.75	0.95
SX	0.010339	0.005377	0.010674	0.004922	-3.24	8.46
SH	0.014816	0.006672	0.014091	0.006767	4.89	-1.42
SR	0.020242	0.012680	0.016052	0.009607	20.70	24.24
SS	0.006273	0.003085	0.005568	0.003047	11.24	1.23

Table 10. Typical value of ATE

Seq.	DS-SLAM		Multi		Improvement
Seq.	RMSE /m	S.D /m	RMSE /m	S.D /m	RMSE /%	S.D /%
WX	0.030283	0.019977	0.024652	0.013576	18.59	32.04
WH	0.031893	0.016458	0.029321	0.016007	8.06	2.74
WR	0.152189	0.116961	0.138137	0.110548	9.23	5.48
WS	0.009347	0.004358	0.009599	0.004705	-2.70	-7.96
SX	0.012692	0.006664	0.012615	0.006442	0.61	3.33
SH	0.017403	0.007808	0.017283	0.007227	0.69	7.44
SR	0.025757	0.014738	0.021212	0.011222	1.76	24.1
SS	0.007327	0.003564	0.007315	0.003871	0.16	-0.03

Table 11. Typical value of translation

Seq.	DS-SLAM		Multi		Improvement
Seq.	RMSE /deg	S.D /deg	RMSE /deg	S.D /deg	RMSE /%	S.D /%
WX	0.766283	0.529380	0.683158	0.440730	10.85	16.75
WH	0.846296	0.422084	0.824658	0.447438	2.56	-6.01
WR	3.043619	2.335232	2.797477	2.189081	8.09	6.26
WS	0.255013	0.106953	0.261905	0.122951	-2.70	-14.96
SX	0.495392	0.270148	0.488689	0.259726	0.14	3.86
SH	0.630755	0.300657	0.618015	0.289567	0.20	3.69
SR	0.843054	0.468068	0.681521	0.333695	19.16	28.71
SS	0.262147	0.117104	0.265180	0.110150	-1.16	5.94

Table 12. Typical value of rotation

Seq.	DS-SLAM		Multi		Improvement
Seq.	RMSE /m	S.D /m	RMSE /m	S.D /m	RMSE /%	S.D /%
WX	0.022180	0.014402	0.019148	0.010682	13.67	25.83
WH	0.032083	0.017715	0.028845	0.015730	10.09	11.21
WR	0.433820	0.228252	0.407781	0.205247	6.00	10.08
WS	0.007709	0.003275	0.007302	0.003459	5.28	-5.62
SX	0.010339	0.005377	0.009962	0.005129	3.65	4.61
SH	0.014816	0.006672	0.014589	0.006602	1.53	1.10
SR	0.020242	0.012680	0.016531	0.010268	18.33	19.02
SS	0.006273	0.003085	0.006142	0.003216	2.09	-4.25

Table 13. Typical value of ATE

Seq.	DS-SLAM		Semantic		Improvement
Seq.	RMSE /m	S.D /m	RMSE /m	S.D /m	RMSE /%	S.D /%
WX	0.030283	0.019977	0.020390	0.010275	32.67	48.57
WH	0.031893	0.016458	0.027500	0.014119	13.77	14.21
WR	0.152189	0.116961	0.072958	0.051138	52.06	56.28
WS	0.009347	0.004358	0.009172	0.004383	1.87	-0.57
SX	0.012692	0.006664	0.012570	0.006306	0.96	5.37
SH	0.017403	0.007808	0.017659	0.007570	-1.47	3.05
SR	0.025757	0.014738	0.020002	0.010545	22.34	28.45
SS	0.007327	0.003564	0.007121	0.003518	2.81	1.29

Table 14. Typical value of translation

Seq.	DS-SLAM		Semantic		Improvements
Seq.	RMSE /deg	S.D /deg	RMSE /deg	S.D /deg	RMSE /%	S.D /%
WX	0.766283	0.529380	0.614950	0.379957	19.75	28.23
WH	0.846296	0.422084	0.799037	0.402792	5.58	4.57
WR	3.043619	2.335232	1.471557	1.021622	51.65	56.25
WS	0.255013	0.106953	0.260383	0.114594	-2.11	-7.14
SX	0.495392	0.270148	0.486103	0.263159	1.88	2.59
SH	0.630755	0.300657	0.649584	0.314669	-2.99	-4.66
SR	0.843054	0.468068	0.667244	0.323382	20.85	30.91
SS	0.262147	0.117104	0.258282	0.113205	1.47	3.33

Table 15. Typical value of rotation

Seq.	DS-SLAM		Semantic		Improvements
Seq.	RMSE /m	S.D /m	RMSE /m	S.D /m	RMSE /%	S.D /%
WX	0.022180	0.014402	0.015859	0.008289	28.50	42.45
WH	0.032083	0.017715	0.026138	0.013565	18.53	23.43
WR	0.433820	0.228252	0.057395	0.041521	86.77	81.81
WS	0.007709	0.003275	0.007095	0.003276	7.96	-0.03
SX	0.010339	0.005377	0.010258	0.004892	0.78	9.02
SH	0.014816	0.006672	0.014299	0.006334	3.49	5.07
SR	0.020242	0.012680	0.016159	0.009816	20.17	22.59
SS	0.006273	0.003085	0.005903	0.002946	5.90	4.51

Table 16. Typical value of ATE

vSLAM	ORB extract	Moving check	Segmentation
Proposed	0.008118	0.012913	0.020810
DS-SLAM	0.008118	0.017299	0.027627

Table 17. Time comparison of feature point extraction, motion consistency judgment, and semantic segmentation

Download Citation

Set citation alerts for the article

Tools

Set citation alerts for the article

Save the article for my favorites

Paper Information