• Laser & Optoelectronics Progress
  • Vol. 57, Issue 4, 040002 (2020)
Jiaying Zhang, Xiaoli Zhao*, and Zheng Chen
Author Affiliations
  • School of Electronic and Electrical Engineering, Shanghai University of Engineering Science, Shanghai 201620, China
  • show less
    DOI: 10.3788/LOP57.040002 Cite this Article Set citation alerts
    Jiaying Zhang, Xiaoli Zhao, Zheng Chen. Review of Semantic Segmentation of Point Cloud Based on Deep Learning[J]. Laser & Optoelectronics Progress, 2020, 57(4): 040002 Copy Citation Text show less
    Fine-grained annotations of 24 objects in the PartNet dataset[26]
    Fig. 1. Fine-grained annotations of 24 objects in the PartNet dataset[26]
    Models of chairs and laptops in the ShapeNet Part dataset[28]
    Fig. 2. Models of chairs and laptops in the ShapeNet Part dataset[28]
    Point cloud scene and semantic segmentation diagram in S3DIS dataset[29]
    Fig. 3. Point cloud scene and semantic segmentation diagram in S3DIS dataset[29]
    Annotated indoor scene maps in ScanNet dataset[30]
    Fig. 4. Annotated indoor scene maps in ScanNet dataset[30]
    Examples of Semantic3D dataset[31]. (a) Point cloud scene; (b) diagram of intensity; (c) diagram of semantic segmentation
    Fig. 5. Examples of Semantic3D dataset[31]. (a) Point cloud scene; (b) diagram of intensity; (c) diagram of semantic segmentation
    Semantic segmentation results of outdoor scene in vKITTI dataset[32]
    Fig. 6. Semantic segmentation results of outdoor scene in vKITTI dataset[32]
    Visual representation of point cloud semantic segmentation methods
    Fig. 7. Visual representation of point cloud semantic segmentation methods
    Framework of multi-view convolutional neural network (MVCNN)[37]
    Fig. 8. Framework of multi-view convolutional neural network (MVCNN)[37]
    Overall flow of semantic marks of SnapNet-R network [39]
    Fig. 9. Overall flow of semantic marks of SnapNet-R network [39]
    Overall architecture of semantic segmentation network of SEGCloud[49]
    Fig. 10. Overall architecture of semantic segmentation network of SEGCloud[49]
    Framework of PointNet for point cloud classification and segmentation[16]
    Fig. 11. Framework of PointNet for point cloud classification and segmentation[16]
    Overall framework of RSNet for point cloud semantic segmentation[53]
    Fig. 12. Overall framework of RSNet for point cloud semantic segmentation[53]
    Point cloud semantic segmentation network of SO-Net formed by SOM[54]
    Fig. 13. Point cloud semantic segmentation network of SO-Net formed by SOM[54]
    Applications of hierarchical convolution in regular gird and point clouds, and PointCNN framework used for semantic segmentation[18]. (a) Application of hierarchical convolution; (b) PointCNN framework
    Fig. 14. Applications of hierarchical convolution in regular gird and point clouds, and PointCNN framework used for semantic segmentation[18]. (a) Application of hierarchical convolution; (b) PointCNN framework
    Architecture of PointNet++ for point cloud classification and segmentation[17]
    Fig. 15. Architecture of PointNet++ for point cloud classification and segmentation[17]
    Overall architectures of PointSIFT module and point segementation of PointSIFT[58]. (a) Structure; (b) whole architecture
    Fig. 16. Overall architectures of PointSIFT module and point segementation of PointSIFT[58]. (a) Structure; (b) whole architecture
    Architecture of A-CNN for point cloud classification and segmentation[61]
    Fig. 17. Architecture of A-CNN for point cloud classification and segmentation[61]
    Point cloud semantic segmentation network of 3DMAX-Net[60] (MS-FLB: multi-scale feature learning block; LGAB: local and global feature aggregation block)
    Fig. 18. Point cloud semantic segmentation network of 3DMAX-Net[60] (MS-FLB: multi-scale feature learning block; LGAB: local and global feature aggregation block)
    Framework of 3P-RNN for point cloud semantic segmentation[62]
    Fig. 19. Framework of 3P-RNN for point cloud semantic segmentation[62]
    Network structural diagram of LDGCNN for point cloud classification and segmentation[65]
    Fig. 20. Network structural diagram of LDGCNN for point cloud classification and segmentation[65]
    Network structural diagram of RGCNN for point cloud classification and segmentation[67]
    Fig. 21. Network structural diagram of RGCNN for point cloud classification and segmentation[67]
    Network framework of GAPNet for point cloud semantic segmentation[68]
    Fig. 22. Network framework of GAPNet for point cloud semantic segmentation[68]
    Forward time of different network models
    Fig. 23. Forward time of different network models
    DatasetNumber ofcategoriesNumber oftraining setsNumber ofverification setsNumber oftesting sets
    PartNet[26]24---
    UWA Dataset[27]55---
    ShapeNet Part[28]161213718702874
    S3DIS[29]13224-48
    ScanNet[30]211201-312
    Semantic3D[31]815-15
    KITTI(Zhang)[34]10140-112
    KITTI(Ros)[35]11170-46
    vKITTI[32]13---
    Table 1. Common datasets of point cloud segmentation
    ModelNetwork parameter quantity /MB
    Subvolume16.6
    MVCNN60
    PointNet3.48
    PointNet++1.48
    DGCNN1.84
    LDGCNN1.08
    PointCNN0.6
    Table 2. Network parameter quantity of different semantic segmentation models
    ModelS3DISScanNetShapeNet PartSemantic 3DvKITTI
    mIoUOA /%mIoUOA /%mIoUmIoUOA /%mIoUOA /%
    Yi[69]----81.4----
    KD-Net[47]----82.3----
    SEGCloud[49]48.92--73.079.461.388.1--
    PointNet[16]47.7178.6214.6973.983.7--34.479.7
    PN++(SSG)[17]---83.3-----
    PN++(MSG+DP)[17]--34.2684.585.1----
    PN++(MRG+DP)[17]---83.4-----
    O-CNN+CRF[70]----85.9----
    SSCNN[71]----84.7----
    MS+CU[72]47.879.2-------
    G+RCU[72]49.781.1-----36.280.6
    DGCNN[64]56.184.1--85.1----
    RGCNN[67]----84.3----
    RSNet[53]53.8361.8139.35-84.9----
    SO-Net[54]----84.6----
    TMLC-MSR[73]-----54.286.2--
    DeePr3SS[74]-----58.588.9--
    SnapNet[38]-----59.188.6--
    SGPN[75]50.3780.78--85.8----
    SpiderCNN[59]----85.3----
    3DMAX-Net[60]47.579.5-------
    SPGraph[76]62.185.5---73.294.0--
    3P-RNN[62]56.386.9-----41.687.8
    PointCNN[18]62.7488.1-85.1-----
    PointSIFT[58]70.2388.72-86.2-----
    ASIS[77]59.386.2-------
    A-CNN[61]-87.3-------
    LDGCNN[65]----85.1----
    GAPNet[68]----84.7----
    Table 3. Segmentation results of different models on typical point cloud datasets
    Jiaying Zhang, Xiaoli Zhao, Zheng Chen. Review of Semantic Segmentation of Point Cloud Based on Deep Learning[J]. Laser & Optoelectronics Progress, 2020, 57(4): 040002
    Download Citation