• Opto-Electronic Engineering
  • Vol. 48, Issue 12, 210340 (2021)
Zhang Ying, Huang Yingping*, Guo Zhiyang, and Zhang Chong
Author Affiliations
  • [in Chinese]
  • show less
    DOI: 10.12086/oee.2021.210340 Cite this Article
    Zhang Ying, Huang Yingping, Guo Zhiyang, Zhang Chong. Point cloud-image data fusion for road segmentation[J]. Opto-Electronic Engineering, 2021, 48(12): 210340 Copy Citation Text show less


    Road detection is the premise of vehicle automatic driving. In recent years, multi-modal data fusion based on deep learning has become a hot spot in the research of automatic driving. In this paper, convolutional neural network is used to fuse LiDAR point cloud and image datato realize road segmentation in traffic scenes. In this paper, a variety of fusion schemes at pixel level, feature level and decision level are proposed. Especially, four cross-fusion schemes are designed in feature level fusion. Various schemes are compared, and the best fusion scheme is given. In the network architecture, the semantic segmentation convolutional neural network with encoding and decoding structure is used as the basic network to cross-fuse the point cloud normal features and RGB image features at dif-ferent levels. The fused data is restored by the decoder, and finally the detection results are obtained by using the activation function. The substantial experiments have been conducted on public KITTI data set to evaluate the performance of various fusion schemes. The results show that the fusion scheme E proposed in this paper has the best segmentation performance. Compared with other road-detection methods, our method gives better overall performance.