• Acta Photonica Sinica
  • Vol. 53, Issue 1, 0111002 (2024)
Wuyue WANG1,*, Zhaofei XU2,3, Chunyan QU3, Ying LIN4..., Yufeng CHEN4 and Jian LIAO1|Show fewer author(s)
Author Affiliations
  • 1Yantai Research Institute,Harbin Engineering University,Yantai 265500,China
  • 2Mechanical and Electrical Engineering Institute,Harbin Engineering University,Harbin 150000,China
  • 3Iray Optoelectronic Technology Co.,LTD,Yantai 265500,China
  • 4Electric Power Research Institute,State Grid Shandong Electric Power Company,Jinan 250014,China
  • show less
    DOI: 10.3788/gzxb20245301.0111002 Cite this Article
    Wuyue WANG, Zhaofei XU, Chunyan QU, Ying LIN, Yufeng CHEN, Jian LIAO. BEV Space 3D Object Detection Algorithm Based on Fusion of Infrared Camera and LiDAR[J]. Acta Photonica Sinica, 2024, 53(1): 0111002 Copy Citation Text show less
    Frame figure of separable fusion sensing system
    Fig. 1. Frame figure of separable fusion sensing system
    Image encoder module
    Fig. 2. Image encoder module
    DepthNet
    Fig. 3. DepthNet
    Improved DepthNet
    Fig. 4. Improved DepthNet
    BEV pooling accelerator kernel
    Fig. 5. BEV pooling accelerator kernel
    Lidar branch structure
    Fig. 6. Lidar branch structure
    Gating attention fusion mechanism module
    Fig. 7. Gating attention fusion mechanism module
    Data collection hardware platform and layout position
    Fig. 8. Data collection hardware platform and layout position
    Dataset scene distribution
    Fig. 9. Dataset scene distribution
    Embedded AI computing platform MIIVII APEX AD10
    Fig. 10. Embedded AI computing platform MIIVII APEX AD10
    City road scene test(RVIZ)
    Fig. 11. City road scene test(RVIZ)
    DSVCPPBEV mAP/%3D mAP/%AOS/%
    26.6923.4446.74
    30.2725.1657.92
    31.8925.9559.31
    Table 1. Ablation experiment of camera branch
    GAFBEV mAP/%3D mAP/%AOS/%
    76.7266.9569.30
    77.1468.0870.19
    Table 2. Ablation experiment of fusion branch
    V methodBEV AP/%3D AP/%AOS/%
    Car pedestrian cyclistCar pedestrian cyclistCar pedestrian cyclist
    SV89.9177.2980.3187.7677.0980.2385.5757.1476.97
    DV90.2078.1980.1588.6778.1380.0985.4858.8377.41
    Table 3. Performance comparison of different voxelization method in lidar branches
    ModalityDSVCPPDVGAFBEV AP/%3D AP/%AOS/%
    Car pedestrian cyclistCar pedestrian cyclistCar pedestrian cyclist
    Camera64.6731.5557.1958.4629.7155.6666.4544.6966.80
    Lidar90.2078.1980.1588.6778.1380.0985.4858.8377.41
    Lidar & camera91.2577.3284.0690.8477.1582.8885.8060.1980.46
    Table 4. Performance comparison of camera branch,lidar branch and fusion branch
    MethodModalityBEV AP/%3D AP/%AOS/%
    Car pedestrian cyclistCar pedestrian cyclistCar pedestrian cyclist
    PointPillars17Lidar81.7268.3678.4681.6568.3578.4279.6651.9375.31
    CenterPoint18Lidar90.2078.1980.1588.6778.1380.0985.4858.8377.41
    MVXNet24Lidar & camera89.1976.1673.1689.0176.1371.7182.9149.0561.03
    OursLidar & camera91.2577.3284.0690.8477.1582.8885.8060.1980.46
    Table 5. Performance comparison of our model with other SOTA model
    Wuyue WANG, Zhaofei XU, Chunyan QU, Ying LIN, Yufeng CHEN, Jian LIAO. BEV Space 3D Object Detection Algorithm Based on Fusion of Infrared Camera and LiDAR[J]. Acta Photonica Sinica, 2024, 53(1): 0111002
    Download Citation