• Infrared and Laser Engineering
  • Vol. 52, Issue 8, 20230427 (2023)
Shiqiang Wang, Zhaozong Meng, Nan Gao, and Zonghua Zhang
Author Affiliations
  • School of Mechanical Engineering, Hebei University of Technology, Tianjin 300130, China
  • show less
    DOI: 10.3788/IRLA20230427 Cite this Article
    Shiqiang Wang, Zhaozong Meng, Nan Gao, Zonghua Zhang. Advancements in fusion calibration technology of lidar and camera[J]. Infrared and Laser Engineering, 2023, 52(8): 20230427 Copy Citation Text show less
    References

    [1] X Q Xu. Survey of multi-sensor data fusion target recognition algorithms. Infrared and Laser Engineering, 35, 321-328(2006).

    [2] Liu Y P, Guo X J, Su S J, et al. Review of a 3D lidar combined with single vision calibration[C]IEEE International Conference on Data Science Computer Application (ICDSCA), 2021, 2021: 397404.

    [3] Kassir A, Peynot T. Reliable automatic cameralaser calibration[C]Australian Robotics Automation Association (ARAA), 2010: 110.

    [4] R K Mishra, Y Zhang. A review of optical imagery and airborne LiDAR data registration methods. The Open Remote Sensing Journal, 5, 54-63(2012).

    [5] Stamos I. Automated registration of 3Drange with 2Dcol images: an overview[C]44th Annual Conference on Infmation Sciences Systems (CISS), 2010: 16.

    [6] Y D Cui, R Chen, W B Chu, et al. Deep learning for image and point cloud fusion in autonomous driving: a review. IEEE Transactions on Intelligent Transportation Systems, 23, 722-739(2022).

    [7] Z Y Qiu, J Martínez-sánchez, P Arias-sánchez, et al. External multi-modal imaging sensor calibration for sensor fusion: A review. Information Fusion, 97, 101806(2023).

    [8] J Zhang, W T Jiang. Registration between laser scanning point cloud and optical images: status and trends. Journal of Geo-information Science, 19, 528-539(2017).

    [9] J Kannala, S S Brandt. A generic camera model and calibration method for conventional, wide-angle, and fish-eye lenses. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28, 1335-1340(2006).

    [10] Mei C, Rives P. Single view point omnidirectional camera calibration from planar grids[C]IEEE International Conference on Robotics Automation (ICRA), 2007: 39453950.

    [11] X R Yang, X W Zhang, S Y Cheng, et al. Progress on camera calibration methods in visual measurement. Machinery Design and Manufacture, 217, 259-261(2009).

    [12] Y I Abdel-aziz, H M Karara. Direct linear transformation from comparator coordinates into object space coordinates in close-range photogrammetry. Photogrammetric Engineering and Remote Sensing, 5, 103-107(2015).

    [13] Tsai R Y. An efficient accurate camera calibration technique f 3D machine vision[C]IEEE Conference of Computer Vision Pattern Recognition(ICCV), 1986: 364374.

    [14] Z Y Zhang. A flexible new technique for camera calibration. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22, 1330-1334(2000).

    [15] Faugeras O D, Luong Q T, Maybank S J. Camera selfcalibration: they experiments[C]European Conference on Computer Vision(ECCV), 1992: 321334.

    [16] Beardsley P, Murray D. Camera calibration using vanishing points[C]Proceedings of the British Machine Vision Conference (BMVC), 1992: 416425.

    [17] Q T Luong, O D Faugeras. Self calibration of a moving camera from point correspondences and fundamental matrices. International Journal of Computer Vision, 22, 261-289(1997).

    [18] S D Ma. A self-calibration technique for active vision systems. IEEE Transactions on Robotics and Automation, 12, 114-120(1996).

    [19] X L Zhang, K Jiang, K Sun, et al. Intrinsic parameters modeling for LiDAR and point cloud correction method. Automotive Engineering, 43, 1228-1237, 1253(2021).

    [20] D Y Qiao, W Z Yuan, Y Ren. Review of MEMS LiDAR. Microelectronics and Computer, 40, 41-49(2023).

    [21] C Glennie, D Lichti. Static calibration and analysis of the velodyne HDL-64 E S2 for high accuracy mobile scanning. Remote Sensing, 2, 1610-1624(2010).

    [22] Z Zhu, J L Liu. Calibration of a multi-beam LIDAR by using linear instrumental error model. Applied Mechanics and Materials, 380, 911-914(2013).

    [23] Muhammad N, Lacroix S. Calibration of a rotating multibeam lidar[C]IEEERSJ International Conference on Intelligent Robots Systems(IROS), 2010: 56485653.

    [24] Bergelt R, Khan O, Hardt W. Improving the intrinsic calibration of a velodyne LiDAR sens[C]IEEE Senss, 2017: 13.

    [25] L Zalud, P Kocmanova, F Burian, et al. Calibration and evaluation of parameters in a 3D proximity rotating scanner. Elektronir Elektrotechnika, 21, 3-12(2015).

    [26] T O Chan, D D Lichti. Automatic in situ calibration of a spinning beam LiDAR system in static and kinematic modes. Remote Sensing, 7, 10480-10500(2015).

    [27] J Levinson, S Thrun. Unsupervised calibration for multi-beam lasers. Experimental Robotics, 1, 179-193(2014).

    [28] N Sun, Q Qiu, Z Q Fan, et al. Intrinsic calibration of multi-beam LiDARs for agricultural robots. Remote Sensing, 14, 4846(2022).

    [29] B S Yang, F X Liang, R G Huang. Progress, challenges and perspectives of 3D lidar point cloud processing. Acta Geodaetica et Cartographica Sinica, 46, 1509-1516(2017).

    [30] R B Rusu, Z C Marton, N Blodow, et al. Towards 3D point cloud based object maps for household environments. Robotics and Autonomous Systems, 56, 927-941(2008).

    [31] Rusu R B, Cousins S. 3D is here: point cloud library (PCL)[C]IEEE International Conference on Robotics Automation (ICRA), 2011: 14.

    [32] P J Besl, R C Jain. Segmentation through variable-order surface fitting. IEEE Transactions on Pattern Analysis and Machine Intelligence, 10, 167-192(1988).

    [33] L Chen, Y Cai, J S Zhang. Adaptive k-means clustering simplification of scattered point cloud. Journal of Image and Graphics, 22, 1089-1097(2017).

    [34] Y H Zhang, G H Geng, X R Wei, et al. Feature extraction of point clouds using the DBSCAN clustering. Journal of Xidian University, 44, 114-120(2017).

    [35] Z L Sun, Z R Zhou, A Q Wu. An improved mean shift algorithm used for point cloud dat a filtering. Engineering of Surveying and Mapping, 20, 57-59(2011).

    [36] Z Y Wang, Y M Zhang, Y B Chen, et al. Optimization of indoor scene semantic segmentation network based on RGB-D image. Automation and Information Engineering, 41, 27-32(2020).

    [37] V Chernozhukov, D Chetverikov, M Demirer, et al. Double/debiased machine learning for treatment and structural parameters. The Econometrics Journal, 21, C1-C68(2017).

    [38] E C Lyons. Digital image processing: an overview. Computer, 10, 12-14(1977).

    [39] H Zhou, K Z Mao. An impulsive noise color image filter using learning-based color morphological operations. Digital Signal Processing, 18, 406-421(2008).

    [41] C L Yuan, Z L Xiong, X H Zhou, et al. Study of infrared image edge detection based on sobel operator. Laser and Infrared, 39, 85-87(2009).

    [42] J S Li, Y Ma, F Z Zhao, et al. A novel arithmetic of image edge detection of canny operator. Acta Photonica Sinica, 40, 50-54(2011).

    [43] M Y Xiao, X F Li, X L Zhang, et al. Medical image segmentation algorithm based on multi-scale region growing. Journal of Jilin University, 47, 1591-1597(2017).

    [44] G Q Wang, X H Zhou, L L Yu. Image segmentation based on watershed algorithm. Computer Simulation, 26, 255-258(2009).

    [45] Zhang Q L, Pless R. Extrinsic calibration of a camera laser range finder (improves camera calibration)[C]\IEEERSJ International Conference on Intelligent Robots Systems(IROS), 2004: 23012306.

    [46] Unnikrishnan R, Hebert M. Fast extrinsic calibration of a laser rangefinder to a camera[D]. Pittsburgh: Carnegie Mellon University, 2005.

    [47] Geiger A Moosmann F, Car O, et al. Automatic camera range sens calibration using a single shot[C]IEEE International Conference on Robotics Automation (ICRA), 2012: 1418.

    [48] Y Chen, Z Chen, S G Wei. Research on point cloud data segmentation based on extrinsic calibration of laser and CCD. Microcomputer Information, 24, 295-297(2008).

    [49] Z Y Han, C G Pu, C Liu. Extrinsic parameters calibration method for pinhole camera and 3D LiDAR. Transducer and Microsystem Technologies, 37, 9-12, 16(2018).

    [50] K Q Wei, Z F Shi, H Li, et al. Research on the joint calibration of monocular camera and 3D Lidar. Navigation Positioning and Timing, 2, 69-74(2015).

    [51] L Li, X Zhang, D W Tu. Joint calibration of 2D and 3D vision integrated sensor system. Chinese Journal of Scientific Instrument, 33, 2473-2479(2012).

    [52] Z Y Xiang, L Zheng. Novel joint calibration method of camera and 3D laser range finder. Journal of Zhejiang University(Engineering Science), 43, 1401-1405(2009).

    [53] H S Li, X N Luo, C G Deng, et al. Joint calibration of sports camera and lidar based on LM algorithm. Journal of Guilin University of Electronic Technology, 42, 345-353(2022).

    [54] G H Kang, Q Zhang, H Zhang, et al. Joint calibration of camera and lidar based on point cloud center. Chinese Journal of Scientific Instrument, 40, 118-126(2019).

    [55] L Zhou. A new minimal solution for the extrinsic calibration of a 2d lidar and a camera using three plane-line correspondences. IEEE Sensors Journal, 14, 442-454(2014).

    [56] F X Dong, J Cai, Y M Xie. Joint calibration method for stereo vision system and 3D laser system. Chinese Journal of Scientific Instrument, 38, 2589-2596(2017).

    [57] D Q Yu, G Y Li, L Wang, et al. Calibration of LiDAR and camera based on 3D feature point sets. Bulletin of Surveying and Mapping, 11, 40-45(2018).

    [58] J E Ha. Extrinsic calibration of a camera and laser range finder using a new calibration structure of a plane with a triangular hole. International Journal of Control, Automation and Systems, 10, 1240-1244(2012).

    [59] J Beltrán, C Guindel, A Escalera, et al. Automatic extrinsic calibration method for LiDAR and camera sensor setups. IEEE Transactions on Intelligent Transportation Systems, 23, 17677-17689(2022).

    [60] Debattisti S, Mazzei L, Panciroli M. Automated extrinsic laser camera intercalibration using triangular targets[C]IEEE Intelligent Vehicles Symposium (IV), 2013: 696–701.

    [61] Z Q Huang, Y Su, Q W Wang, et al. Research on extrinsic parameter calibration method of 2D laser rangefinder (LRF) and visible light camera. Chinese Journal of Scientific Instrument, 41, 121-129(2020).

    [62] Z H Deng, M Y Liu, M Y Fu. An improved method of matching point extracting for vision sensor and LMS space alignment. Optical Technique, 36, 43-47(2010).

    [63] Z Chen, L Zhuo, K Q Sun, et al. Extrinsic calibration of a camera and a laser range finder using point to line constraint. Procedia Engineering, 29, 4348-4352(2012).

    [64] W B Dong, V Isler. A novel method for the extrinsic calibration of a 2D laser rangefinder and a camera. IEEE Sensors Journal, 18, 4200-4211(2018).

    [65] Pusztai Z, Hajder L. Accurate calibration of LiDARcamera systems using dinary boxes[C]IEEE International Conference on Computer Vision Wkshops (ICCV), 2017: 394402.

    [66] S Fan, Y Yu, M Xu, et al. High-precision external parameter calibration method for camera and LiDAR based on a calibration device. IEEE Access, 11, 18750-18760(2023).

    [67] F Zhu, J Fan, Y C Huang, et al. Extrinsic calibration of camera and 2D laser-rangefinder with various chessboard constrains. Geomatics and Information Science of Wuhan University, 44, 1524-1529, 1537(2019).

    [68] X J Gong, Y Li, J L Liu. 3D Lidar-camera extrinsic calibration using an arbitrary trihedron. Sensors, 13, 1902-1918(2013).

    [69] Toth T, Pusztai Z, Hajder L. Automatic LiDARcamera calibration of extrinsic parameters using a spherical target[C]IEEE International Conference on Robotics Automation (ICRA), 2020: 85808586.

    [70] Liu C, Jiang Y, Yang M. Joint calibration of a camera a laser range finder from grid calibration target[C]IEEE International Conference on Cyber Technology in Automation, Control, Intelligent Systems (CYBER), 2016: 497501.

    [71] Moghadam P, Bosse M, Zlot R. Linebased extrinsic calibration of range image senss[C]\IEEE International Conference on Robotics Automation(ICRA), 2013: 6–10.

    [72] Scaramuzza D, Harati A, Siegwart R. Extrinsic selfcalibration of a camera a 3D laser range finder from natural scenes[C]IEEERSJ International Conference on Intelligent Robots Systems (IROS), 2007: 41644169.

    [73] GomezOjeda R, Briales J, FernezMal E, et al. Extrinsic calibration of a 2D laserrangefinder a camera based on scene cners[C]IEEE International Conference on Robotics Automation (ICRA), 2015: 36113616.

    [74] Z X Bai, G Jiang, A L Xu. LiDAR-camera calibration using line correspondences. Sensors, 20, 6319-6336(2020).

    [75] J Kang, N L Doh. Automatic targetless camera-LiDAR calibration by aligning edge with Gaussian mixture model. Journal of Field Robotics, 37, 25-37(2020).

    [76] Levinson J, Thrun S. Automatic online calibration of cameras lasers[MOL]. (20130624)[2023027]. https:rss2013.robotics.tuberlin.deimagesbooklet.online.pdf.

    [77] C J Yuan, X Y Liu, X P Hong, et al. Pixel-level extrinsic self calibration of high resolution LiDAR and camera in targetless environments. IEEE Robotics and Automation Letters, 6, 7517-7524(2021).

    [78] X Y Liu, C J Yuan, F Zhang. Targetless Extrinsic Calibration of Multiple Small FoV LiDARs and Cameras Using Adaptive Voxelization. IEEE Transactions on Instrumentation and Measurement, 71, 1-12(2022).

    [79] Wang W M, Nobuhara S, Nakamura R, et al. SOIC: semantic online initialization calibration f LiDAR camera[EBOL]. (20200309)[20230615]. https:arxiv.gabs2003.04260.

    [80] Abedini A. An investigation into the registration of LiDAR intensity data aerial images using the SIFT approach[C]The International Archives of The Photoghrammetry, Remote Sensing Spatial Infmation Sciences(ISPRS), 2008: 169–176.

    [81] Pey G, McBride J, Savarese S, et al. Automatic targetless extrinsic calibration of a 3D LiDAR camera by maximizing mutual infmation[C]AAAI Conference on Artificial Intelligence (AAAI), 2012, 26(1): 20532059.

    [82] G Pandey, J R Mcbride, S Savarese, et al. Automatic extrinsic calibration of vision and LiDAR by maximizing mutual information. Journal of Field Robotics, 32, 696-722(2015).

    [83] Tayl Z, Nieto J. A mutual infmation approach to automatic calibration of camera lidar in natural environments[C]Australian Conference on Robotics Automation (ACRA), 2012: 39.

    [84] W S Song, Z H Zhang, N Gao, et al. Spatial pose calibration method for lidar and camera based on intensity information. Laser & Optoelectronics Progress, 59, 356-364(2022).

    [85] Huang K H, Stachniss C. Extrinsic multisens calibration f mobile robots using the GaussHelmert model[C]IEEE, RSJ International Conference on Intelligent Robots Systems (IROS), 2017: 19701976.

    [86] Tayl Z, Nieto J. Motionbased calibration of multi modal sens arrays[C]\IEEE International Conference on Robotics Automation (ICRA), 2015: 48434850.

    [87] Ishikawa R, Oishi T, Ikeuchi K. LiDAR camera calibration using motion estimated by sens fusion odometry[C]IEEERSJ International Conference on Intelligent Robots Systems (IROS), 2018: 73427349.

    [88] W Zhao, D Nister, S Hsu. Alignment of continuous video onto 3D point clouds. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27, 1305-1318(2005).

    [89] C Yang, T J Li, H Hu, et al. Online lidar calibration algorithm based on nonlinear optimization. Computer Engineering and Design, 42, 1919-1925(2021).

    [90] Nedevschi S. Online crosscalibration of camera lidar[C]IEEE International Conference on Intelligent Computer Communication Processing (ICCP), 2017: 295301.

    [91] Schneider N, Piewak F, Stiller C, et al. Reg: multimodal sens registration using deep neural wks[C]IEEE Intelligent Vehicles Symposium (IV), 2017: 18031810.

    [92] Iyer G, Ram R K, Murthy J K, et al. Calib: Geometrically supervised extrinsic calibration using 3D spatial transfmer wks[C]IEEERSJ International Conference on Intelligent Robots Systems (IROS), 2018: 11101117.

    [93] K Yuan, Z Guo, Z J Wang. RGGNet: Tolerance aware LiDAR-camera online calibration with geometric deep learning and generative model. IEEE Robotics and Automation Letters, 5, 6956-6963(2020).

    [94] Lv X D, Wang B, Dou Z, et al. LCC: LiDAR camera selfcalibration using cost volume wk[C]IEEE Conference on Computer Vision Pattern Recognition Wkshops (CVPR), 2021: 28882895.

    [95] Cattaneo D, Vaghi M, Ballardini A L, at el. CMR: Camera to LiDARmap registration[C]IEEE Intelligent Transptation Systems Conference(ITSC), 2019: 12831289.

    [96] Shi J Y, Zhu Z H, Zhang J H, et al. CalibRCNN: Calibrating camera LiDAR by recurrent convolutional neural wk geometric constraints[C]IEEE, RSJ International Conference on Intelligent Robots Systems(IROS), 2020: 12831289.

    [97] Zhao G N, Hu J S, You S Y, et al. CalibDNN: multimodal sens calibration f perception using deep neural wks[C]Conference on Signal Processing, SensInfmation Fusion, Target Recognition, 2021: 324335.

    [98] X D Lv, S Wang, D Ye. CFNet: LiDAR-camera registration using calibration flow network. Sensors, 21, 8112-8124(2021).

    [99] Jing X, Ding X Q, Xiong R, et al. DXQ: Differentiable LiDARcamera extrinsic calibration using qualityaware flow[C]International Conference on Intelligent Robots Systems(IROS), 2022: 62356241.

    [100] W Yi, M Zhu, J Liang. PSNet: LiDAR and camera registration using parallel Subnetworks. IEEE Access, 10, 70553-70561(2022).

    [101] Y Sun, J Li, Y Wang, et al. ATOP: An attention-to-optimization approach for automatic LiDAR-camera calibration via cross-modal object matching. IEEE Transactions on Intelligent Vehicles, 8, 696-708(2023).

    Shiqiang Wang, Zhaozong Meng, Nan Gao, Zonghua Zhang. Advancements in fusion calibration technology of lidar and camera[J]. Infrared and Laser Engineering, 2023, 52(8): 20230427
    Download Citation