• Laser & Optoelectronics Progress
  • Vol. 60, Issue 3, 0312010 (2023)
Zhenzhong Wei*, Guangkun Feng, Danya Zhou, Yueming Ma..., Mingkun Liu, Qifeng Luo and Tengda Huang|Show fewer author(s)
Author Affiliations
  • Key Laboratory of Precision Opto-Mechatronics Technology, Ministry of Education, School of Instrumentation and Optoelectronic Engineering, Beihang University, Beijing 100191, China
  • show less
    DOI: 10.3788/LOP223420 Cite this Article Set citation alerts
    Zhenzhong Wei, Guangkun Feng, Danya Zhou, Yueming Ma, Mingkun Liu, Qifeng Luo, Tengda Huang. A Review of Position and Orientation Visual Measurement Methods and Applications[J]. Laser & Optoelectronics Progress, 2023, 60(3): 0312010 Copy Citation Text show less
    References

    [1] Bi D M, Huang H, Fan J et al. Nonlinear moving path following control and its application to carrier landing[J]. Journal of Beijing University of Aeronautics and Astronautics, 47, 45-55(2021).

    [2] Wei Z Z. Overview of visual measurement technology for landing position and attitude of carrier-based aircraft[J]. Measurement & Control Technology, 39, 2-6(2020).

    [3] Li D F, Zhang Y C, Liu C et al. Review of photoacoustic imaging for microrobots tracking in vivo[J]. Chinese Optics Letters, 19, 111701(2021).

    [4] Zhou L Y, Huang X W, Fu Q et al. Fine edge detection in single-pixel imaging[J]. Chinese Optics Letters, 19, 121101(2021).

    [5] Gao Z R, Su Y, Zhang Q C. Single-event-camera-based 3D trajectory measurement method for high-speed moving targets[J]. Chinese Optics Letters, 20, 061101(2022).

    [6] Harris C G, Stephens M. A J combined corner and edge detector[C], 147-151(1988).

    [7] Lowe D G. Distinctive image features from scale-invariant keypoints[J]. International Journal of Computer Vision, 60, 91-110(2004).

    [8] David P, DeMenthon D, Duraiswami R et al. SoftPOSIT: simultaneous pose and correspondence determination[J]. International Journal of Computer Vision, 59, 259-284(2004).

    [9] Ravela S, Draper B, Lim J et al. Adaptive tracking and model registration across distinct aspects[C], 174-180(2002).

    [10] Bay H, Ess A, Tuytelaars T et al. Speeded-up robust features (SURF)[J]. Computer Vision and Image Understanding, 110, 346-359(2008).

    [11] Rublee E, Rabaud V, Konolige K et al. ORB: an efficient alternative to SIFT or SURF[C], 2564-2571(2012).

    [12] Haralick R M, Lee D, Ottenburg K et al. Analysis and solutions of the three point perspective pose estimation problem[C], 592-598(2002).

    [13] Horaud R, Conio B, Leboulleux O et al. An analytic solution for the perspective 4-point problem[J]. Computer Vision, Graphics, and Image Processing, 47, 33-44(1989).

    [14] Lepetit V, Moreno-Noguer F, Fua P. EPnP: an accurate O(n) solution to the PnP problem[J]. International Journal of Computer Vision, 81, 155-166(2009).

    [15] Li S Q, Xu C, Xie M. A robust O(n) solution to the perspective-n-point problem[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34, 1444-1450(2012).

    [16] Zheng Y Q, Kuang Y B, Sugimoto S et al. Revisiting the PnP problem: a fast, general and optimal solution[C], 2344-2351(2014).

    [17] Fischler M A, Bolles R C. Random sample consensus[J]. Communications of the ACM, 24, 381-395(1981).

    [18] Rad M, Lepetit V. BB8: a scalable, accurate, robust to partial occlusion method for predicting the 3D poses of challenging objects without using depth[C], 3848-3856(2017).

    [19] Pavlakos G, Zhou X W, Chan A et al. 6-DoF object pose from semantic keypoints[C], 2011-2018(2017).

    [20] Tekin B, Sinha S N, Fua P. Real-time seamless single shot 6D object pose prediction[C], 292-301(2018).

    [21] Fan R Z, Xu T B, Wei Z Z. Estimating 6D aircraft pose from keypoints and structures[J]. Remote Sensing, 13, 663(2021).

    [22] Peng S D, Liu Y, Huang Q X et al. PVNet: pixel-wise voting network for 6DoF pose estimation[C], 4556-4565(2020).

    [23] Li Z G, Wang G, Ji X Y. CDPN: coordinates-based disentangled pose network for real-time RGB-based 6-DoF object pose estimation[C], 7677-7686(2020).

    [24] Zakharov S, Shugurov I, Ilic S. DPOD: 6D pose object detector and refiner[C], 1941-1950(2020).

    [25] Hu Y L, Fua P, Wang W et al. Single-stage 6D object pose estimation[C], 2927-2936(2020).

    [26] Chen H S, Wang P C, Wang F et al. EPro-PnP: generalized end-to-end probabilistic perspective-n-points for monocular object pose estimation[C], 2771-2780(2022).

    [27] Wang G, Manhardt F, Tombari F et al. GDR-net: geometry-guided direct regression network for monocular 6D object pose estimation[C], 16606-16616(2021).

    [28] Duda R O, Hart P E. Use of the Hough transformation to detect lines and curves in pictures[J]. Communications of the ACM, 15, 11-15(1972).

    [29] von Gioi R G, Jakubowicz J, Morel J M et al. LSD: a fast line segment detector with a false detection control[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32, 722-732(2010).

    [30] Akinlar C, Topal C. EDLines: a real-time line segment detector with a false detection control[J]. Pattern Recognition Letters, 32, 1633-1642(2011).

    [31] Xue N, Bai S, Wang F D et al. Learning attraction field representation for robust line segment detection[C], 1595-1603(2020).

    [32] Wang Z H, Wu F C, Hu Z Y. MSLD: a robust descriptor for line matching[J]. Pattern Recognition, 42, 941-953(2009).

    [33] Zhang L L, Koch R. Line matching using appearance similarities and geometric constraints[M]. Pinz A, Pock T, Bischof H, et al. Joint DAGM (German Association for Pattern Recognition) and OAGM symposium. Lecture notes in computer science, 7476, 236-245(2012).

    [34] Wang P, He W L, Zhang A H et al. EPnL: an efficient and accurate algorithm to the PnL problem[J]. Acta Automatica Sinica, 48, 2600-2610(2022).

    [35] Dhome M, Richetin M, Lapreste J T et al. Determination of the attitude of 3D objects from a single perspective view[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 11, 1265-1278(1989).

    [36] Liu Y, Huang T S, Faugeras O D. Determination of camera location from 2-D to 3-D line and point correspondences[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 12, 28-37(1990).

    [37] Kumar R, Hanson A R. Robust methods for estimating pose and a sensitivity analysis[J]. CVGIP: Image Understanding, 60, 313-342(1994).

    [38] David P, DeMenthon D, Duraiswami R et al. Simultaneous pose and correspondence determination using line features[C], 424-431(2003).

    [39] Ansar A, Daniilidis K. Linear pose estimation from points or lines[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25, 578-589(2003).

    [40] Xu C, Zhang L L, Cheng L et al. Pose estimation from line correspondences: a complete analysis and a series of solutions[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39, 1209-1222(2017).

    [41] Přibyl B, Zemčík P, Čadík M. Absolute pose estimation from line correspondences using direct linear transformation[J]. Computer Vision and Image Understanding, 161, 130-144(2017).

    [42] Mirzaei F M, Roumeliotis S I. Globally optimal pose estimation from line correspondences[C], 5581-5588(2011).

    [43] Zhang L L, Xu C, Lee K M et al. Robust and efficient pose estimation from line correspondences[M]. Lee K M, Matsushita Y, Rehg J M, et al. Computer vision-ACCV 2012. Lecture notes in computer science, 7726, 217-230(2013).

    [44] Wang P, Xu G L, Cheng Y H et al. Camera pose estimation from lines: a fast, robust and general method[J]. Machine Vision and Applications, 30, 603-614(2019).

    [45] Wang B, Zhong F, Qin X Y. Robust edge-based 3D object tracking with direction-based pose validation[J]. Multimedia Tools and Applications, 78, 12307-12331(2019).

    [46] Harris C, Stennett C. RAPID-a video rate object tracker[C], 1-6(1990).

    [47] Vacchetti L, Lepetit V, Fua P. Combining edge and texture information for real-time accurate 3D camera tracking[C], 48-56(2005).

    [48] Yoon Y, Kosaka A, Kak A C. A new Kalman-filter-based framework for fast and accurate visual tracking of rigid objects[J]. IEEE Transactions on Robotics, 24, 1238-1251(2008).

    [49] Teulière C, Marchand E, Eck L. Using multiple hypothesis in model-based tracking[C], 4559-4565(2010).

    [50] Hodaň T, Baráth D, Matas J. EPOS: estimating 6D pose of objects with symmetries[C], 11700-11709(2020).

    [51] Pitteri G, Bugeau A, Ilic S et al. 3D Object detection and pose estimation of unseen objects in color images with local surface embeddings[M]. Ishikawa H, Liu C L, Pajdla T, et al. Computer Vision-ACCV 2020. Lecture notes in computer science, 12622, 38-54(2021).

    [52] Shi Y F, Huang J W, Xu X et al. StablePose: learning 6D object poses from geometrically stable patches[C], 15217-15226(2021).

    [53] Schmaltz C, Rosenhahn B, Brox T et al. Region-based pose tracking[M]. Martí J, Benedí J M, Mendonça A M, et al. Iberian conference on pattern recognition and image analysis. Lecture notes in computer science, 4478, 56-63(2007).

    [54] Brox T, Rosenhahn B, Gall J et al. Combined region and motion-based 3D tracking of rigid and articulated objects[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32, 402-415(2010).

    [55] Bibby C, Reid I. Robust real-time visual tracking using pixel-wise posteriors[M]. Forsyth D, Torr P, Zisserman Z. Computer vision-ECCV 2008, 5303, 831-844(2008).

    [56] Prisacariu V A, Reid I D. PWP3D: real-time segmentation and tracking of 3D objects[J]. International Journal of Computer Vision, 98, 335-354(2012).

    [57] Tjaden H, Schwanecke U, Schömer E. Real-time monocular pose estimation of 3D objects using temporally consistent local color histograms[C], 124-132(2017).

    [58] Liu F L, Wei Z Z, Zhang G J. An off-board vision system for relative attitude measurement of aircraft[J]. IEEE Transactions on Industrial Electronics, 69, 4225-4233(2022).

    [59] Zhong L S, Zhang Y, Zhao H et al. Seeing through the occluders: robust monocular 6-DOF object pose tracking via model-guided video object segmentation[J]. IEEE Robotics and Automation Letters, 5, 5159-5166(2020).

    [60] Chen L C, Zhu Y K, Papandreou G et al. Encoder-decoder with atrous separable convolution for semantic image segmentation[M]. Ferrari V, Hebert M, Sminchisescu C, et al. Computer vision-ECCV 2018. Lecture notes in computer science, 11211, 833-851(2018).

    [61] Zhong L S, Zhang L. A robust monocular 3D object tracking method combining statistical and photometric constraints[J]. International Journal of Computer Vision, 127, 973-992(2019).

    [62] Zhong L S, Zhao X L, Zhang Y et al. Occlusion-aware region-based 3D pose tracking of objects with temporally consistent polar-based local partitioning[J]. IEEE Transactions on Image Processing, 29, 5065-5078(2020).

    [63] Choi C, Christensen H I. Real-time 3D model-based tracking using edge and keypoint features for robotic manipulation[C], 4048-4055(2010).

    [64] Choi C, Christensen H I. Robust 3D visual tracking using particle filtering on the SE(3) group[C], 4384-4390(2011).

    [65] Pauwels K, Rubio L, Díaz J et al. Real-time model-based rigid object pose estimation and tracking combining dense and sparse visual cues[C], 2347-2354(2013).

    [66] Tuzel O, Liu M Y, Taguchi Y et al. Learning to rank 3D features[M]. Fleet D, Pajdla T, Schiele B, et al. Computer vision-ECCV 2014. Lecture notes in computer science, 8689, 520-535(2014).

    [67] Hu Y L, Hugonot J, Fua P et al. Segmentation-driven 6D object pose estimation[C], 3380-3389(2020).

    [68] Song C, Song J R, Huang Q X. HybridPose: 6D object pose estimation under hybrid representations[C], 428-437(2020).

    [69] Hartley R, Zisserman A[M]. Multiple view geometry in computer vision(2000).

    [70] Wahba G. A least squares estimate of satellite attitude[J]. SIAM Review, 7, 409(1965).

    [71] Liu X Y, Jonschkowski R, Angelova A et al. KeyPose: multi-view 3D labeling and keypoint estimation for transparent objects[C], 11599-11607(2020).

    [72] Li P L, Chen X Z, Shen S J. Stereo R-CNN based 3D object detection for autonomous driving[C], 7636-7644(2020).

    [73] Triggs B, McLauchlan P F, Hartley R I et al. Bundle adjustment: a modern synthesis[M]. Triggs B, Zisserman A, Szeliski R. Vision algorithms: theory and practice. Lecture notes in computer science, 1883, 298-372(2000).

    [74] Hirschmüller H. Stereo processing by semiglobal matching and mutual information[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 30, 328-341(2008).

    [75] Kendall A, Martirosyan H, Dasgupta S et al. End-to-end learning of geometry and context for deep stereo regression[C], 66-75(2017).

    [76] Rusu R B, Marton Z C, Blodow N et al. Persistent point feature histograms for 3D point clouds[C], 119-128(2008).

    [77] Rusu R B, Blodow N, Beetz M. Fast point feature histograms (FPFH) for 3D registration[C], 3212-3217(2009).

    [78] Salti S, Tombari F, Di Stefano L. SHOT: Unique signatures of histograms for surface and texture description[J]. Computer Vision and Image Understanding, 125, 251-264(2014).

    [79] Besl P J, McKay N D. Method for registration of 3-D shapes[J]. Proceedings of SPIE, 1611, 586-606(1992).

    [80] Geng M Z. The application of visual navigation in auto-landing of unmanned aerial vehicle[D], 74-80(2007).

    [81] Zhang Z Y. Research on multi-source information based guidance technology for unmanned aerial vehicle autonomous carrier-landing[D], 51-67(2017).

    [82] Mourikis A I, Trawny N, Roumeliotis S I et al. Vision-aided inertial navigation for spacecraft entry, descent, and landing[J]. IEEE Transactions on Robotics, 25, 264-280(2009).

    [83] Li J J, Wang D Y. An image-based autonomous navigation method for precise landing on Mars[J]. Journal of Astronautics, 37, 687-694(2016).

    [84] Yu X Z. Research on head pose estimation and follow-up control based on multi-sensor[D], 34-47(2022).

    [85] Wang N. Vision navigation technology based on multi-source information fusion for UAV[D], 55-62(2019).

    [86] Zuo X X, Yang Y L, Geneva P et al. LIC-fusion 2.0: LiDAR-inertial-camera odometry with sliding-window plane-feature tracking[C], 5112-5119(2021).

    [87] Xin Q, Shi Z K. Flight attitude determination base on multiple measurements[J]. Flight Dynamics, 30, 527-531(2012).

    [88] Zhang S K. State estimation for micro aerial vehicles via wireless-inertial fusion[D], 68-92(2021).

    [89] Cheng J Y, Yi J J, He L et al. Improvement of shipboard landing performance of shipborne UAV using multi-sensor fusion[C], 304-309(2019).

    [90] He Y S, Huang H B, Fan H Q et al. FFB6D: a full flow bidirectional fusion network for 6D pose estimation[C], 3002-3012(2021).

    [91] He Y S, Sun W, Huang H B et al. PVN3D: a deep point-wise 3D keypoints voting network for 6DoF pose estimation[C], 11629-11638(2020).

    [92] Wang C, Xu D F, Zhu Y K et al. DenseFusion: 6D object pose estimation by iterative dense fusion[C], 3338-3347(2020).

    [93] Chatterji G B, Menon P K, Sridhar B. Vision-based position and attitude determination for aircraft night landing[J]. Journal of Guidance, Control, and Dynamics, 21, 84-92(1998).

    [94] Oszust M, Kapuscinski T, Warchol D et al. A vision-based method for supporting autonomous aircraft landing[J]. Aircraft Engineering and Aerospace Technology, 90, 973-982(2018).

    [95] Gibert V, Burlion L, Chriette A et al. New pose estimation scheme in perspective vision system during civil aircraft landing[J]. IFAC-PapersOnLine, 48, 238-243(2015).

    [96] Gibert V, Plestan F, Burlion L et al. Visual estimation of deviations for the civil aircraft landing[J]. Control Engineering Practice, 75, 17-25(2018).

    [97] Meng Y, Wang W, Han H et al. A vision/radar/INS integrated guidance method for shipboard landing[J]. IEEE Transactions on Industrial Electronics, 66, 8803-8810(2019).

    [99] Martínez C, Campoy P, Mondragón I et al. Trinocular ground system to control UAVs[C], 3361-3367(2009).

    [100] Ma Z W, Hu T J, Shen L C. Stereo vision guiding for the autonomous landing of fixed-wing UAVs: a saliency-inspired approach[J]. International Journal of Advanced Robotic Systems, 13, 43(2016).

    [101] Santos N P, Lobo V, Bernardino A. Autoland project: fixed-wing UAV landing on a fast patrol boat using computer vision[C], 1-5(2019).

    [102] Santos N P, Lobo V, Bernardino A. Unmanned aerial vehicle tracking using a particle filter based approach[C](2019).

    [103] Kaiser C, Sjöberg F, Delcura J M et al. SMART-OLEV: an orbital life extension vehicle for servicing commercial spacecrafts in GEO[J]. Acta Astronautica, 63, 400-410(2008).

    [104] Li S L, Guo M, Du W B et al. Measurement of tank gun muzzle vibration based on image processing with high sub-pixel precision[J]. Journal of Test and Measurement Technology, 31, 131-136(2017).

    [105] Song W, Ai B C, Jiang Z H et al. Prediction and assessment of drop separation compatibility of internal weapons by wind tunnel drop-test[J]. Acta Aeronautica et Astronautica Sinica, 41, 523415(2020).

    [106] Wu R R, Wang J, Wang Y L. Study on high speed penetration of projectile into ice body[J]. Journal of Projectiles, Rockets, Missiles and Guidance, 42, 74-80(2022).

    [107] Liu J B, Wang F, Sun Y L. Experimental study on a large aircraft impacting reinforced concrete nuclear containment structure model[J]. Journal of Building Structures, 43, 185-195(2022).

    [108] Marguet B, Ribere B. Measurement-assisted assembly applications on airbus final assembly lines[J]. SAE Transactions, 112, 372-375(2003).

    [109] Ruel S, Luu T. STS-128 on-orbit demonstration of the TriDAR targetless rendezvous and docking sensor[C](2010).

    [110] Samson C, English C, Deslauriers A et al. The neptec three-dimensional laser camera system: from space mission STS-105 to terrestrial applications[J]. Canadian Aeronautics and Space Journal, 50, 115-123(2004).

    [111] Chen G Y, Cheng Q L, Zhang J Y et al. Multi-sensor measurement based position and pose adjustment method for automatic docking of spacecraft cabins[J]. Journal of Beijing University of Aeronautics and Astronautics, 45, 1232-1239(2019).

    [112] Wang J X, Chen K Y, Jiang L C et al. Research on automatic assembly system of aero engine low pressure turbine shaft based on visual guidance[J]. Machine Design & Research, 37, 94-100, 104(2021).

    [113] Liu S, Xu D, Zhang D P et al. High precision automatic assembly based on microscopic vision and force information[J]. IEEE Transactions on Automation Science and Engineering, 13, 382-393(2016).

    [114] Tamadazte B, Marchand E, Dembélé S et al. CAD model-based tracking and 3D visual-based control for MEMS microassembly[J]. International Journal of Robotics Research, 29, 1416-1434(2010).

    [115] Du F. Automatic assembly fixture and assembly operation control for flexible components[D], 34-44(2021).

    [116] Rogez G, Rihan J, Ramalingam S et al. Randomized trees for human pose detection[C](2008).

    [117] Dai Q, Shi X B, Qiao J Z et al. Articulated human pose estimation with occlusion level[J]. Journal of Computer-Aided Design & Computer Graphics, 29, 279-289(2017).

    [118] Toshev A, Szegedy C. DeepPose: human pose estimation via deep neural networks[C], 1653-1660(2014).

    [119] Wei S H, Ramakrishna V, Kanade T et al. Convolutional pose machines[C], 4724-4732(2016).

    [120] Chen Y L, Wang Z C, Peng Y X et al. Cascaded pyramid network for multi-person pose estimation[C], 7103-7112(2018).

    [121] Cheng B W, Xiao B, Wang J D et al. HigherHRNet: scale-aware representation learning for bottom-up human pose estimation[C], 5385-5394(2020).

    [122] Pavlakos G, Zhou X W, Derpanis K G et al. Coarse-to-fine volumetric prediction for single-image 3D human pose[C], 1263-1272(2017).

    [123] Liu M Y, Yuan J S. Recognizing human actions as the evolution of pose estimation maps[C], 1159-1168(2018).

    [124] Luvizon D C, Picard D, Tabia H. 2D/3D pose estimation and action recognition using multitask deep learning[C], 5137-5146(2018).

    [125] Yan A, Wang Y L, Li Z F et al. PA3D: pose-action 3D machine for video recognition[C], 7914-7923(2020).

    [126] Wang M C, Tighe J, Modolo D. Combining detection and tracking for human pose estimation in videos[C], 11085-11093(2020).

    [127] Zhang Y F, Wang C Y, Wang X G et al. VoxelTrack: multi-person 3D human pose estimation and tracking in the wild[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45, 2613-2626(2023).

    [128] Tang Y Z. Research on human pose estimation and action recognition in complex scenes[D], 21-44(2021).

    [129] Johnson S, Everingham M. Clustered pose and nonlinear appearance models for human pose estimation[C], 1-11(2010).

    [132] Xia P. Pedestrian posture estimation for smart car active safety[D], 40-72(2019).

    [133] Zhao Y, Yuan Z J, Chen B D. Accurate pedestrian detection by human pose regression[J]. IEEE Transactions on Image Processing, 29, 1591-1605(2019).

    [134] Liu N. Human pose recognition and pilot posture tracking[D], 27-49(2020).

    [135] Wu Q Q, Xu G H, Zhang S C et al. Human 3D pose estimation in a lying position by RGB-D images for medical diagnosis and rehabilitation[C], 5802-5805(2020).

    [136] Cao T, Armin M A, Denman S et al. In-bed human pose estimation from unseen and privacy-preserving image domains[C](2022).

    [137] Rohan A, Rabah M, Hosny T et al. Human pose estimation-based real-time gait analysis using convolutional neural network[J]. IEEE Access, 8, 191542-191550(2020).

    Zhenzhong Wei, Guangkun Feng, Danya Zhou, Yueming Ma, Mingkun Liu, Qifeng Luo, Tengda Huang. A Review of Position and Orientation Visual Measurement Methods and Applications[J]. Laser & Optoelectronics Progress, 2023, 60(3): 0312010
    Download Citation