Videometrics Methods and Applications of Static and Dynamic Deformation of Large-Scale Structures (Invited)

Qifeng Yu; Qiang Zhang; Wenjun Chen; Yihe Yin; Mingjie Chen; Yu Lei; Lihao Liu; Xiaolin Liu; Yueqiang Zhang; Biao Hu; Xiaohua Ding

doi:10.3788/LOP240449

[1] K V S , Roy B K. Online implementation of an adaptive calibration technique for displacement measurement using LVDT[J]. Applied Soft Computing, 53, 19-26(2017).

[2] Gomez F, Park J W, Spencer B F,. Reference-free structural dynamic displacement estimation method[J]. Structural Control and Health Monitoring, 25, e2209(2018).

[3] Park J W, Sim S H, Jung H J. Displacement estimation using multimetric data fusion[J]. IEEE/ASME Transactions on Mechatronics, 18, 1675-1682(2013).

[4] Zhang Q, Fu X, Ren L. Deflection estimation of beam structures based on the measured strain mode shape[J]. Smart Materials and Structures, 30, 105003(2021).

[5] Sigurdardottir D H, Stearns J, Glisic B. Error in the determination of the deformed shape of prismatic beams using the double integration of curvature[J]. Smart Materials and Structures, 26, 075002(2017).

[6] Zhang W, Sun L M, Sun S W. Bridge-deflection estimation through inclinometer data considering structural damages[J]. Journal of Bridge Engineering, 22, 04016117(2017).

[7] Zhou J Y, Sun Z, Wei B et al. Deflection-based multilevel structural condition assessment of long-span prestressed concrete girder bridges using a connected pipe system[J]. Measurement, 169, 108352(2021).

[8] Bonopera M, Chang K C, Chen C C et al. Fiber Bragg grating–differential settlement measurement system for bridge displacement monitoring: case study[J]. Journal of Bridge Engineering, 24, 05019011(2019).

[9] Lee Z K, Bonopera M, Hsu C C et al. Long-term deflection monitoring of a box girder bridge with an optical-fiber, liquid-level system[J]. Structures, 44, 904-919(2022).

[10] Xiong C B, Wang M, Chen W. Data analysis and dynamic characteristic investigation of large-scale civil structures monitored by RTK-GNSS based on a hybrid filtering algorithm[J]. Journal of Civil Structural Health Monitoring, 12, 857-874(2022).

[11] Nassif H H, Gindy M, Davis J. Comparison of laser Doppler vibrometer with contact sensors for monitoring bridge deflection and vibration[J]. NDT & E International, 38, 213-218(2005).

[12] Reu P L, Rohe D P, Jacobs L D. Comparison of DIC and LDV for practical vibration and modal measurements[J]. Mechanical Systems and Signal Processing, 86, 2-16(2017).

[13] Zhou J G, Xiao H L, Jiang W W et al. Automatic subway tunnel displacement monitoring using robotic total station[J]. Measurement, 151, 107251(2020).

[14] Lee J, Kim R E. Noncontact dynamic displacements measurements for structural identification using a multi-channel Lidar[J]. Structural Control and Health Monitoring, 29, e3100(2022).

[15] Zhang G W, Wu Y L, Zhao W J et al. Radar-based multipoint displacement measurements of a 1200-m-long suspension bridge[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 167, 71-84(2020).

[16] Rodrigues D V Q, Zuo D L, Li C Z. Wind-induced displacement analysis for a traffic light structure based on a low-cost Doppler radar array[J]. IEEE Transactions on Instrumentation and Measurement, 70, 6503909(1969).

[17] Yu Q F, Guan B L, Shang Y et al. Flexible camera series network for deformation measurement of large scale structures[J]. Smart Structures and Systems, 24, 587(2019).

[18] Song Q S, Wu J R, Wang H L et al. Computer vision-based illumination-robust and multi-point simultaneous structural displacement measuring method[J]. Mechanical Systems and Signal Processing, 170, 108822(2022).

[19] Shang Y, Yu Q F, Yang Z et al. Displacement and deformation measurement for large structures by camera network[J]. Optics and Lasers in Engineering, 54, 247-254(2014).

[20] Yu Q F, Yin Y H, Zhang Y Q et al. Displacement measurement of large structures using nonoverlapping field of view multi-camera systems under six degrees of freedom ego-motion[J]. Computer-Aided Civil and Infrastructure Engineering, 38, 1483-1503(2023).

[21] Hu B, Chen W J, Zhang Y Q et al. Vision-based multi-point real-time monitoring of dynamic displacement of large-span cable-stayed bridges[J]. Mechanical Systems and Signal Processing, 204, 110790(2023).

[22] Yin Y H, Yu Q F, Zhang Y Q et al. Deflection monitoring of immersed tunnel element during floating transportation and installation based on series camera network[J]. Optics and Lasers in Engineering, 172, 107857(2024).

[23] Wang M M, Xu F Y, Xu Y et al. A robust subpixel refinement technique using self-adaptive edge points matching for vision-based structural displacement measurement[J]. Computer-Aided Civil and Infrastructure Engineering, 38, 562-579(2023).

[24] Yin Y H, Yu Q F, Hu B et al. A vision monitoring system for multipoint deflection of large-span bridge based on camera networking[J]. Computer-Aided Civil and Infrastructure Engineering, 38, 1879-1891(2023).

[25] Yu Q, Shang Y[M]. Videometrics: principles and researches(2009).

[26] Hu S T, Sun H, Wang M C et al. Multi-viewpoint and high-precision measurement method for spatial structure deformation[J]. Laser & Optoelectronics Progress, 59, 1912004(2022).

[27] Zhou W, Du X Y, Xue X T et al. Stereo vision-based measurement system for geometric parameters of high-speed railway catenary[J]. Acta Optica Sinica, 43, 0512001(2023).

[28] Zhuang Y Z, Chen W M, Jin T et al. A review of computer vision-based structural deformation monitoring in field environments[J]. Sensors, 22, 3789(2022).

[29] Peng C, Miao W D, Zeng C. Three-dimensional vibration measurement method for lightweight beam based on machine vision[J]. Journal of Beijing University of Aeronautics and Astronautics, 47, 207-212(2021).

[30] Yang M, Liu W F, Liu Z H et al. Binocular vision-based method used for determining the static and dynamic parameters of the long-stroke shakers in low-frequency vibration calibration[J]. IEEE Transactions on Industrial Electronics, 70, 8537-8545(2023).

[31] Shao Y D, Li L, Li J et al. Computer vision based target-free 3D vibration displacement measurement of structures[J]. Engineering Structures, 246, 113040(2021).

[32] Zuo C L, Ma J, Yue T R et al. Displacement and deformation measurements of helicopter rotor blades based on binocular stereo vision[J]. Journal of Experiments in Fluid Mechanics, 34, 87-95(2020).

[33] Su Z L, Lu L, Yang F J et al. Geometry constrained correlation adjustment for stereo reconstruction in 3D optical deformation measurements[J]. Optics Express, 28, 12219-12232(2020).

[34] Hu Q J, Feng Z Y, He L P et al. Accuracy improvement of binocular vision measurement system for slope deformation monitoring[J]. Sensors, 20, 1994(2020).

[35] Lee J, Lee K C, Jeong S et al. Long-term displacement measurement of full-scale bridges using camera ego-motion compensation[J]. Mechanical Systems and Signal Processing, 140, 106651(2020).

[36] Shao X X, Zhou J F, Yuan F et al. Real-time panoramic-deformation measurement of cabin structure of launch vehicle based on camera network[J]. Scientia Sinica Technologica, 52, 1849-1858(2022).

[37] Shao X X, Yuan F, Wei K et al. Advances in high-accuracy three-dimensional dynamic deformation measurement and its applications for large structures[J]. Laser & Optoelectronics Progress, 60, 0811013(2023).

[38] Zhang Z. A flexible new technique for camera calibration[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22, 1330-1334(2000).

[39] Dong C Z, Catbas F N. A non-target structural displacement measurement method using advanced feature matching strategy[J]. Advances in Structural Engineering, 22, 3461-3472(2019).

[40] Ye X W, Jin T, Ang P P et al. Computer vision-based monitoring of the 3-D structural deformation of an ancient structure induced by shield tunneling construction[J]. Structural Control and Health Monitoring, 28, e2702(2021).

[41] Feng D M, Feng M Q. Experimental validation of cost-effective vision-based structural health monitoring[J]. Mechanical Systems and Signal Processing, 88, 199-211(2017).

[42] Song Y Z, Bowen C R, Kim A H et al. Virtual visual sensors and their application in structural health monitoring[J]. Structural Health Monitoring, 13, 251-264(2014).

[43] Ye X W, Ni Y Q, Wai T T et al. A vision-based system for dynamic displacement measurement of long-span bridges: algorithm and verification[J]. Smart Structures and Systems, 12, 363-379(2013).

[44] Dong C Z, Celik O, Catbas F N. Marker-free monitoring of the grandstand structures and modal identification using computer vision methods[J]. Structural Health Monitoring, 18, 1491-1509(2019).

[45] Xiao Y F, Hu W. High-precision calibration based on multi-camera system[J]. Laser & Optoelectronics Progress, 60, 2015003(2023).

[46] Lu R S, Li Y F. A global calibration method for large-scale multi-sensor visual measurement systems[J]. Sensors and Actuators A: Physical, 116, 384-393(2004).

[47] Lu Y K, Liu W, Zhang Y et al. An error analysis and optimization method for combined measurement with binocular vision[J]. Chinese Journal of Aeronautics, 34, 282-292(2021).

[48] Miyata S, Saito H, Takahashi K et al. Extrinsic camera calibration without visible corresponding points using omnidirectional cameras[J]. IEEE Transactions on Circuits and Systems for Video Technology, 28, 2210-2219(2018).

[49] Takahashi K, Nobuhara S, Matsuyama T. A new mirror-based extrinsic camera calibration using an orthogonality constraint[C], 1051-1058(2012).

[50] Zhao Z L, Zhang Z H, Gao N et al. Calibration of multiple cameras based on ChArUco board[J]. Journal of Applied Optics, 42, 848-852(2021).

[51] Strauß T, Ziegler J, Beck J. Calibrating multiple cameras with non-overlapping views using coded checkerboard targets[C], 2623-2628(2014).

[52] Ghosal S, Mehrotra R. Orthogonal moment operators for subpixel edge detection[J]. Pattern Recognition, 26, 295-306(1993).

[53] Sobel I. Neighborhood coding of binary images for fast contour following and general binary array processing[J]. Computer Graphics and Image Processing, 8, 127-135(1978).

[54] Canny J. A computational approach to edge detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 8, 679-698(1986).

[55] Lou Q, Lü J H, Wen L H et al. High-precision camera calibration method based on sub-pixel edge detection[J]. Acta Optica Sinica, 42, 2012002(2022).

[56] Ballard D H. Generalizing the Hough transform to detect arbitrary shapes[J]. Pattern Recognition, 13, 111-122(1981).

[57] Li Y S, Shi W, Liu A L. A Harris corner detection algorithm for multispectral images based on the correlation[C], 161-165(2015).

[58] Bansal M, Kumar M, Kumar M et al. An efficient technique for object recognition using Shi-Tomasi corner detection algorithm[J]. Soft Computing, 25, 4423-4432(2021).

[59] Križaj J, Štruc V, Pavešić N. Adaptation of SIFT features for robust face recognition[M]. Campilho A, Kamel M. Image analysis and recognition. Lecture notes in computer science, 6111, 394-404(2010).

[60] You Z, Luan Z, Wei X. Performance analysis of SURF descriptor with different local region partitions[J]. Optics and Precision Engineering, 21, 2395-2403(2013).

[61] Mohammad S, Morris T. Binary robust independent elementary feature features for texture segmentation[J]. Advanced Science Letters, 23, 5178-5182(2017).

[62] Yang S Q, Li B, Zeng K. SBRISK: speed-up binary robust invariant scalable keypoints[J]. Journal of Real-Time Image Processing, 12, 583-591(2016).

[63] Alahi A, Ortiz R, Vandergheynst P. FREAK: fast retina keypoint[C], 510-517(2012).

[64] Feng D M, Feng M Q. Computer vision for SHM of civil infrastructure: from dynamic response measurement to damage detection-a review[J]. Engineering Structures, 156, 105-117(2018).

[65] Pan B, Qian K M, Xie H M et al. Two-dimensional digital image correlation for in-plane displacement and strain measurement: a review[J]. Measurement Science and Technology, 20, 062001(2009).

[66] Li S H, Gao X, Liu Z W et al. Algorithm for sub-pixel detection of fringe image displacement based on gray-level interpolation[J]. Acta Optica Sinica, 41, 1012002(2021).

[67] Ma S H, Guo P K, You H R et al. An image matching optimization algorithm based on pixel shift clustering RANSAC[J]. Information Sciences, 562, 452-474(2021).

[68] Trimeche M, Tico M, Gabbouj M. Dense optical flow field estimation using recursive LMS filtering[C](2006).

[69] Khuc T, Catbas F N. Computer vision-based displacement and vibration monitoring without using physical target on structures[J]. Structure and Infrastructure Engineering, 13, 505-516(2017).

[70] Hu Q J, He S S, Wang S L et al. A high-speed target-free vision-based sensor for bus rapid transit viaduct vibration measurements using CMT and ORB algorithms[J]. Sensors, 17, 1305(2017).

[71] Horn B K P, Schunck B G. Determining optical flow[J]. Artificial Intelligence, 17, 185-203(1981).

[72] Liu B, Zaccarin A. New fast algorithms for the estimation of block motion vectors[J]. IEEE Transactions on Circuits and Systems for Video Technology, 3, 148-157(1993).

[73] Farnebäck G. Two-frame motion estimation based on polynomial expansion[M]. Bigun J, Gustavsson T. Image analysis. Lecture notes in computer science, 2749, 363-370(2003).

[74] Collier S, Dare T. Informed pixel pushing: a new method of large-motion handling for phase-based optical flow[J]. Measurement, 213, 112711(2023).

[75] Zhang Z, Wang J, Zhao H J et al. Applicability of deep learning optical flow estimation for PIV methods[J]. Flow Measurement and Instrumentation, 93, 102398(2023).

[76] Savian S, Elahi M, Tillo T. Benchmarking the imbalanced behavior of deep learning based optical flow estimators[C], 151-158(2019).

[77] Ren Z, Yan J C, Ni B B et al. Unsupervised deep learning for optical flow estimation[C], 1495-1501(2017).

[78] Guo J, Zhu C A. Dynamic displacement measurement of large-scale structures based on the Lucas-Kanade template tracking algorithm[J]. Mechanical Systems and Signal Processing, 66/67, 425-436(2016).

[79] Dong C Z, Celik O, Catbas F N et al. Structural displacement monitoring using deep learning-based full field optical flow methods[J]. Structure and Infrastructure Engineering, 16, 51-71(2020).

[80] Hoskere V, Park J W, Yoon H et al. Vision-based modal survey of civil infrastructure using unmanned aerial vehicles[J]. Journal of Structural Engineering, 145, 04019062(2019).

[81] Feng W W, Zhang S Q, Liu H B et al. Unmanned aerial vehicle-aided stereo camera calibration for outdoor applications[J]. Optical Engineering, 59, 014110(2020).

[82] Qiu Z C, Wang X F. Vibration measurement and control based on binocular vision[J]. Journal of Vibration, Measurement & Diagnosis, 38, 51-58, 205(2018).

[83] Long X H, Zhan W W, Gui X L. Research on displacement monitoring method of heavy load bridge structure based on stereo vision[C], 432-437(2020).

[84] Liu Y, Ge Z D, Yuan Y T et al. Study of the error caused by camera movement for the stereo-vision system[J]. Applied Sciences, 11, 9384(2021).

[85] Liu Y, Ge Z D, Yuan Y T et al. Wing deformation measurement using the stereo-vision methods in the presence of camera movements[J]. Aerospace Science and Technology, 119, 107161(2021).

[86] Zhang Y Q, Chen M J, Hu B et al. Transmission mechanism and suppression methods of measurement error based on camera networking[J]. Acta Optica Sinica, 43, 2112002(2023).

[87] Yu Q F, Jiang G W, Shang Y et al. A displacement-relay videometric method for surface subsidence surveillance in unstable areas[J]. Science China Technological Sciences, 58, 1105-1111(2015).

[88] Chen W J, Yin Y H, Zhang Y Q et al. Pose correction method for micro-motion stages based on dual-orthogonal-camera[J]. Acta Optica Sinica, 41, 2315001(2021).

[89] Luo L X, Feng M Q, Wu J P. A comprehensive alleviation technique for optical-turbulence-induced errors in vision-based displacement measurement[J]. Structural Control and Health Monitoring, 27, e2496(2020).

[90] Luo L X, Feng M Q, Wu J P et al. Modeling and detection of heat haze in computer vision based displacement measurement[J]. Measurement, 182, 109772(2021).

[91] Feng D M, Feng M Q. Vision-based multipoint displacement measurement for structural health monitoring[J]. Structural Control and Health Monitoring, 23, 876-890(2016).

[92] Jeong J H, Jo H. Real-time generic target tracking for structural displacement monitoring under environmental uncertainties via deep learning[J]. Structural Control and Health Monitoring, 29, e2902(2022).

[93] Cai E J, Zhang Y, Lu X Z et al. A target-free video structural motion estimation method based on multi-path optimization[J]. Mechanical Systems and Signal Processing, 198, 110452(2023).

[94] Lee S, Kim H, Sim S H. Nontarget-based displacement measurement using LiDAR and camera[J]. Automation in Construction, 142, 104493(2022).

[95] Shao Y D, Li L, Li J et al. Target-free 3D tiny structural vibration measurement based on deep learning and motion magnification[J]. Journal of Sound and Vibration, 538, 117244(2022).

[96] Yoon H, Shin J, Spencer B F. Structural displacement measurement using an unmanned aerial system[J]. Computer-Aided Civil and Infrastructure Engineering, 33, 183-192(2018).

[97] Ribeiro D, Santos R, Cabral R et al. Non-contact structural displacement measurement using Unmanned Aerial Vehicles and video-based systems[J]. Mechanical Systems and Signal Processing, 160, 107869(2021).

[98] Zhuge S, Xu X P, Zhong L J et al. Noncontact deflection measurement for bridge through a multi-UAVs system[J]. Computer-Aided Civil and Infrastructure Engineering, 37, 746-761(2022).

[99] Xu Y, Zhang J. UAV-based bridge geometric shape measurement using automatic bridge component detection and distributed multi-view reconstruction[J]. Automation in Construction, 140, 104376(2022).

[100] Yoneyama S, Ueda H. Bridge deflection measurement using digital image correlation with camera movement correction[J]. Materials Transactions, 53, 285-290(2012).