Generalized framework for non-sinusoidal fringe analysis using deep learning

Shijie Feng; Chao Zuo; Liang Zhang; Wei Yin; Qian Chen

doi:10.1364/PRJ.420944

[1] K. Harding. Industrial metrology: engineering precision. Nat. Photonics, 2, 667-669(2008).

[2] J. M. Schmitt. Optical coherence tomography (OCT): a review. IEEE J. Sel. Top. Quantum Electron., 5, 1205-1215(1999).

[3] J. Han, L. Shao, D. Xu, J. Shotton. Enhanced computer vision with Microsoft Kinect sensor: a review. IEEE Trans. Cybern., 43, 1290-1334(2013).

[4] P. Paysan, R. Knothe, B. Amberg, S. Romdhani, T. Vetter. A 3D face model for pose and illumination invariant face recognition. 6th IEEE International Conference on Advanced Video and Signal Based Surveillance, 296-301(2009).

[5] M. M. P. A. Vermeulen, P. Rosielle, P. Schellekens. Design of a high-precision 3D-coordinate measuring machine. CIRP Ann., 47, 447-450(1998).

[6] F. Chen, G. M. Brown, M. Song. Overview of 3-D shape measurement using optical methods. Opt. Eng., 39, 10-22(2000).

[7] R. Leach. Optical Measurement of Surface Topography, 14(2011).

[8] J. Geng. Structured-light 3D surface imaging: a tutorial. Adv. Opt. Photonics, 3, 128-160(2011).

[9] J. Salvi, J. Pagès, J. Batlle. Pattern codification strategies in structured light systems. Pattern Recogn., 37, 827-849(2004).

[10] C. Zuo, S. Feng, L. Huang, T. Tao, W. Yin, Q. Chen. Phase shifting algorithms for fringe projection profilometry: a review. Opt. Lasers Eng., 109, 23-59(2018).

[11] L. Zhang, B. Curless, S. Seitz. Rapid shape acquisition using color structured light and multi-pass dynamic programming. 1st International Symposium on 3D Data Processing Visualization and Transmission, 24-36(2002).

[12] M. Schaffer, M. Grosse, B. Harendt, R. Kowarschik. High-speed three-dimensional shape measurements of objects with laser speckles and acousto-optical deflection. Opt. Lett., 36, 3097-3099(2011).

[13] S. Heist, P. Lutzke, I. Schmidt, P. Dietrich, P. Kühmstedt, A. Tünnermann, G. Notni. High-speed three-dimensional shape measurement using GOBO projection. Opt. Lasers Eng., 87, 90-96(2016).

[14] M. Takeda, K. Mutoh. Fourier transform profilometry for the automatic measurement of 3-D object shapes. Appl. Opt., 22, 3977-3982(1983).

[15] X. Su, Q. Zhang. Dynamic 3-D shape measurement method: a review. Opt. Lasers Eng., 48, 191-204(2010).

[16] Q. Kemao. Two-dimensional windowed Fourier transform for fringe pattern analysis: principles, applications and implementations. Opt. Lasers Eng., 45, 304-317(2007).

[17] J. Zhong, J. Weng. Spatial carrier-fringe pattern analysis by means of wavelet transform: wavelet transform profilometry. Appl. Opt., 43, 4993-4998(2004).

[18] L. Huang, Q. Kemao, B. Pan, A. K. Asundi. Comparison of Fourier transform, windowed Fourier transform, and wavelet transform methods for phase extraction from a single fringe pattern in fringe projection profilometry. Opt. Lasers Eng., 48, 141-148(2010).

[19] V. Srinivasan, H.-C. Liu, M. Halioua. Automated phase-measuring profilometry of 3-D diffuse objects. Appl. Opt., 23, 3105-3108(1984).

[20] P. S. Huang, Q. J. Hu, F.-P. Chiang. Double three-step phase-shifting algorithm. Appl. Opt., 41, 4503-4509(2002).

[21] P. Hariharan, B. Oreb, T. Eiju. Digital phase-shifting interferometry: a simple error-compensating phase calculation algorithm. Appl. Opt., 26, 2504-2506(1987).

[22] S. Zhang, S.-T. Yau. High-speed three-dimensional shape measurement system using a modified two-plus-one phase-shifting algorithm. Opt. Eng., 46, 113603(2007).

[23] P. Jia, J. Kofman, C. E. English. Two-step triangular-pattern phase-shifting method for three-dimensional object-shape measurement. Opt. Eng., 46, 083201(2007).

[24] P. S. Huang, S. Zhang, F.-P. Chiang. Trapezoidal phase-shifting method for three-dimensional shape measurement. Opt. Eng., 44, 123601(2005).

[25] T. Anna, S. K. Dubey, C. Shakher, A. Roy, D. S. Mehta. Sinusoidal fringe projection system based on compact and non-mechanical scanning low-coherence michelson interferometer for three-dimensional shape measurement. Opt. Commun., 282, 1237-1242(2009).

[26] Y. Guan, Y. Yin, A. Li, X. Liu, X. Peng. Dynamic 3D imaging based on acousto-optic heterodyne fringe interferometry. Opt. Lett., 39, 3678-3681(2014).

[27] S. Yoneyama, Y. Morimoto, M. Fujigaki, M. Yabe. Phase-measuring profilometry of moving object without phase-shifting device. Opt. Lasers Eng., 40, 153-161(2003).

[28] C. Zuo, Q. Chen, G. Gu, S. Feng, F. Feng. High-speed three-dimensional profilometry for multiple objects with complex shapes. Opt. Express, 20, 19493-19510(2012).

[29] S. Ma, C. Quan, R. Zhu, L. Chen, B. Li, C. Tay. A fast and accurate gamma correction based on Fourier spectrum analysis for digital fringe projection profilometry. Opt. Commun., 285, 533-538(2012).

[30] K. Liu, Y. Wang, D. L. Lau, Q. Hao, L. G. Hassebrook. Gamma model and its analysis for phase measuring profilometry. J. Opt. Soc. Am. A, 27, 553-562(2010).

[31] S. Zhang, S.-T. Yau. Generic nonsinusoidal phase error correction for three-dimensional shape measurement using a digital video projector. Appl. Opt., 46, 36-43(2007).

[32] Z. Li, Y. Shi, C. Wang, Y. Wang. Accurate calibration method for a structured light system. Opt. Eng., 47, 053604(2008).

[33] B. Pan, Q. Kemao, L. Huang, A. Asundi. Phase error analysis and compensation for nonsinusoidal waveforms in phase-shifting digital fringe projection profilometry. Opt. Lett., 34, 416-418(2009).

[34] H. Guo, H. He, M. Chen. Gamma correction for digital fringe projection profilometry. Appl. Opt., 43, 2906-2914(2004).

[35] T. Hoang, B. Pan, D. Nguyen, Z. Wang. Generic gamma correction for accuracy enhancement in fringe-projection profilometry. Opt. Lett., 35, 1992-1994(2010).

[36] C. Jiang, S. Xing, H. Guo. Fringe harmonics elimination in multi-frequency phase-shifting fringe projection profilometry. Opt. Express, 28, 2838-2856(2020).

[37] B. Li, Y. Wang, J. Dai, W. Lohry, S. Zhang. Some recent advances on superfast 3D shape measurement with digital binary defocusing techniques. Opt. Lasers Eng., 54, 236-246(2014).

[38] H. Fujita, K. Yamatan, M. Yamamoto, Y. Otani, A. Suguro, S. Morokawa, T. Yoshizawa. Three-dimensional profilometry using liquid crystal grating. Proc. SPIE, 5058, 51-60(2003).

[39] T. Yoshizawa, H. Fujita. Liquid crystal grating for profilmetry using structured light. Proc. SPIE, 6000, 60000H(2005).

[40] G. A. Ayubi, J. A. Ayubi, J. M. Di Martino, J. A. Ferrari. Pulse-width modulation in defocused three-dimensional fringe projection. Opt. Lett., 35, 3682-3684(2010).

[41] C. Zuo, Q. Chen, S. Feng, F. Feng, G. Gu, X. Sui. Optimized pulse width modulation pattern strategy for three-dimensional profilometry with projector defocusing. Appl. Opt., 51, 4477-4490(2012).

[42] Y. Wang, S. Zhang. Optimal pulse width modulation for sinusoidal fringe generation with projector defocusing. Opt. Lett., 35, 4121-4123(2010).

[43] J. Sun, C. Zuo, S. Feng, S. Yu, Y. Zhang, Q. Chen. Improved intensity-optimized dithering technique for 3D shape measurement. Opt. Lasers Eng., 66, 158-164(2015).

[44] W. Lohry, S. Zhang. Genetic method to optimize binary dithering technique for high-quality fringe generation. Opt. Lett., 38, 540-542(2013).

[45] S. Feng, L. Zhang, C. Zuo, T. Tao, Q. Chen, G. Gu. High dynamic range 3D measurements with fringe projection profilometry: a review. Meas. Sci. Technol., 29, 122001(2018).

[46] S. Feng, Y. Zhang, Q. Chen, C. Zuo, R. Li, G. Shen. General solution for high dynamic range three-dimensional shape measurement using the fringe projection technique. Opt. Lasers Eng., 59, 56-71(2014).

[47] S. Zhang, S.-T. Yau. High dynamic range scanning technique. Opt. Eng., 48, 033604(2009).

[48] Z. Song, H. Jiang, H. Lin, S. Tang. A high dynamic range structured light means for the 3D measurement of specular surface. Opt. Lasers Eng., 95, 8-16(2017).

[49] S. Feng, Q. Chen, C. Zuo, A. Asundi. Fast three-dimensional measurements for dynamic scenes with shiny surfaces. Opt. Commun., 382, 18-27(2017).

[50] C. Waddington, J. Kofman. Saturation avoidance by adaptive fringe projection in phase-shifting 3D surface-shape measurement. International Symposium on Optomechatronic Technologies, 1-4(2010).

[51] L. Zhang, Q. Chen, C. Zuo, S. Feng. High dynamic range 3D shape measurement based on the intensity response function of a camera. Appl. Opt., 57, 1378-1386(2018).

[52] H. Lin, J. Gao, Q. Mei, Y. He, J. Liu, X. Wang. Adaptive digital fringe projection technique for high dynamic range three-dimensional shape measurement. Opt. Express, 24, 7703-7718(2016).

[53] Z. Cai, X. Liu, X. Peng, Y. Yin, A. Li, J. Wu, B. Z. Gao. Structured light field 3D imaging. Opt. Express, 24, 20324-20334(2016).

[54] V. Suresh, Y. Wang, B. Li. High-dynamic-range 3D shape measurement utilizing the transitioning state of digital micromirror device. Opt. Lasers Eng., 107, 176-181(2018).

[55] H. Jiang, H. Zhao, X. Li. High dynamic range fringe acquisition: a novel 3-D scanning technique for high-reflective surfaces. Opt. Lasers Eng., 50, 1484-1493(2012).

[56] L. Zhang, Q. Chen, C. Zuo, S. Feng. Real-time high dynamic range 3D measurement using fringe projection. Opt. Express, 28, 24363-24378(2020).

[57] L. Zhang, Q. Chen, C. Zuo, S. Feng. High-speed high dynamic range 3D shape measurement based on deep learning. Opt. Lasers Eng., 134, 106245(2020).

[58] J. H. Bruning, D. R. Herriott, J. Gallagher, D. Rosenfeld, A. White, D. Brangaccio. Digital wavefront measuring interferometer for testing optical surfaces and lenses. Appl. Opt., 13, 2693-2703(1974).

[59] Y. Yin, Z. Cai, H. Jiang, X. Meng, J. Xi, X. Peng. High dynamic range imaging for fringe projection profilometry with single-shot raw data of the color camera. Opt. Lasers Eng., 89, 138-144(2017).

[60] M. Wang, G. Du, C. Zhou, C. Zhang, S. Si, H. Li, Z. Lei, Y. Li. Enhanced high dynamic range 3D shape measurement based on generalized phase-shifting algorithm. Opt. Commun., 385, 43-53(2017).

[61] Y. Chen, Y. He, E. Hu. Phase deviation analysis and phase retrieval for partial intensity saturation in phase-shifting projected fringe profilometry. Opt. Commun., 281, 3087-3090(2008).

[62] Z. Qi, Z. Wang, J. Huang, C. Xing, J. Gao. Error of image saturation in the structured-light method. Appl. Opt., 57, A181-A188(2018).

[63] P. Vincent, H. Larochelle, I. Lajoie, Y. Bengio, P.-A. Manzagol, L. Bottou. Stacked denoising autoencoders: learning useful representations in a deep network with a local denoising criterion. J. Mach. Learn. Res., 11, 3371-3408(2010).

[64] D. J. Im, S. Ahn, R. Memisevic, Y. Bengio. Denoising criterion for variational auto-encoding framework. AAAI Conference on Artificial Intelligence, 2059-2062(2017).

[65] Y. Kiarashinejad, M. Zandehshahvar, S. Abdollahramezani, O. Hemmatyar, R. Pourabolghasem, A. Adibi. Knowledge discovery in nanophotonics using geometric deep learning. Adv. Intell. Syst., 2, 1900132(2020).

[66] Y. Rivenson, Z. Göröcs, H. Günaydin, Y. Zhang, H. Wang, A. Ozcan. Deep learning microscopy. Optica, 4, 1437-1443(2017).

[67] Y. Rivenson, Y. Zhang, H. Günaydn, D. Teng, A. Ozcan. Phase recovery and holographic image reconstruction using deep learning in neural networks. Light Sci. Appl., 7, 17141(2018).

[68] M. Lyu, W. Wang, H. Wang, H. Wang, G. Li, N. Chen, G. Situ. Deep-learning-based ghost imaging. Sci. Rep., 7, 17865(2017).

[69] Y. Li, Y. Xue, L. Tian. Deep speckle correlation: a deep learning approach toward scalable imaging through scattering media. Optica, 5, 1181-1190(2018).

[70] C. S. Lee, A. J. Tyring, N. P. Deruyter, Y. Wu, A. Rokem, A. Y. Lee. Deep-learning based, automated segmentation of macular edema in optical coherence tomography. Biomed. Opt. Express, 8, 3440-3448(2017).

[71] S. Feng, Q. Chen, G. Gu, T. Tao, L. Zhang, Y. Hu, W. Yin, C. Zuo. Fringe pattern analysis using deep learning. Adv. Photonics, 1, 025001(2019).

[72] J. Qian, S. Feng, Y. Li, T. Tao, J. Han, Q. Chen, C. Zuo. Single-shot absolute 3D shape measurement with deep-learning-based color fringe projection profilometry. Opt. Lett., 45, 1842-1845(2020).

[73] J. Shi, X. Zhu, H. Wang, L. Song, Q. Guo. Label enhanced and patch based deep learning for phase retrieval from single frame fringe pattern in fringe projection 3D measurement. Opt. Express, 27, 28929-28943(2019).

[74] T. Yang, Z. Zhang, H. Li, X. Li, X. Zhou. Single-shot phase extraction for fringe projection profilometry using deep convolutional generative adversarial network. Meas. Sci. Technol., 32, 015007(2020).

[75] W. Yin, Q. Chen, S. Feng, T. Tao, L. Huang, M. Trusiak, A. Asundi, C. Zuo. Temporal phase unwrapping using deep learning. Sci. Rep., 9, 20175(2019).

[76] J. Qian, S. Feng, T. Tao, Y. Hu, Y. Li, Q. Chen, C. Zuo. Deep-learning-enabled geometric constraints and phase unwrapping for single-shot absolute 3D shape measurement. APL Photonics, 5, 046105(2020).

[77] O. Ronneberger, P. Fischer, T. Brox. U-net: convolutional networks for biomedical image segmentation. International Conference on Medical Image Computing and Computer-Assisted Intervention, 234-241(2015).

[78] C. Zuo, T. Tao, S. Feng, L. Huang, A. Asundi, Q. Chen. Micro Fourier transform profilometry (μftp): 3D shape measurement at 10,000 frames per second. Opt. Lasers Eng., 102, 70-91(2018).

CLP Journals