• Photonics Research
  • Vol. 9, Issue 5, B210 (2021)
Shuo Zhu1、†, Enlai Guo1、2、†,*, Jie Gu1, Lianfa Bai1, and Jing Han1、3、*
Author Affiliations
  • 1Jiangsu Key Laboratory of Spectral Imaging and Intelligent Sense, Nanjing University of Science and Technology, Nanjing 210094, China
  • 2e-mail: njustgel@njust.edu.cn
  • 3e-mail: eohj@njust.edu.cn
  • show less
    DOI: 10.1364/PRJ.416551 Cite this Article Set citation alerts
    Shuo Zhu, Enlai Guo, Jie Gu, Lianfa Bai, Jing Han. Imaging through unknown scattering media based on physics-informed learning[J]. Photonics Research, 2021, 9(5): B210 Copy Citation Text show less
    References

    [1] J. W. Goodman. Speckle Phenomena in Optics: Theory and Applications(2007).

    [2] M. C. Roggemann, B. M. Welsh, B. R. Hunt. Imaging Through Turbulence(1996).

    [3] R. K. Tyson. Principles of Adaptive Optics(2015).

    [4] E. J. McCartney. Optics of the Atmosphere: Scattering by Molecules and Particles(1976).

    [5] V. Ntziachristos. Going deeper than microscopy: the optical imaging frontier in biology. Nat. Methods, 7, 603-614(2010).

    [6] S. Yoon, M. Kim, M. Jang, Y. Choi, W. Choi, S. Kang, W. Choi. Deep optical imaging within complex scattering media. Nat. Rev. Phys., 2, 141-158(2020).

    [7] L. V. Wang, H.-I. Wu. Biomedical Optics: Principles and Imaging(2012).

    [8] A. P. Mosk, A. Lagendijk, G. Lerosey, M. Fink. Controlling waves in space and time for imaging and focusing in complex media. Nat. Photonics, 6, 283-292(2012).

    [9] I. M. Vellekoop, A. Mosk. Focusing coherent light through opaque strongly scattering media. Opt. Lett., 32, 2309-2311(2007).

    [10] S. Rotter, S. Gigan. Light fields in complex media: mesoscopic scattering meets wave control. Rev. Mod. Phys., 89, 015005(2017).

    [11] K. Wang, W. Sun, C. T. Richie, B. K. Harvey, E. Betzig, N. Ji. Direct wavefront sensing for high-resolution in vivo imaging in scattering tissue. Nat Commun, 6, 7276(2015).

    [12] S. Popoff, G. Lerosey, R. Carminati, M. Fink, A. Boccara, S. Gigan. Measuring the transmission matrix in optics: an approach to the study and control of light propagation in disordered media. Phys. Rev. Lett., 104, 100601(2010).

    [13] A. Drémeau, A. Liutkus, D. Martina, O. Katz, C. Schülke, F. Krzakala, S. Gigan, L. Daudet. Reference-less measurement of the transmission matrix of a highly scattering material using a DMD and phase retrieval techniques. Opt. Express, 23, 11898-11911(2015).

    [14] E. Tajahuerce, V. Durán, P. Clemente, E. Irles, F. Soldevila, P. Andrés, J. Lancis. Image transmission through dynamic scattering media by single-pixel photodetection. Opt. Express, 22, 16945-16955(2014).

    [15] Y.-K. Xu, W.-T. Liu, E.-F. Zhang, Q. Li, H.-Y. Dai, P.-X. Chen. Is ghost imaging intrinsically more powerful against scattering?. Opt. Express, 23, 32993-33000(2015).

    [16] Q. Fu, Y. Bai, X. Huang, S. Nan, P. Xie, X. Fu. Positive influence of the scattering medium on reflective ghost imaging. Photon. Res., 7, 1468-1472(2019).

    [17] D. Lu, M. Liao, W. He, Z. Cai, X. Peng. Imaging dynamic objects hidden behind scattering medium by retrieving the point spread function. Proc. SPIE, 10834, 1083428(2018).

    [18] H. He, X. Xie, Y. Liu, H. Liang, J. Zhou. Exploiting the point spread function for optical imaging through a scattering medium based on deconvolution method. J. Innov. Opt. Health Sci., 12, 1930005(2019).

    [19] X. Xu, X. Xie, A. Thendiyammal, H. Zhuang, J. Xie, Y. Liu, J. Zhou, A. P. Mosk. Imaging of objects through a thin scattering layer using a spectrally and spatially separated reference. Opt. Express, 26, 15073-15083(2018).

    [20] J. Bertolotti, E. G. Van Putten, C. Blum, A. Lagendijk, W. L. Vos, A. P. Mosk. Non-invasive imaging through opaque scattering layers. Nature, 491, 232-234(2012).

    [21] O. Katz, P. Heidmann, M. Fink, S. Gigan. Non-invasive single-shot imaging through scattering layers and around corners via speckle correlations. Nat. Photonics, 8, 784-790(2014).

    [22] A. Porat, E. R. Andresen, H. Rigneault, D. Oron, S. Gigan, O. Katz. Widefield lensless imaging through a fiber bundle via speckle correlations. Opt. Express, 24, 16835-16855(2016).

    [23] J. R. Fienup. Phase retrieval algorithms: a comparison. Appl. Opt., 21, 2758-2769(1982).

    [24] J. Chang, G. Wetzstein. Single-shot speckle correlation fluorescence microscopy in thick scattering tissue with image reconstruction priors. J. Biophoton., 11, e201700224(2018).

    [25] P. Schniter, S. Rangan. Compressive phase retrieval via generalized approximate message passing. IEEE Trans. Signal Process., 63, 1043-1055(2014).

    [26] Y. LeCun, Y. Bengio, G. Hinton. Deep learning. Nature, 521, 436-444(2015).

    [27] I. Goodfellow, Y. Bengio, A. Courville, Y. Bengio. Deep Learning, 1(2016).

    [28] G. Barbastathis, A. Ozcan, G. Situ. On the use of deep learning for computational imaging. Optica, 6, 921-943(2019).

    [29] Y. Rivenson, Y. Zhang, H. Günaydın, D. Teng, A. Ozcan. Phase recovery and holographic image reconstruction using deep learning in neural networks. Light Sci. Appl., 7, 17141(2018).

    [30] Y. Wu, Y. Rivenson, Y. Zhang, Z. Wei, H. Günaydin, X. Lin, A. Ozcan. Extended depth-of-field in holographic imaging using deep-learning-based autofocusing and phase recovery. Optica, 5, 704-710(2018).

    [31] Z. Ren, Z. Xu, E. Y. Lam. Learning-based nonparametric autofocusing for digital holography. Optica, 5, 337-344(2018).

    [32] Z. Ren, Z. Xu, E. Y. Lam. End-to-end deep learning framework for digital holographic reconstruction. Adv. Photon., 1, 016004(2019).

    [33] T. Nguyen, Y. Xue, Y. Li, L. Tian, G. Nehmetallah. Deep learning approach for Fourier ptychography microscopy. Opt. Express, 26, 26470-26484(2018).

    [34] A. Kappeler, S. Ghosh, J. Holloway, O. Cossairt, A. Katsaggelos. Ptychnet: CNN based Fourier ptychography. IEEE International Conference on Image Processing (ICIP), 1712-1716(2017).

    [35] S. Jiang, K. Guo, J. Liao, G. Zheng. Solving Fourier ptychographic imaging problems via neural network modeling and TensorFlow. Biomed. Opt. Express, 9, 3306-3319(2018).

    [36] Y. F. Cheng, M. Strachan, Z. Weiss, M. Deb, D. Carone, V. Ganapati. Illumination pattern design with deep learning for single-shot Fourier ptychographic microscopy. Opt. Express, 27, 644-656(2019).

    [37] M. Lyu, W. Wang, H. Wang, H. Wang, G. Li, N. Chen, G. Situ. Deep-learning-based ghost imaging. Sci. Rep., 7, 17865(2017).

    [38] Y. He, G. Wang, G. Dong, S. Zhu, H. Chen, A. Zhang, Z. Xu. Ghost imaging based on deep learning. Sci. Rep., 8, 6469(2018).

    [39] H. Wang, Y. Rivenson, Y. Jin, Z. Wei, R. Gao, H. Günaydın, L. A. Bentolila, C. Kural, A. Ozcan. Deep learning enables cross-modality super-resolution in fluorescence microscopy. Nat. Methods, 16, 103-110(2019).

    [40] E. Nehme, L. E. Weiss, T. Michaeli, Y. Shechtman. Deep-storm: super-resolution single-molecule microscopy by deep learning. Optica, 5, 458-464(2018).

    [41] W. Ouyang, A. Aristov, M. Lelek, X. Hao, C. Zimmer. Deep learning massively accelerates super-resolution localization microscopy. Nat. Biotechnol., 36, 460-468(2018).

    [42] C. Ling, C. Zhang, M. Wang, F. Meng, L. Du, X. Yuan. Fast structured illumination microscopy via deep learning. Photon. Res., 8, 1350-1359(2020).

    [43] L. Fang, D. Cunefare, C. Wang, R. H. Guymer, S. Li, S. Farsiu. Automatic segmentation of nine retinal layer boundaries in OCT images of non-exudative AMD patients using deep learning and graph search. Biomed. Opt. Express, 8, 2732-2744(2017).

    [44] L. Waller, L. Tian. Computational imaging: Machine learning for 3D microscopy. Nature, 523, 416-417(2015).

    [45] T. C. Nguyen, V. Bui, G. Nehmetallah. Computational optical tomography using 3-D deep convolutional neural networks. Opt. Eng., 57, 041406(2018).

    [46] A. Goy, K. Arthur, S. Li, G. Barbastathis. Low photon count phase retrieval using deep learning. Phys. Rev. Lett., 121, 243902(2018).

    [47] C. Chen, Q. Chen, J. Xu, V. Koltun. Learning to see in the dark. IEEE Conference on Computer Vision and Pattern Recognition, 3291-3300(2018).

    [48] S. Feng, Q. Chen, G. Gu, T. Tao, L. Zhang, Y. Hu, W. Yin, C. Zuo. Fringe pattern analysis using deep learning. Adv. Photon., 1, 025001(2019).

    [49] K. Wang, Y. Li, Q. Kemao, J. Di, J. Zhao. One-step robust deep learning phase unwrapping. Opt. Express, 27, 15100-15115(2019).

    [50] H. Yu, X. Chen, Z. Zhang, C. Zuo, Y. Zhang, D. Zheng, J. Han. Dynamic 3-D measurement based on fringe-to-fringe transformation using deep learning. Opt. Express, 28, 9405-9418(2020).

    [51] H. Yu, D. Zheng, J. Fu, Y. Zhang, C. Zuo, J. Han. Deep learning-based fringe modulation-enhancing method for accurate fringe projection profilometry. Opt. Express, 28, 21692-21703(2020).

    [52] S. Li, M. Deng, J. Lee, A. Sinha, G. Barbastathis. Imaging through glass diffusers using densely connected convolutional networks. Optica, 5, 803-813(2018).

    [53] M. Lyu, H. Wang, G. Li, S. Zheng, G. Situ. Learning-based lensless imaging through optically thick scattering media. Adv. Photon., 1, 036002(2019).

    [54] N. Borhani, E. Kakkava, C. Moser, D. Psaltis. Learning to see through multimode fibers. Optica, 5, 960-966(2018).

    [55] E. Guo, S. Zhu, Y. Sun, L. Bai, C. Zuo, J. Han. Learning-based method to reconstruct complex targets through scattering medium beyond the memory effect. Opt. Express, 28, 2433-2446(2020).

    [56] E. Guo, Y. Sun, S. Zhu, D. Zheng, C. Zuo, L. Bai, J. Han. Single-shot color object reconstruction through scattering medium based on neural network. Opt. Lasers Eng., 136, 106310(2020).

    [57] Y. Li, Y. Xue, L. Tian. Deep speckle correlation: a deep learning approach toward scalable imaging through scattering media. Optica, 5, 1181-1190(2018).

    [58] Y. Sun, J. Shi, L. Sun, J. Fan, G. Zeng. Image reconstruction through dynamic scattering media based on deep learning. Opt. Express, 27, 16032-16046(2019).

    [59] M. Liao, S. Zheng, D. Lu, G. Situ, X. Peng. Real-time imaging through moving scattering layers via a two-step deep learning strategy. Proc. SPIE, 11351, 113510V(2020).

    [60] Y. Li, S. Cheng, Y. Xue, L. Tian. Displacement-agnostic coherent imaging through scatter with an interpretable deep neural network. Opt. Express, 29, 2244-2257(2020).

    [61] K. Goda, B. Jalali, C. Lei, G. Situ, P. Westbrook. AI boosts photonics and vice versa. APL Photon., 5, 070401(2020).

    [62] S. Feng, C. Kane, P. A. Lee, A. D. Stone. Correlations and fluctuations of coherent wave transmission through disordered media. Phys. Rev. Lett., 61, 834-837(1988).

    [63] I. Freund, M. Rosenbluh, S. Feng. Memory effects in propagation of optical waves through disordered media. Phys. Rev. Lett., 61, 2328-2331(1988).

    [64] H. Liu, Z. Liu, M. Chen, S. Han, L. V. Wang. Physical picture of the optical memory effect. Photon. Res., 7, 1323-1330(2019).

    [65] O. Ronneberger, P. Fischer, T. Brox. U-Net: convolutional networks for biomedical image segmentation. International Conference on Medical Image Computing and Computer-Assisted Intervention, 234-241(2015).

    [66] Y. LeCun, C. Cortes, C. J. C. Burges. The MNIST database of handwritten digits.

    [67] C. E. Thomaz. FEI face database.

    [68] C. Guo, J. Liu, W. Li, T. Wu, L. Zhu, J. Wang, G. Wang, X. Shao. Imaging through scattering layers exceeding memory effect range by exploiting prior information. Opt. Commun., 434, 203-208(2019).

    [69] D. Tang, S. K. Sahoo, V. Tran, C. Dang. Single-shot large field of view imaging with scattering media by spatial demultiplexing. Appl. Opt., 57, 7533-7538(2018).

    CLP Journals

    [1] Li Gao, Yang Chai, Darko Zibar, Zongfu Yu. Deep learning in photonics: introduction[J]. Photonics Research, 2021, 9(8): DLP1

    Shuo Zhu, Enlai Guo, Jie Gu, Lianfa Bai, Jing Han. Imaging through unknown scattering media based on physics-informed learning[J]. Photonics Research, 2021, 9(5): B210
    Download Citation