• Photonics Research
  • Vol. 9, Issue 8, B262 (2021)
Yunqi Luo1、†, Suxia Yan1、†, Huanhao Li2、3、†, Puxiang Lai2、3、4、*, and Yuanjin Zheng1、5、*
Author Affiliations
  • 1School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore 639798, Singapore
  • 2Department of Biomedical Engineering, The Hong Kong Polytechnic University, Hong Kong SAR, China
  • 3The Hong Kong Polytechnic University Shenzhen Research Institute, Shenzhen 518034, China
  • 4e-mail: puxiang.lai@polyu.edu.hk
  • 5e-mail: yjzheng@ntu.edu.sg
  • show less
    DOI: 10.1364/PRJ.415590 Cite this Article Set citation alerts
    Yunqi Luo, Suxia Yan, Huanhao Li, Puxiang Lai, Yuanjin Zheng. Towards smart optical focusing: deep learning-empowered dynamic wavefront shaping through nonstationary scattering media[J]. Photonics Research, 2021, 9(8): B262 Copy Citation Text show less
    References

    [1] J. Bertolotti, E. G. Van Putten, C. Blum, A. Lagendijk, W. L. Vos, A. P. Mosk. Non-invasive imaging through opaque scattering layers. Nature, 491, 232-234(2012).

    [2] P. Lai, L. Wang, J. W. Tay, L. V. Wang. Photoacoustically guided wavefront shaping for enhanced optical focusing in scattering media. Nat. Photonics, 9, 126-132(2015).

    [3] Z. Yu, M. Xia, H. Li, T. Zhong, F. Zhao, H. Deng, Z. Li, D. Li, D. Wang, P. Lai. Implementation of digital optical phase conjugation with embedded calibration and phase rectification. Sci. Rep., 9, 1537(2019).

    [4] J. Yang, Y. Shen, Y. Liu, A. S. Hemphill, L. V. Wang. Focusing light through scattering media by polarization modulation based generalized digital optical phase conjugation. Appl. Phys. Lett., 111, 201108(2017).

    [5] J. Yang, J. Li, S. He, L. V. Wang. Angular-spectrum modeling of focusing light inside scattering media by optical phase conjugation. Optica, 6, 250-256(2019).

    [6] Y. Shen, Y. Liu, C. Ma, L. V. Wang. Sub-Nyquist sampling boosts targeted light transport through opaque scattering media. Optica, 4, 97-102(2017).

    [7] I. M. Vellekoop, A. P. Mosk. Focusing coherent light through opaque strongly scattering media. Opt. Lett., 32, 2309-2311(2007).

    [8] S. M. Popoff, G. Lerosey, R. Carminati, M. Fink, A. C. Boccara, S. Gigan. Measuring the transmission matrix in optics: an approach to the study and control of light propagation in disordered media. Phys. Rev. Lett., 104, 100601(2010).

    [9] O. Katz, E. Small, Y. Silberberg. Looking around corners and through thin turbid layers in real time with scattered incoherent light. Nat. Photonics, 6, 549-553(2012).

    [10] H. Yu, T. R. Hillman, W. Choi, J. O. Lee, M. S. Feld, R. R. Dasari, Y. Park. Measuring large optical transmission matrices of disordered media. Phys. Rev. Lett., 111, 153902(2013).

    [11] T. Chaigne, O. Katz, A. C. Boccara, M. Fink, E. Bossy, S. Gigan. Controlling light in scattering media non-invasively using the photoacoustic transmission matrix. Nat. Photonics, 8, 58-64(2014).

    [12] A. Sanjeev, Y. Kapellner, N. Shabairou, E. Gur, M. Sinvani, Z. Zalevsky. Non-invasive imaging through scattering medium by using a reverse response wavefront shaping technique. Sci. Rep., 9, 12275(2019).

    [13] A. Drémeau, A. Liutkus, D. Martina, O. Katz, C. Schülke, F. Krzakala, S. Gigan, L. Daudet. Reference-less measurement of the transmission matrix of a highly scattering material using a DMD and phase retrieval techniques. Opt. Express, 23, 11898-11911(2015).

    [14] K. T. Takasaki, J. W. Fleischer. Phase-space measurement for depth-resolved memory-effect imaging. Opt. Express, 22, 31426-31433(2014).

    [15] O. Katz, E. Small, Y. Guan, Y. Silberberg. Noninvasive nonlinear focusing and imaging through strongly scattering turbid layers. Optica, 1, 170-174(2014).

    [16] E. Edrei, G. Scarcelli. Memory-effect based deconvolution microscopy for super-resolution imaging through scattering media. Sci. Rep., 6, 33558(2016).

    [17] X. Xu, H. Liu, L. V. Wang. Time-reversed ultrasonically encoded optical focusing into scattering media. Nat. Photonics, 5, 154-157(2011).

    [18] B. Judkewitz, Y. M. Wang, R. Horstmeyer, A. Mathy, C. Yang. Speckle-scale focusing in the diffusive regime with time-reversal of variance-encoded light (TROVE). Nat. Photonics, 7, 300-305(2013).

    [19] S. Resisi, Y. Viernik, S. M. Popoff, Y. Bromberg. Wavefront shaping in multimode fibers by transmission matrix engineering. APL Photon., 5, 036103(2020).

    [20] X. Wei, Y. Shen, J. C. Jing, A. S. Hemphill, C. Yang, S. Xu, Z. Yang, L. V. Wang. Real-time frequency-encoded spatiotemporal focusing through scattering media using a programmable 2D ultrafine optical frequency comb. Sci. Adv., 6, eaay1192(2020).

    [21] G. Huang, D. Wu, J. Luo, Y. Huang, Y. Shen. Retrieving the optical transmission matrix of a multimode fiber using the extended Kalman filter. Opt. Express, 28, 9487-9500(2020).

    [22] J. Thompson, B. Hokr, V. Yakovlev. Optimization of focusing through scattering media using the continuous sequential algorithm. J. Mod. Opt., 63, 80-84(2016).

    [23] D. B. Conkey, A. N. Brown, A. M. Caravaca-Aguirre, R. Piestun. Genetic algorithm optimization for focusing through turbid media in noisy environments. Opt. Express, 20, 4840-4849(2012).

    [24] J. Luo, Z. Wu, D. Wu, Z. Liu, X. Wei, Y. Shen, Z. Li. Efficient glare suppression with Hadamard-encoding-algorithm-based wavefront shaping. Opt. Lett., 44, 4067-4070(2019).

    [25] Z. Wu, J. Luo, Y. Feng, X. Guo, Y. Shen, Z. Li. Controlling 1550-nm light through a multimode fiber using a Hadamard encoding algorithm. Opt. Express, 27, 5570-5580(2019).

    [26] O. Katz, P. Heidmann, M. Fink, S. Gigan. Non-invasive single-shot imaging through scattering layers and around corners via speckle correlations. Nat. Photonics, 8, 784-790(2014).

    [27] Y. Liu, C. Ma, Y. Shen, J. Shi, L. V. Wang. Focusing light inside dynamic scattering media with millisecond digital optical phase conjugation. Optica, 4, 280-288(2017).

    [28] D. Wang, E. H. Zhou, J. Brake, H. Ruan, M. Jang, C. Yang. Focusing through dynamic tissue with millisecond digital optical phase conjugation. Optica, 2, 728-735(2015).

    [29] M. Chen, H. Liu, Z. Liu, P. Lai, S. Han. Expansion of the FOV in speckle autocorrelation imaging by spatial filtering. Opt. Lett., 44, 5997-6000(2019).

    [30] J.-H. Park, Z. Yu, K. Lee, P. Lai, Y. Park. Perspective: wavefront shaping techniques for controlling multiple light scattering in biological tissues: toward in vivo applications. APL Photon., 3, 100901(2018).

    [31] Y. Luo, S. Yan, H. Li, P. Lai, Y. Zheng. Focusing light through scattering media by reinforced hybrid algorithms. APL Photon., 5, 016109(2020).

    [32] E. Bossy, S. Gigan. Photoacoustics with coherent light. Photoacoustics, 4, 22-35(2016).

    [33] Z. Li, Z. Yu, H. Hui, H. Li, T. Zhong, H. Liu, P. Lai. Edge enhancement through scattering media enabled by optical wavefront shaping. Photon. Res., 8, 954-962(2020).

    [34] Y. Shen, Y. Liu, C. Ma, L. V. Wang. Focusing light through scattering media by full-polarization digital optical phase conjugation. Opt. Lett., 41, 1130-1133(2016).

    [35] Y.-K. Xu, W.-T. Liu, E.-F. Zhang, Q. Li, H.-Y. Dai, P.-X. Chen. Is ghost imaging intrinsically more powerful against scattering?. Opt. Express, 23, 32993-33000(2015).

    [36] E. Edrei, G. Scarcelli. Optical imaging through dynamic turbid media using the Fourier-domain shower-curtain effect. Optica, 3, 71-74(2016).

    [37] B. Hwang, T. Woo, J.-H. Park. Fast diffraction-limited image recovery through turbulence via subsampled bispectrum analysis. Opt. Lett., 44, 5985-5988(2019).

    [38] B. Blochet, L. Bourdieu, S. Gigan. Focusing light through dynamical samples using fast continuous wavefront optimization. Opt. Lett., 42, 4994-4997(2017).

    [39] R. H. Benjamin Judkewitz, I. M. Vellekoop, I. N. Papadopoulos, Y. Changhuei. Translation correlations in anisotropically scattering media. Nat. Phys., 11, 684-689(2015).

    [40] J. Xie, L. Xu, E. Chen. Image denoising and inpainting with deep neural networks. Advances in Neural Information Processing Systems, 341-349(2012).

    [41] G. Barbastathis, A. Ozcan, G. Situ. On the use of deep learning for computational imaging. Optica, 6, 921-943(2019).

    [42] A. Sinha, J. Lee, S. Li, G. Barbastathis. Lensless computational imaging through deep learning. Optica, 4, 1117-1125(2017).

    [43] L. Waller, L. Tian. Computational imaging: machine learning for 3D microscopy. Nature, 523, 416-417(2015).

    [44] A. Goy, K. Arthur, S. Li, G. Barbastathis. Low photon count phase retrieval using deep learning. Phys. Rev. Lett., 121, 243902(2018).

    [45] Y. Rivenson, Y. Wu, A. Ozcan. Deep learning in holography and coherent imaging. Light Sci. Appl., 8, 1(2019).

    [46] Y. Wu, Y. Rivenson, H. Wang, Y. Luo, E. Ben-David, L. A. Bentolila, C. Pritz, A. Ozcan. Three-dimensional virtual refocusing of fluorescence microscopy images using deep learning. Nat. Methods, 16, 1323-1331(2019).

    [47] M. T. McCann, K. H. Jin, M. Unser. Convolutional neural networks for inverse problems in imaging: a review. IEEE Signal Process. Mag., 34, 85-95(2017).

    [48] C. Dong, C. C. Loy, K. He, X. Tang. Image super-resolution using deep convolutional networks. IEEE Trans. Pattern Anal. Mach. Intell., 38, 295-307(2015).

    [49] Y. LeCun, Y. Bengio, G. Hinton. Deep learning. Nature, 521, 436-444(2015).

    [50] A. Turpin, I. Vishniakou, J. D. Seelig. Light scattering control in transmission and reflection with neural networks. Opt. Express, 26, 30911-30929(2018).

    [51] Y. Zhang, C. Wu, Y. Song, K. Si, Y. Zheng, L. Hu, J. Chen, L. Tang, W. Gong. Machine learning based adaptive optics for doughnut-shaped beam. Opt. Express, 27, 16871-16881(2019).

    [52] S. Cheng, H. Li, Y. Luo, Y. Zheng, P. Lai. Artificial intelligence-assisted light control and computational imaging through scattering media. J. Innov. Opt. Health Sci., 12, 1930006(2019).

    [53] Y. Li, Y. Xue, L. Tian. Deep speckle correlation: a deep learning approach toward scalable imaging through scattering media. Optica, 5, 1181-1190(2018).

    [54] S. Li, M. Deng, J. Lee, A. Sinha, G. Barbastathis. Imaging through glass diffusers using densely connected convolutional networks. Optica, 5, 803-813(2018).

    [55] B. Rahmani, D. Loterie, G. Konstantinou, D. Psaltis, C. Moser. Multimode optical fiber transmission with a deep learning network. Light Sci. Appl., 7, 69(2018).

    [56] Y. Sun, J. Shi, L. Sun, J. Fan, G. Zeng. Image reconstruction through dynamic scattering media based on deep learning. Opt. Express, 27, 16032-16046(2019).

    [57] Y. Wang, J. Zhang, H. Zhu, M. Long, J. Wang, P. S. Yu. Memory in memory: a predictive neural network for learning higher-order non-stationarity from spatiotemporal dynamics. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 9154-9162(2019).

    [58] Q. Luo, J. A. Newman, K. J. Webb. Motion-based coherent optical imaging in heavily scattering random media. Opt. Lett., 44, 2716-2719(2019).

    [59] H. Yilmaz, E. G. van Putten, J. Bertolotti, A. Lagendijk, W. L. Vos, A. P. Mosk. Speckle correlation resolution enhancement of wide-field fluorescence imaging. Optica, 2, 424-429(2015).

    [60] A. Porat, E. R. Andresen, H. Rigneault, D. Oron, S. Gigan, O. Katz. Widefield lensless imaging through a fiber bundle via speckle correlations. Opt. Express, 24, 16835-16855(2016).

    [61] I. M. Vellekoop. Controlling the propagation of light in disordered scattering media(2008).

    [62] Z. Wei, X. Chen. Deep-learning schemes for full-wave nonlinear inverse scattering problems. IEEE Trans. Geosci. Remote Sens., 57, 1849-1860(2018).

    [63] W. C. Chew, Y.-M. Wang. Reconstruction of two-dimensional permittivity distribution using the distorted Born iterative method. IEEE Trans. Med. Imaging, 9, 218-225(1990).

    [64] X. Chen. Subspace-based optimization method for solving inverse-scattering problems. IEEE Trans. Geosci. Remote Sens., 48, 42-49(2009).

    [65] U. S. Kamilov, H. Mansour. Learning optimal nonlinearities for iterative thresholding algorithms. IEEE Signal Process. Lett., 23, 747-751(2016).

    [66] K. H. Jin, M. T. McCann, E. Froustey, M. Unser. Deep convolutional neural network for inverse problems in imaging. IEEE Trans. Image Process., 26, 4509-4522(2017).

    [67] E. Rueckert, M. Nakatenus, S. Tosatto, J. Peters. Learning inverse dynamics models in o(n) time with LSTM networks. IEEE-RAS 17th International Conference on Humanoid Robotics (Humanoids), 811-816(2017).

    [68] J. S. Marron, M. P. Wand. Exact mean integrated squared error. Ann. Stat., 20, 712-736(1992).

    [69] S. Feng, C. Kane, P. A. Lee, A. D. Stone. Correlations and fluctuations of coherent wave transmission through disordered media. Phys. Rev. Lett., 61, 834-837(1988).

    [70] M. Breitkreiz, P. W. Brouwer. Semiclassical theory of speckle correlations. Phys. Rev. E, 88, 062905(2013).

    [71] P. Sebbah. Waves and Imaging through Complex Media(2001).

    [72] L. Belfore, A. Arkadan, B. Lenhardt. ANN inverse mapping technique applied to electromagnetic design. IEEE Trans. Magn., 37, 3584-3587(2001).

    [73] L. Li, J. Pan, W.-S. Lai, C. Gao, N. Sang, M.-H. Yang. Learning a discriminative prior for blind image deblurring. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 6616-6625(2018).

    [74] J. Adler, O. Öktem. Solving ill-posed inverse problems using iterative deep neural networks. Inverse Prob., 33, 124007(2017).

    [75] A. Lucas, M. Iliadis, R. Molina, A. K. Katsaggelos. Using deep neural networks for inverse problems in imaging: beyond analytical methods. IEEE Signal Process. Mag., 35, 20-36(2018).

    [76] J. Rick Chang, C.-L. Li, B. Poczos, B. Vijaya Kumar, A. C. Sankaranarayanan. One network to solve them all--solving linear inverse problems using deep projection models. Proceedings of the IEEE International Conference on Computer Vision, 5888-5897(2017).

    [77] J. Yosinski, J. Clune, Y. Bengio, H. Lipson. How transferable are features in deep neural networks?. Advances in Neural Information Processing Systems, 3320-3328(2014).

    [78] H.-C. Shin, H. R. Roth, M. Gao, L. Lu, Z. Xu, I. Nogues, J. Yao, D. Mollura, R. M. Summers. Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning. IEEE Trans. Med. Imaging, 35, 1285-1298(2016).

    [79] L. Castrejon, Y. Aytar, C. Vondrick, H. Pirsiavash, A. Torralba. Learning aligned cross-modal representations from weakly aligned data. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2940-2949(2016).

    [80] V. Tran, S. K. Sahoo, C. Dang. Fast 3D movement of a laser focusing spot behind scattering media by utilizing optical memory effect and optical conjugate planes. Sci. Rep., 9, 1(2019).

    [81] M. M. Qureshi, J. Brake, H.-J. Jeon, H. Ruan, Y. Liu, A. M. Safi, T. J. Eom, C. Yang, E. Chung. In vivo study of optical speckle decorrelation time across depths in the mouse brain. Biomed. Opt. Express, 8, 4855-4864(2017).

    [82] Z. Yu, H. Li, P. Lai. Wavefront shaping and its application to enhance photoacoustic imaging. Appl. Sci., 7, 1320(2017).

    [83] Y. Luo, S. Yan, H. Li, P. Lai, Y. Zheng. Datafile_towards smart optical focusing(2020).

    [84] I. Nissilä, T. Noponen, J. Heino, T. Kajava, T. Katila. Diffuse optical imaging. Advances in Electromagnetic Fields in Living Systems, 4(2005).

    [85] A. Xavier. An Introduction to ConvLSTM(2019).

    [86] S. Xingjian, Z. Chen, H. Wang, D.-Y. Yeung, W.-K. Wong, W.-C. Woo. Convolutional LSTM network: a machine learning approach for precipitation nowcasting. Advances in Neural Information Processing Systems, 802-810(2015).

    [87] L. Kaufman, D. M. Kramer, L. E. Crooks, D. A. Ortendahl. Measuring signal-to-noise ratios in MR imaging. Radiology, 173, 265-267(1989).

    [88] B. M. Welsh. Speckle imaging signal-to-noise ratio performance as a function of frame integration time. J. Opt. Soc. Am. A, 12, 1364-1374(1995).

    [89] J. W. Tay, P. Lai, Y. Suzuki, L. V. Wang. Ultrasonically encoded wavefront shaping for focusing into random media. Sci. Rep., 4, 3918(2014).

    [90] Z. Fayyaz, N. Mohammadian, M. R. Avanaki. Comparative assessment of five algorithms to control an SLM for focusing coherent light through scattering media. Proc. SPIE, 10494, 104946I(2018).

    [91] L. Zhou, Y. Xiao, W. Chen. Learning complex scattering media for optical encryption. Opt. Lett., 45, 5279-5282(2020).

    CLP Journals

    [1] Qian Zhao, Shijie Tu, Qiannan Lei, Chengshan Guo, Qiwen Zhan, Yangjian Cai. Creation of cylindrical vector beams through highly anisotropic scattering media with a single scalar transmission matrix calibration[J]. Photonics Research, 2022, 10(7): 1617

    [2] Li Gao, Yang Chai, Darko Zibar, Zongfu Yu. Deep learning in photonics: introduction[J]. Photonics Research, 2021, 9(8): DLP1

    Yunqi Luo, Suxia Yan, Huanhao Li, Puxiang Lai, Yuanjin Zheng. Towards smart optical focusing: deep learning-empowered dynamic wavefront shaping through nonstationary scattering media[J]. Photonics Research, 2021, 9(8): B262
    Download Citation