• Photonics Research
  • Vol. 8, Issue 8, 1350 (2020)
Chang Ling1, Chonglei Zhang1,2,*, Mingqun Wang1, Fanfei Meng1..., Luping Du1,3,* and Xiaocong Yuan1,4,*|Show fewer author(s)
Author Affiliations
  • 1Nanophotonics Research Center, Shenzhen Key Laboratory of Micro-Scale Optical Information Technology & Institute of Microscale Optoelectronics, Shenzhen University, Shenzhen 518060, China
  • 2e-mail: clzhang@szu.edu.cn
  • 3e-mail: lpdu@szu.edu.cn
  • 4e-mail: xcyuan@szu.edu.cn
  • show less
    DOI: 10.1364/PRJ.396122 Cite this Article Set citation alerts
    Chang Ling, Chonglei Zhang, Mingqun Wang, Fanfei Meng, Luping Du, Xiaocong Yuan, "Fast structured illumination microscopy via deep learning," Photonics Res. 8, 1350 (2020) Copy Citation Text show less
    References

    [1] E. Abbe. Contributions to the theory of the microscope and that microscopic perception. Arch. Microsc. Anat., 9, 413-468(1873).

    [2] M. J. Rust, M. Bates, X. Zhuang. Sub-diffraction-limit imaging by stochastic optical reconstruction microscopy (STORM). Nat. Methods, 3, 793-795(2006).

    [3] M. Bates, B. Huang, G. T. Dempsey, X. Zhuang. Multicolor super-resolution imaging with photo-switchable fluorescent probes. Science, 317, 1749-1753(2007).

    [4] S. T. Hess, T. P. K. Girirajan, M. D. Mason. Ultra-high resolution imaging by fluorescence photoactivation localization microscopy. Biophys. J., 91, 4258-4272(2006).

    [5] H. Shroff, C. G. Galbraith, J. A. Galbraith, E. Betzig. Live-cell photoactivated localization microscopy of nanoscale adhesion dynamics. Nat. Methods, 5, 417-423(2008).

    [6] M. G. L. Gustafsson, D. A. Agard, J. W. Sedat. Doubling the lateral resolution of wide-field fluorescence microscopy using structured illumination. Proc. SPIE, 3919, 141-150(2000).

    [7] M. G. L. Gustafsson, L. Shao, P. M. Carlton, C. J. R. Wang, I. N. Golubovskaya, W. Z. Cande, D. A. Agard, J. W. Sedat. Three-dimensional resolution doubling in wide-field fluorescence microscopy by structured illumination. Biophys. J., 94, 4957-4970(2008).

    [8] T. A. Klar, S. Jakobs, M. Dyba, A. Egner, S. W. Hell. Fluorescence microscopy with diffraction resolution barrier broken by stimulated emission. Proc. Natl. Acad. Sci. USA, 97, 8206-8210(2000).

    [9] E. Betzig, G. H. Patterson, R. Sougrat, O. W. Lindwasser, S. Olenych, J. S. Bonifacino, M. W. Davidson, J. Lippincott-Schwartz, H. F. Hess. Imaging intracellular fluorescent proteins at nanometer resolution. Science, 313, 1642-1645(2006).

    [10] H. Linnenbank, T. Steinle, F. Morz, M. Floss, C. Han, A. Glidle, H. Giessen. Robust and rapidly tunable light source for SRS/CARS microscopy with low-intensity noise. Adv. Photon., 1, 055001(2019).

    [11] P. Fei, J. Nie, J. Lee, Y. Ding, S. Li, H. Zhang, M. Hagiwara, T. Yu, T. Segura, C.-M. Ho, D. Zhu, T. K. Hsiai. Subvoxel light-sheet microscopy for high-resolution high-throughput volumetric imaging of large biomedical specimens. Adv. Photon., 1, 016002(2019).

    [12] E. Narimanov. Resolution limit of label-free far-field microscopy. Adv. Photon., 1, 056003(2019).

    [13] E. F. Fornasiero, K. Wicker, S. O. Rizzoli. Super-resolution fluorescence microscopy using structured illumination. Super-Resolution Microscopy Techniques in the Neurosciences, 133-165(2014).

    [14] M. G. L. Gustafsson. Nonlinear structured-illumination microscopy: wide-field fluorescence imaging with theoretically unlimited resolution. Proc. Natl. Acad. Sci. USA, 102, 13081-13086(2005).

    [15] F. Orieux, E. Sepulveda, V. Loriette, B. Dubertret, J.-C. Olivo-Marin. Bayesian estimation for optimized structured illumination microscopy. IEEE Trans. Image Process., 21, 601-614(2012).

    [16] S. Dong, J. Liao, K. Guo, L. Bian, J. Suo, G. Zheng. Resolution doubling with a reduced number of image acquisitions. Biomed. Opt. Express, 6, 2946-2952(2015).

    [17] A. Lal, C. Shan, K. Zhao, W. Liu, X. Huang, W. Zong, L. Chen, P. Xi. A frequency domain SIM reconstruction algorithm using reduced number of images. IEEE Trans. Image Process., 27, 4555-4570(2018).

    [18] F. Strohl, C. F. Kaminski. Speed limits of structured illumination microscopy. Opt. Lett., 42, 2511-2514(2017).

    [19] W. H. Richardson. Bayesian-based iterative method of image restoration. J. Opt. Soc. Am., 62, 55-59(1972).

    [20] M. Ingaramo, A. G. York, E. Hoogendoorn, M. Postma, H. Shroff, G. H. Patterson. Richardson-Lucy deconvolution as a general tool for combining images with complementary strengths. Chem. Phys. Chem., 15, 794-800(2014).

    [21] M. I. Jordan, T. M. Mitchell. Machine learning: trends, perspectives, and prospects. Science, 349, 255-260(2015).

    [22] Y. LeCun, Y. Bengio, G. Hinton. Deep learning. Nature, 521, 436-444(2015).

    [23] A. Krizhevsky, I. Sutskever, G. E. Hinton. ImageNet classification with deep convolutional neural networks. Commun. ACM, 60, 84-90(2017).

    [24] Y. Rivenson, Z. Gorocs, H. Gunaydin, Y. Zhang, H. Wang, A. Ozcan. Deep learning microscopy. Optica, 4, 1437-1443(2017).

    [25] W. Ouyang, A. Aristov, M. Lelek, X. Hao, C. Zimmer. Deep learning massively accelerates super-resolution localization microscopy. Nat. Biotechnol., 36, 460-468(2018).

    [26] E. Nehme, L. E. Weiss, T. Michaeli, Y. Shechtman. Deep-STORM: super-resolution single-molecule microscopy by deep learning. Optica, 5, 458-464(2018).

    [27] N. Thanh, Y. Xue, Y. Li, L. Tian, G. Nehmetallah. Deep learning approach to Fourier ptychographic microscopy. Opt. Express, 26, 26470-26484(2018).

    [28] Z. Ghahramani, I. J. Goodfellow, J. Pouget-Abadie, M. Welling, C. Cortes, M. Mirza, N. D. Lawrence, B. Xu, D. Warde-Farley, K. Q. Weinberger, S. Ozair, A. Courville, Y. Bengio. Generative adversarial nets. Proceedings of the 27th International Conference on Neural Information Processing Systems, 2672-2680(2014).

    [29] , J.-Y. Zhu, T. Park, P. Isola, A. A. Efros. Unpaired image-to-image translation using cycle-consistent adversarial networks. IEEE International Conference on Computer Vision, 2242-2251(2017).

    [30] M. Mirza, S. Osindero. Conditional generative adversarial nets(2014).

    [31] L. A. Gatys, A. S. Ecker, M. Bethge. Image style transfer using convolutional neural networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2414-2423(2016).

    [32] , P. Isola, J.-Y. Zhu, T. Zhou, A. A. Efros. Image-to-image translation with conditional adversarial networks. 30th IEEE Conference on Computer Vision and Pattern Recognition, 5967-5976(2017).

    [33] B. Leibe, C. Li, J. Matas, M. Wand, N. Sebe, M. Welling. Precomputed real-time texture synthesis with Markovian generative adversarial networks. Computer Vision—European Conference on Computer Vision (ECCV), 702-716(2016).

    [34] K. Daniilidis, N. Sundaram, T. Brox, P. Maragos, K. Keutzer, N. Paragios. Dense point trajectories by GPU-accelerated large displacement optical flow. Computer Vision—European Conference on Computer Vision (ECCV), 438-451(2010).

    [35] , C. Godard, O. Mac Aodha, G. J. Brostow. Unsupervised monocular depth estimation with left-right consistency. 30th IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 6602-6611(2017).

    [36] , K. He, X. Zhang, S. Ren, J. Sun. Deep residual learning for image recognition. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 770-778(2016).

    [37] A. Lal, C. Shan, P. Xi. Structured illumination microscopy image reconstruction algorithm. IEEE J. Sel. Top. Quantum Electron., 22, 6803414(2016).

    [38] M. Mueller, V. Moenkemoeller, S. Hennig, W. Huebner, T. Huser. Open-source image reconstruction of super-resolution structured illumination microscopy data in ImageJ. Nat. Commun., 7, 10980(2016).

    [39] Z. Wang, A. C. Bovik, H. R. Sheikh, E. P. Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process., 13, 600-612(2004).

    [40] M. B. Matthews, Z. Wang, E. P. Simoncelli, A. C. Bovik. Multi-scale structural similarity for image quality assessment. Conference Record of the 37th Asilomar Conference on Signals, Systems & Computers, 1398-1402(2003).

    [41] H. Wang, Y. Rivenson, Y. Jin, Z. Wei, R. Gao, H. Gunaydin, L. A. Bentolila, C. Kural, A. Ozcan. Deep learning enables cross-modality super-resolution in fluorescence microscopy. Nat. Methods, 16, 103-110(2019).

    [42] L. Jin, B. Liu, F. Zhao, S. Hahn, B. Dong, R. Song, T. C. Elston, Y. Xu, K. M. Hahn. Deep learning enables structured illumination microscopy with low light levels and enhanced speed. Nat. Commun., 11, 1934(2020).

    CLP Journals

    [1] Shuo Zhu, Enlai Guo, Jie Gu, Lianfa Bai, Jing Han, "Imaging through unknown scattering media based on physics-informed learning," Photonics Res. 9, B210 (2021)

    [2] Zafran Hussain Shah, Marcel Müller, Tung-Cheng Wang, Philip Maurice Scheidig, Axel Schneider, Mark Schüttpelz, Thomas Huser, Wolfram Schenck, "Deep-learning based denoising and reconstruction of super-resolution structured illumination microscopy images," Photonics Res. 9, B168 (2021)

    Chang Ling, Chonglei Zhang, Mingqun Wang, Fanfei Meng, Luping Du, Xiaocong Yuan, "Fast structured illumination microscopy via deep learning," Photonics Res. 8, 1350 (2020)
    Download Citation