• Laser & Optoelectronics Progress
  • Vol. 59, Issue 6, 0617009 (2022)
Yujun Tang1、2, Linbo Wang2, Gang Wen2, and Hui Li1、2、*
Author Affiliations
  • 1School of Biomedical Engineering, University of Science and Technology of China, Suzhou , Jiangsu 215163, China
  • 2Jiangsu Key Laboratory of Medical Optics, Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Sciences, Suzhou , Jiangsu 215163, China
  • show less
    DOI: 10.3788/LOP202259.0617009 Cite this Article Set citation alerts
    Yujun Tang, Linbo Wang, Gang Wen, Hui Li. Recent Advances in Structured Illumination Microscope Super-Resolution Image Reconstruction[J]. Laser & Optoelectronics Progress, 2022, 59(6): 0617009 Copy Citation Text show less
    References

    [1] Leung B O, Chou K C. Review of super-resolution fluorescence microscopy for biology[J]. Applied Spectroscopy, 65, 967-980(2011).

    [2] Bates M, Huang B, Zhuang X W. Super-resolution microscopy by nanoscale localization of photo-switchable fluorescent probes[J]. Current Opinion in Chemical Biology, 12, 505-514(2008).

    [3] Fu Y, Wang T L, Zhao S. Imaging principles and applications of super-resolution optical microscopy[J]. Laser & Optoelectronics Progress, 56, 240002(2019).

    [4] Yang J, Tian C P, Zhong G S. Stochastic optical reconstruction microscopy imaging technique and application[J]. Acta Optica Sinica, 37, 0318005(2017).

    [5] Anwar S, Khan S, Barnes N. A deep journey into super-resolution: a survey[J]. ACM Computing Surveys, 53, 1-34(2021).

    [6] Wu Y, FasterShroff H.. sharper, and deeper: structured illumination microscopy for biological imaging[J]. Nature Methods, 15, 1011-1019(2018).

    [7] Hirvonen L M, Wicker K, Mandula O et al. Structured illumination microscopy of a living cell[J]. European Biophysics Journal: EBJ, 38, 807-812(2009).

    [8] Jost A, Heintzmann R. Superresolution multidimensional imaging with structured illumination microscopy[J]. Annual Review of Materials Research, 43, 261-282(2013).

    [9] Heintzmann R, Huser T. Super-resolution structured illumination microscopy[J]. Chemical Reviews, 117, 13890-13908(2017).

    [10] Samanta K, Joseph J. An overview of structured illumination microscopy: recent advances and perspectives[J]. Journal of Optics, 23, 123002(2021).

    [11] Demmerle J, Innocent C, North A J et al. Strategic and practical guidelines for successful structured illumination microscopy[J]. Nature Protocols, 12, 988-1010(2017).

    [12] Hirano Y, Matsuda A, Hiraoka Y. Recent advancements in structured-illumination microscopy toward live-cell imaging[J]. Microscopy, 64, 237-249(2015).

    [13] Gustafsson M G L. Nonlinear structured-illumination microscopy: wide-field fluorescence imaging with theoretically unlimited resolution[J]. Proceedings of the National Academy of Sciences of the United States of America, 102, 13081-13086(2005).

    [14] Gustafsson M G L, Agard D A, Sedat J W. Doubling the lateral resolution of wide-field fluorescence microscopy using structured illumination[J]. Proceedings of SPIE, 3919, 141-150(2000).

    [15] Kner P, Chhun B B, Griffis E R et al. Super-resolution video microscopy of live cells by structured illumination[J]. Nature Methods, 6, 339-342(2009).

    [16] Chang B J, Chou L J, Chang Y C et al. Isotropic image in structured illumination microscopy patterned with a spatial light modulator[J]. Optics Express, 17, 14710-14721(2009).

    [17] Dan D, Lei M, Yao B et al. DMD-based LED-illumination super-resolution and optical sectioning microscopy[J]. Scientific Reports, 3, 1116(2013).

    [18] Righolt C H, Slotman J A, Young I T et al. Image filtering in structured illumination microscopy using the Lukosz bound[J]. Optics Express, 21, 24431-24451(2013).

    [19] Fan J C, Huang X S, Li L J et al. A protocol for structured illumination microscopy with minimal reconstruction artifacts[J]. Biophysics Reports, 5, 80-90(2019).

    [20] Shi R, Janunts N, Heintzmann R et al. Fast-physical optics modeling of microscopy system with structured illumination[J]. Proceedings of SPIE, 10694, 106940I(2018).

    [21] Patwary N, Doblas A, Preza C. Image restoration approach to address reduced modulation contrast in structured illumination microscopy[J]. Biomedical Optics Express, 9, 1630-1647(2018).

    [22] Hagen N, Gao L, Tkaczyk T S. Quantitative sectioning and noise analysis for structured illumination microscopy[J]. Optics Express, 20, 403-413(2012).

    [23] Lal A, Shan C Y, Xi P. Structured illumination microscopy image reconstruction algorithm[J]. IEEE Journal of Selected Topics in Quantum Electronics, 22, 50-63(2016).

    [24] Gustafsson M G L, Shao L, Carlton P M et al. Three-dimensional resolution doubling in wide-field fluorescence microscopy by structured illumination[J]. Biophysical Journal, 94, 4957-4970(2008).

    [25] Lahrberg M, Singh M, Khare K et al. Accurate estimation of the illumination pattern’s orientation and wavelength in sinusoidal structured illumination microscopy[J]. Applied Optics, 57, 1019-1025(2018).

    [26] Cao R Z, Chen Y H, Liu W J et al. Inverse matrix based phase estimation algorithm for structured illumination microscopy[J]. Biomedical Optics Express, 9, 5037-5051(2018).

    [27] Shroff S A, Fienup J R, Williams D R. Phase-shift estimation in sinusoidally illuminated images for lateral superresolution[J]. Journal of the Optical Society of America. A, Optics, Image Science, and Vision, 26, 413-424(2009).

    [28] Wicker K, Mandula O, Best G et al. Phase optimisation for structured illumination microscopy[J]. Optics Express, 21, 2032-2049(2013).

    [29] Wicker K. Non-iterative determination of pattern phase in structured illumination microscopy using auto-correlations in Fourier space[J]. Optics Express, 21, 24692-24701(2013).

    [30] Zhou X, Lei M, Dan D et al. Image recombination transform algorithm for superresolution structured illumination microscopy[J]. Journal of Biomedical Optics, 21, 096009(2016).

    [31] Gustafsson M G L. Surpassing the lateral resolution limit by a factor of two using structured illumination microscopy[J]. Journal of Microscopy, 198, 82-87(2000).

    [32] Karras C, Smedh M, Förster R et al. Successful optimization of reconstruction parameters in structured illumination microscopy-a practical guide[J]. Optics Communications, 436, 69-75(2019).

    [33] York A G, Chandris P, Nogare D D et al. Instant super-resolution imaging in live cells and embryos via analog image processing[J]. Nature Methods, 10, 1122-1126(2013).

    [34] Shaw M, Zajiczek L, O’Holleran K. High speed structured illumination microscopy in optically thick samples[J]. Methods, 88, 11-19(2015).

    [35] Zhang Y W, Lang S, Wang H W et al. Super-resolution algorithm based on Richardson-Lucy deconvolution for three-dimensional structured illumination microscopy[J]. Journal of the Optical Society of America. A, Optics, Image Science, and Vision, 36, 173-178(2019).

    [36] Chen X Y, Zhanghao K, Li M Q et al. Enhanced reconstruction of structured illumination microscopy on a polarized specimen[J]. Optics Express, 28, 25642-25654(2020).

    [37] Chakrova N, Rieger B, Stallinga S. Deconvolution methods for structured illumination microscopy[J]. Journal of the Optical Society of America A, 33, B12-B20(2016).

    [38] York A G, Parekh S H, Nogare D D et al. Resolution doubling in live, multicellular organisms via multifocal structured illumination microscopy[J]. Nature Methods, 9, 749-754(2012).

    [39] Wu J J, Li S W, Cao H Q et al. Resolution improvement of multifocal structured illumination microscopy with sparse Bayesian learning algorithm[J]. Optics Express, 26, 31430-31438(2018).

    [40] Chakrova N, Heintzmann R, Rieger B et al. Studying different illumination patterns for resolution improvement in fluorescence microscopy[J]. Optics Express, 23, 31367-31383(2015).

    [41] Ströhl F, Kaminski C F. A joint Richardson-Lucy deconvolution algorithm for the reconstruction of multifocal structured illumination microscopy data[J]. Methods and Applications in Fluorescence, 3, 014002(2015).

    [42] Schultz R R, Stevenson R L. Extraction of high-resolution frames from video sequences[J]. IEEE Transactions on Image Processing: a Publication of the IEEE Signal Processing Society, 5, 996-1011(1996).

    [43] Richardson W H. Bayesian-based iterative method of image restoration[J]. Journal of the Optical Society of America, 62, 55-59(1972).

    [44] Lucy L B. An iterative technique for the rectification of observed distributions[J]. The Astronomical Journal, 79, 745(1974).

    [45] Ingaramo M, York A G, Hoogendoorn E et al. Richardson–Lucy deconvolution as a general tool for combining images with complementary strengths[J]. ChemPhysChem, 15, 794-800(2014).

    [46] Zhao W, Zhao S, Li L et al. Sparse deconvolution improves the resolution of live-cell super-resolution fluorescence microscopy[J]. Nature Biotechnology, 1-12(2021).

    [47] Perez V, Chang B J, Stelzer E H K. Optimal 2D-SIM reconstruction by two filtering steps with Richardson-Lucy deconvolution[J]. Scientific Reports, 6, 37149(2016).

    [48] Müller M, Mönkemöller V, Hennig S et al. Open-source image reconstruction of super-resolution structured illumination microscopy data in ImageJ[J]. Nature Communications, 7, 10980(2016).

    [49] Chu K Q, McMillan P J, Smith Z J et al. Image reconstruction for structured-illumination microscopy with low signal level[J]. Optics Express, 22, 8687-8702(2014).

    [50] Lukeš T, Křížek P, Švindrych Z et al. Three-dimensional super-resolution structured illumination microscopy with maximum a posteriori probability image estimation[J]. Optics Express, 22, 29805-29817(2014).

    [51] Huang X, Fan J, Li L et al. Fast, long-term, super-resolution imaging with Hessian structured illumination microscopy[J]. Nature Biotechnology, 36, 451-459(2018).

    [52] Zhu H F, Sun Y L, Yin L et al. 3D super-resolution microscopy based on nonlinear gradient descent structured illumination[J]. Optics Express, 29, 21428-21443(2021).

    [53] Ströhl F, Kaminski C F. Speed limits of structured illumination microscopy[J]. Optics Letters, 42, 2511-2514(2017).

    [54] Lal A, Shan C Y, Zhao K et al. A frequency domain SIM reconstruction algorithm using reduced number of images[J]. IEEE Transactions on Image Processing: a Publication of the IEEE Signal Processing Society, 27, 4555-4570(2018).

    [55] Lal A, Huang X S, Xi P. A frequency domain reconstruction of SIM image using four raw images[J]. Proceedings of SPIE, 10024, 1002411(2016).

    [56] Zhao T Y, Wang Z J, Chen T S et al. Advances in high-speed structured illumination microscopy[J]. Frontiers in Physics, 9, 672555(2021).

    [57] Wen G, Li S, Wang L et al. High-fidelity structured illumination microscopy by point-spread-function engineering[J]. Light: Science & Applications, 10, 70(2021).

    [58] Smith C S, Slotman J A, Schermelleh L et al. Structured illumination microscopy with noise-controlled image reconstructions[J]. Nature Methods, 18, 821-828(2021).

    [59] Dong S Y, Nanda P, Shiradkar R et al. High-resolution fluorescence imaging via pattern-illuminated Fourier ptychography[J]. Optics Express, 22, 20856-20870(2014).

    [60] Dong S Y, Liao J, Guo K K et al. Resolution doubling with a reduced number of image acquisitions[J]. Biomedical Optics Express, 6, 2946-2952(2015).

    [61] Orieux F, Sepulveda E, Loriette V et al. Bayesian estimation for optimized structured illumination microscopy[J]. IEEE Transactions on Image Processing: a Publication of the IEEE Signal Processing Society, 21, 601-614(2012).

    [62] Tu S J, Liu Q L, Liu X et al. Fast reconstruction algorithm for structured illumination microscopy[J]. Optics Letters, 45, 1567-1570(2020).

    [63] Bezzubik V V, Belashenkov N R, Vasilyev V N et al. Optimization-based image reconstruction method for super-resolution structured-illumination microscopy[J]. Journal of Optical Technology, 86, 748-757(2019).

    [64] Ball G, Demmerle J, Kaufmann R et al. SIMcheck: a toolbox for successful super-resolution structured illumination microscopy[J]. Scientific Reports, 5, 15915(2015).

    [65] Křížek P, Lukeš T, Ovesný M et al. SIMToolbox: a MATLAB toolbox for structured illumination fluorescence microscopy[J]. Bioinformatics, 32, 318-320(2015).

    [66] Hang Y C, Liao Q M, Yang W M et al. Attention cube network for image restoration[C], 2562-2570(2020).

    [67] Zhang Y L, Tian Y P, Kong Y et al. Residual dense network for image restoration[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43, 2480-2495(2021).

    [68] Vaswani A, Shazeer N, Parmar N et al. Attention is all you need[C], 5998-6008(2017).

    [69] Zhang X, Zhang M S, Li D et al. Highly photostable, reversibly photoswitchable fluorescent protein with high contrast ratio for live-cell superresolution microscopy[J]. Proceedings of the National Academy of Sciences of the United States of America, 113, 10364-10369(2016).

    [70] Liu Z C, Jin L H, Chen J C et al. A survey on applications of deep learning in microscopy image analysis[J]. Computers in Biology and Medicine, 134, 104523(2021).

    [71] Bai C, Qian J, Dang S P et al. Full-color optically-sectioned imaging by wide-field microscopy via deep-learning[J]. Biomedical Optics Express, 11, 2619-2632(2020).

    [72] Shajkofci A, Liebling M. Spatially-variant CNN-based point spread function estimation for blind deconvolution and depth estimation in optical microscopy[J]. IEEE Transactions on Image Processing, 29, 5848-5861(2020).

    [73] Yang X L, Huang L Z, Luo Y L et al. Deep learning-based virtual refocusing of images using an engineered point-spread function[J]. ACS Photonics, 8, 2174-2182(2021).

    [74] Ding H, Li F J, Meng Z et al. Auto-focusing and quantitative phase imaging using deep learning for the incoherent illumination microscopy system[J]. Optics Express, 29, 26385-26403(2021).

    [75] Chen J, Sasaki H, Lai H et al. Three-dimensional residual channel attention networks denoise and sharpen fluorescence microscopy image volumes[J]. Nature Methods, 18, 678-687(2021).

    [76] Wang H D, Rivenson Y, Jin Y Y et al. Deep-learning enables cross-modality super-resolution in fluorescence microscopy[J]. Nature methods, 16, 103-110(2019).

    [77] Laine R F, Goodfellow G, Young L J et al. Structured illumination microscopy combined with machine learning enables the high throughput analysis and classification of virus structure[J]. eLife, 7, e40183(2018).

    [78] Jin L, Liu B, Zhao F et al. Deep learning enables structured illumination microscopy with low light levels and enhanced speed[J]. Nature communications, 11, 1-7(2020).

    [79] Qiao C, Li D, Guo Y et al. Evaluation and development of deep neural networks for image super-resolution in optical microscopy[J]. Nature Methods, 18, 194-202(2021).

    [80] Shah Z H, Müller M, Wang T C et al. Deep-learning based denoising and reconstruction of super-resolution structured illumination microscopy images[J]. Photonics Research, 9, B168-B181(2021).

    [81] Huang L, Chen H, Luo Y et al. Recurrent neural network-based volumetric fluorescence microscopy[J]. Light: Science & Applications, 10, 62(2021).

    [82] Christensen C N, Ward E N, Lu M et al. ML-SIM: universal reconstruction of structured illumination microscopy images using transfer learning[J]. Biomedical Optics Express, 12, 2720-2733(2021).

    [83] Ling C, Zhang C L, Wang M Q et al. Fast structured illumination microscopy via deep learning[J]. Photonics Research, 8, 1350-1359(2020).

    [84] Qiao C, Chen X Y, Zhang S W et al. 3D structured illumination microscopy via channel attention generative adversarial network[J]. IEEE Journal of Selected Topics in Quantum Electronics, 27, 6801711(2021).

    Yujun Tang, Linbo Wang, Gang Wen, Hui Li. Recent Advances in Structured Illumination Microscope Super-Resolution Image Reconstruction[J]. Laser & Optoelectronics Progress, 2022, 59(6): 0617009
    Download Citation