Super-Resolution Fluorescence Microscopy Image Reconstruction Algorithm Based on Structured Illumination

Zhi Liu; Zewei Luo; Zhengyin Wang; Zhuang Tu; Zhengfei Zhuang; Tongsheng Chen

doi:10.3788/CJL202148.0307001

[2] Sigal Y M, Zhou R, Zhuang X. Visualizing and discovering cellular structures with super-resolution microscopy[J]. Science, 361, 880-887(2018). http://www.researchgate.net/publication/327332369_Visualizing_and_discovering_cellular_structures_with_super-resolution_microscopy

[3] Hell S W, Wichmann J. Breaking the diffraction resolution limit by stimulated emission: stimulated-emission-depletion fluorescence microscopy[J]. Optics Letters, 19, 780-782(1994). http://www.opticsinfobase.org/ol/abstract.cfm?uri=ol-19-11-780

[4] Gustafsson M G L. Surpassing the lateral resolution limit by a factor of two using structured illumination microscopy[J]. Journal of Microscopy, 198, 82-87(2000). http://www.tandfonline.com/servlet/linkout?suffix=cit0008&dbid=8&doi=10.1080%2F05704928.2017.1323309&key=10810003

[5] Betzig E, Patterson G H, Sougrat R et al. Imaging intracellular fluorescent proteins at nanometer resolution[J]. Science, 313, 1642-1645(2006).

[6] Rust M J, Bates M, Zhuang X W. Sub-diffraction-limit imaging by stochastic optical reconstruction microscopy (STORM)[J]. Nature Methods, 3, 793-795(2006).

[7] Pan W H, Chen B L, Zhang J G et al. Compressed sensing STORM super-resolution image reconstruction based on noise correction-principal component analysis preprocessing algorithm[J]. Chinese Journal of Lasers, 47, 0207024(2020).

[8] Schermelleh L, Ferrand A, Huser T et al. Super-resolution microscopy demystified[J]. Nature Cell Biology, 21, 72-84(2019).

[9] Sauer M, Heilemann M. Single-molecule localization microscopy in eukaryotes[J]. Chemical Reviews, 117, 7478-7509(2017). http://www.ncbi.nlm.nih.gov/pubmed/28287710

[10] Hofmann M, Eggeling C, Jakobs S et al. Breaking the diffraction barrier in fluorescence microscopy at low light intensities by using reversibly photoswitchable proteins[J]. Proceedings of the National Academy of Sciences, 102, 17565-17569(2005). http://www.ncbi.nlm.nih.gov/pmc/PMC1308899

[11] York A G, Chandris P, Nogare D D et al. Instant super-resolution imaging in live cells and embryos via analog image processing[J]. Nature Methods, 10, 1122-1126(2013). http://www.tandfonline.com/servlet/linkout?suffix=cit0099&dbid=8&doi=10.1080%2F05704928.2017.1323309&key=24097271

[12] Huang X, Fan J, Li L et al. Fast, long-term, super-resolution imaging with Hessian structured illumination microscopy[J]. Nature Biotechnology, 36, 451-459(2018). http://europepmc.org/abstract/MED/29644998

[13] Balzarotti F, Eilers Y, Gwosch K C et al. Nanometer resolution imaging and tracking of fluorescent molecules with minimal photon fluxes[J]. Science, 355, 606-612(2017).

[14] Wu Y, Shroff H. Faster, sharper, and deeper: structured illumination microscopy for biological imaging[J]. Nature Methods, 15, 1011-1019(2018). http://www.nature.com/articles/s41592-018-0211-z

[15] Heintzmann R, Huser T. Super-resolution structured illumination microscopy[J]. Chemical Reviews, 117, 13890-13908(2017). http://europepmc.org/abstract/MED/29125755

[16] Wicker K, Heintzmann R. Resolving a misconception about structured illumination[J]. Nature Photonics, 8, 342-344(2014). http://www.nature.com/articles/nphoton.2014.88

[17] Wu M R, Yang X B, Xiong D X et al. Structured illumination fluorescence microscopy: diffraction-limit breaking principle and application in life science[J]. Laser & Optoelectronics Progress, 52, 010003(2015).

[18] Gustafsson M G. Nonlinear structured-illumination microscopy: wide-field fluorescence imaging with theoretically unlimited resolution[J]. Proceedings of the National Academy of Sciences of the United States of America, 102, 13081-13086(2005). http://europepmc.org/articles/PMC1201569

[19] Mennella V, Keszthelyi B. McDonald K L, et al. Subdiffraction-resolution fluorescence microscopy reveals a domain of the centrosome critical for pericentriolar material organization[J]. Nature Cell Biology, 14, 1159-1168(2012).

[20] Lawo S, Hasegan M, Gupta G D et al. Subdiffraction imaging of centrosomes reveals higher-order organizational features of pericentriolar material[J]. Nature Cell Biology, 14, 1148-1158(2012). http://www.nature.com/articles/ncb2591

[21] Conduit P T, Richens J H, Wainman A et al. A molecular mechanism of mitotic centrosome assembly in Drosophila[J]. Elife, 3, e03399(2014). http://europepmc.org/abstract/med/25149451

[22] Burnette D T, Shao L, Ott C et al. A contractile and counterbalancing adhesion system controls the 3D shape of crawling cells[J]. Journal of Cell Biology, 205, 83-96(2014). http://www.ncbi.nlm.nih.gov/pubmed/24711500

[23] Sonnen K F, Schermelleh L, Leonhardt H et al. 3D-structured illumination microscopy provides novel insight into architecture of human centrosomes[J]. Biology Open, 1, 965-976(2012). http://europepmc.org/articles/PMC3507176

[24] Baddeley D, Chagin V O, Schermelleh L et al. Measurement of replication structures at the nanometer scale using super-resolution light microscopy[J]. Nucleic Acids Res, 38, e8(2010). http://www.researchgate.net/publication/286181132_Measurement_of_replication_structures_at_the_nanometer_scale_using_super-resolution_light_microscopy

[25] Chagin V O. Casas-Delucchi C S, Reinhart M, et al. 4D visualization of replication foci in mammalian cells corresponding to individual replicons[J]. Nature Communications, 7, 11231(2016).

[26] Li D, Shao L, Chen B C, cytoskeletal dynamics[J]. Science et al. 349(6251): aab3500(2015).

[27] Schermelleh L, Carlton P M, Haase S et al. Subdiffraction multicolor imaging of the nuclear periphery with 3D structured illumination microscopy[J]. Science, 320, 1332-1336(2008). http://pubmedcentralcanada.ca/pmcc/articles/PMC2916659/

[28] Smeets D, Markaki Y, Schmid V J et al. Three-dimensional super-resolution microscopy of the inactive X chromosome territory reveals a collapse of its active nuclear compartment harboring distinct Xist RNA foci[J]. Epigenetics & Chromatin, 7, 1-27(2014). http://europepmc.org/articles/pmc4108088/

[29] Wen G, Li S M, Yang X B et al. Super-resolution fluorescence microscopy system by structured light illumination based on laser interference[J]. Acta Optica Sinica, 37, 0318003(2017).

[30] Zhanghao K, Chen X Y, Liu W H et al. Super-resolution imaging of fluorescent dipoles via polarized structured illumination microscopy[J]. Nature Communications, 10, 4694(2019). http://www.ncbi.nlm.nih.gov/pubmed/31619676

[31] Lal A, Shan C Y, Zhao K et al. A frequency domain SIM reconstruction algorithm using reduced number of images[J]. IEEE Transactions on Image Processing, 27, 4555-4570(2018). http://ieeexplore.ieee.org/document/8369094

[32] Heintzmann R, Cremer C G. Laterally modulated excitation microscopy: improvement of resolution by using a diffraction grating[J]. Proceedings of SPIE, 3568, 185-196(1999). http://jmicro.oxfordjournals.org/external-ref?access_num=10.1117/12.336833&link_type=DOI

[33] Lal A, Shan C Y, Xi P. Structured illumination microscopy image reconstruction algorithm[J]. IEEE Journal of Selected Topics in Quantum Electronics, 22, 50-63(2016). http://ieeexplore.ieee.org/document/7400963/

[34] Zhou X, Dan D, Qian J et al. Super-resolution reconstruction theory in structured illumination microscopy[J]. Acta Optica Sinica, 37, 0318001(2017).

[35] Demmerle J, Innocent C, North A J et al. Strategic and practical guidelines for successful structured illumination microscopy[J]. Nature Protocols, 12, 988-1010(2017). http://www.nature.com/articles/nprot.2017.019/figures/

[36] Hirvonen L M, Wicker K, Mandula O et al. Structured illumination microscopy of a living cell[J]. European Biophysics Journal, 38, 807-812(2009).

[37] Heintzmann R, Benedetti P A. High-resolution image reconstruction in fluorescence microscopy with patterned excitation[J]. Applied Optics, 45, 5037-5045(2006). http://www.opticsinfobase.org/abstract.cfm?URI=ao-45-20-5037

[38] Shroff S A, Fienup J R, Williams D R. Phase-shift estimation in sinusoidally illuminated images for lateral superresolution[J]. Journal of the Optical Society of America A, 26, 413-424(2009). http://www.opticsinfobase.org/josaa/abstract.cfm?uri=josaa-26-2-413

[39] Gustafsson M G, Shao L, Carlton P M et al. Three-dimensional resolution doubling in wide-field fluorescence microscopy by structured illumination[J]. Biophysical Journal, 94, 4957-4970(2008). http://europepmc.org/abstract/med/18326650

[40] Chang B J, Chou L J, Chang Y C et al. Isotropic image in structured illumination microscopy patterned with a spatial light modulator[J]. Optics Express, 17, 14710-14721(2009). http://www.opticsinfobase.org/abstract.cfm?URI=oe-17-17-14710

[41] Kner P, Chhun B B, Griffis E R et al. Super-resolution video microscopy of live cells by structured illumination[J]. Nature Methods, 6, 339-342(2009). http://pubs.acs.org/servlet/linkout?suffix=ref106/cit106&dbid=8&doi=10.1021%2Fac9024889&key=19404253

[42] Somekh M G, Hsu K, Pitter M C. Stochastic transfer function for structured illumination microscopy[J]. Journal of the Optical Society of America A, 26, 1630-1637(2009).

[43] Shao L, Kner P, Rego E H et al. Super-resolution 3D microscopy of live whole cells using structured illumination[J]. Nature Methods, 8, 1044-1046(2011). http://www.nature.com/nmeth/journal/v8/n12/fig_tab/nmeth.1734_f1.html

[44] Orieux F, Sepulveda E, Loriette V et al. Bayesian estimation for optimized structured illumination microscopy[J]. IEEE Transactions on Image Processing, 21, 601-614(2012). http://ieeexplore.ieee.org/document/5959980

[45] Shroff S A, Fienup J R, Williams D R. Phase-shift estimation in sinusoidally illuminated images for lateral superresolution[J]. Journal of the Optical Society of America A, 26, 413-424(2009). http://www.opticsinfobase.org/josaa/abstract.cfm?uri=josaa-26-2-413

[46] Wiener N[M]. Extrapolation, interpolation, and smoothing of stationary time series(1950).

[47] Dan D. Research on the theory and experiment of structured illumination microscopy for super-resolution and optical sectioning[D]. Beijing: University of Chinese Academy of Sciences(2017).

[48] Lukeš T, Ovesny M et al. SIMToolbox: a MATLAB toolbox for structured illumination fluorescence microscopy[J]. Bioinformatics, 32, 318-320(2016). http://smartsearch.nstl.gov.cn/paper_detail.html?id=712f1e40839580da1ff8e2b7c69a3037

[49] Lukeš T. K rˇížek P, Švindrych Z, et al. Three-dimensional super-resolution structured illumination microscopy with maximum a posteriori probability image estimation[J]. Optics Express, 22, 29805-29817(2014).

[50] Righolt C H, Slotman J A, Young I T et al. Image filtering in structured illumination microscopy using the Lukosz bound[J]. Optics Express, 21, 24431-24451(2013). http://europepmc.org/abstract/med/24150288

[51] Richardson W H. Bayesian-based iterative method of image restoration[J]. Journal of the Optical Society of America A, 62, 55-59(1972). http://www.opticsinfobase.org/josa/abstract.cfm?uri=josa-62-1-55

[52] Lucy L B. An iterative technique for the rectification of observed distributions[J]. The Astronomical Journal, 79, 745-754(1974). http://gji.oxfordjournals.org/external-ref?access_num=10.1086/111605&link_type=DOI

[53] Ingaramo M, York A G, Wawrzusin P et al. Two-photon excitation improves multifocal structured illumination microscopy in thick scattering tissue[J]. Proceedings of the National Academy of Sciences of the United States of America, 111, 5254-5259(2014).

[54] Perez V, Chang B J, Stelzer E H. Optimal 2D-SIM reconstruction by two filtering steps with Richardson-Lucy deconvolution[J]. Scientific Reports, 6, 37149(2016). http://europepmc.org/articles/PMC5111067

[55] Sage D, Donati L, Soulez F et al. DeconvolutionLab2: an open-source software for deconvolution microscopy[J]. Methods, 115, 28-41(2017). http://europepmc.org/abstract/MED/28057586

[56] Xie X L, Chen R, Zhao Y X et al. Combination light-sheet illumination with super-resolution three-dimensional fluorescence microimaging[J]. Chinese Journal of Lasers, 45, 0307006(2018).

[57] Neil M A, Juskaitis R, Wilson T. Method of obtaining optical sectioning by using structured light in a conventional microscope[J]. Optics Letters, 22, 1905-1907(1997).

[58] Wilson T. Neil M A A, Juskaitis R. Optically sectioned images in wide-field fluorescence microscopy[J]. Proceedings of SPIE, 3261, 205-216(1998).

[59] Wicker K. Increasing resolution and light efficiency in fluorescence microscopy London: King's[D]. College London(2010).

[60] Shroff S A, Fienup J R, Williams D R. Lateral superresolution using a posteriori phase shift estimation for a moving object: experimental results[J]. Journal of the Optical Society of America A, 27, 1770-1782(2010).

[61] Wicker K, Mandula O, Best G et al. Phase optimisation for structured illumination microscopy[J]. Optics Express, 21, 2032-2049(2013). http://dx.doi.org/10.1364/oe.21.002032

[62] Wicker K. Non-iterative determination of pattern phase in structured illumination microscopy using auto-correlations in Fourier space[J]. Optics Express, 21, 24692-24701(2013).

[63] Zhou X, Lei M, Dan D et al. Image recombination transform algorithm for superresolution structured illumination microscopy[J]. Journal of Biomedical Optics, 21, 96009(2016). http://www.ncbi.nlm.nih.gov/pubmed/27653935

[64] Gao L, Shao L, Higgins C D et al. Noninvasive imaging beyond the diffraction limit of 3D dynamics in thickly fluorescent specimens[J]. Cell, 151, 1370-1385(2012). http://smartsearch.nstl.gov.cn/paper_detail.html?id=40b1080efe448988adf9d0ed920d7693

[65] Fiolka R, Shao L, Rego E H et al. Time-lapse two-color 3D imaging of live cells with doubled resolution using structured illumination[J]. Proceedings of the National Academy of Sciences, 109, 5311-5315(2012). http://europepmc.org/abstract/med/22431626

[66] Dan D, Lei M, Yao B et al. DMD-based LED-illumination super-resolution and optical sectioning microscopy[J]. Scientific Reports, 3, 1116(2013). http://www.irgrid.ac.cn/handle/1471x/780211

[67] Shaw M, Zajiczek L. O'Holleran K. High speed structured illumination microscopy in optically thick samples[J]. Methods, 88, 11-19(2015). http://europepmc.org/abstract/MED/25839410

[68] Mudry E, Belkebir K, Girard J et al. Structured illumination microscopy using unknown speckle patterns[J]. Nature Photonics, 6, 312-315(2012). http://www.nature.com/articles/nphoton.2012.83

[69] Ayuk R, Giovannini H, Jost A et al. Structured illumination fluorescence microscopy with distorted excitations using a filtered blind-SIM algorithm[J]. Optics Letters, 38, 4723-4726(2013).

[70] Chu K. McMillan P J, Smith Z J, et al. Image reconstruction for structured-illumination microscopy with low signal level[J]. Optics Express, 22, 8687-8702(2014).

[71] Dong S, Liao J, Guo K et al. Resolution doubling with a reduced number of image acquisitions[J]. Biomedical Optics Express, 6, 2946-2952(2015). http://www.osapublishing.org/boe/abstract.cfm?uri=boe-6-8-2946

[72] Markwirth A, Lachetta M, Mönkemöller V et al. Video-rate multi-color structured illumination microscopy with simultaneous real-time reconstruction[J]. Nature Communications, 10, 4315(2019). http://www.nature.com/articles/s41467-019-12165-x

[73] Ball G, Demmerle J, Kaufmann R et al. SIMcheck: a toolbox for successful super-resolution structured illumination microscopy[J]. Scientific Reports, 5, 15915(2015). http://www.onacademic.com/detail/journal_1000039698806110_2890.html

[74] Müller M, Mönkemöller V, Hennig S et al. Open-source image reconstruction of super-resolution structured illumination microscopy data in Image[J]. Nature Communications, 7, 10980(2016). http://www.mendeley.com/research/opensource-image-reconstruction-superresolution-structured-illumination-microscopy-data-imagej-tl-7/

[75] York A G, Parekh S H, Dalle Nogare D et al. Resolution doubling in live, multicellular organisms via multifocal structured illumination microscopy[J]. Nature Methods, 9, 749-754(2012). http://europepmc.org/articles/PMC3462167

[76] Wang H D, Rivenson Y, Jin Y Y et al. Deep-learning enables cross-modality super-resolution in fluorescence microscopy[J]. Proceedings of SPIE, 1093, 109370G(2019). http://www.researchgate.net/publication/331512035_Deep-learning_enables_cross-modality_super-resolution_in_fluorescence_microscopy_Conference_Presentation

[77] Xiao L, Wei Z H, Shao W Z[M]. Image priors modeling based super-resolution theory and algorithms: variational PDE, sparse regularization and Bayesian method(2017).

[78] Wang Z, Bovik A C, Sheikh H R et al. Image quality assessment: from error visibility to structural similarity[J]. IEEE Transactions on Image Processing, 13, 600-612(2004).