• Advanced Photonics Nexus
  • Vol. 2, Issue 1, 016012 (2023)
Jianyong Wang1、2、†, Junchao Fan3, Bo Zhou1, Xiaoshuai Huang4、5、*, and Liangyi Chen1、6、7、8、*
Author Affiliations
  • 1Peking University, Institute of Molecular Medicine, College of Future Technology, Center for Life Sciences, State Key Laboratory of Membrane Biology, Beijing Key Laboratory of Cardiometabolic Molecular Medicine, Beijing, China
  • 2Peking University, School of Software and Microelectronics, Beijing, China
  • 3Chongqing University of Posts and Telecommunications, College of Computer Science and Technology, Chongqing Key Laboratory of Image Cognition, Chongqing, China
  • 4Peking University, Biomedical Engineering Department, Beijing, China
  • 5Peking University, International Cancer Institute, Beijing, China
  • 6PKU-IDG/McGovern Institute for Brain Research, Beijing, China
  • 7Beijing Academy of Artificial Intelligence, Beijing, China
  • 8National Biomedical Imaging Center, Beijing, China
  • show less
    DOI: 10.1117/1.APN.2.1.016012 Cite this Article Set citation alerts
    Jianyong Wang, Junchao Fan, Bo Zhou, Xiaoshuai Huang, Liangyi Chen. Hybrid reconstruction of the physical model with the deep learning that improves structured illumination microscopy[J]. Advanced Photonics Nexus, 2023, 2(1): 016012 Copy Citation Text show less

    Abstract

    Structured illumination microscopy (SIM) has been widely used in live-cell superresolution (SR) imaging. However, conventional physical model-based SIM SR reconstruction algorithms are prone to artifacts in handling raw images with low signal-to-noise ratios (SNRs). Deep-learning (DL)-based methods can address this challenge but may lead to degradation and hallucinations. By combining the physical inversion model with a total deep variation (TDV) regularization, we propose a hybrid restoration method (TDV-SIM) that outperforms conventional or DL methods in suppressing artifacts and hallucinations while maintaining resolutions. We demonstrate the performance superiority of TDV-SIM in restoring actin filaments, endoplasmic reticulum, and mitochondrial cristae from extremely low SNR raw images. Thus TDV-SIM represents the ideal method for prolonged live-cell SR imaging with minimal exposure and photodamage. Overall, TDV-SIM proves the power of integrating model-based reconstruction methods with DL ones, possibly leading to the rapid exploration of similar strategies in high-fidelity reconstructions of other microscopy methods.
    minfD(f,g)+λR(f),

    View in Article

    fk+1=fkηD(fk,g)ηλR(fk),

    View in Article

    lMSE(X,Y)=1H×Wi=1Hj=1W(Xi,jYi,j)2,

    View in Article

    lcombination(X,Y)=lMSE(X,Y)+k[1SSIM(X,Y)],

    View in Article

    Norm(X)=Xmin(X)max(X)min(X).

    View in Article

    PSNR(X,Y)=10×lg[MAXI2i=1Hj=1W(Xi,jYi,j)2/(H×W)],

    View in Article

    SSIM(X,Y)=(2μXμY+c1)(2σXY+c2)(μX2+μY2+c1)(σX2+σY2+c2),

    View in Article

    NRMSE(X,Y)=i=1Hj=1W(Xi,jYi,j)2/(H×W)max(Y)min(Y),

    View in Article

    Jianyong Wang, Junchao Fan, Bo Zhou, Xiaoshuai Huang, Liangyi Chen. Hybrid reconstruction of the physical model with the deep learning that improves structured illumination microscopy[J]. Advanced Photonics Nexus, 2023, 2(1): 016012
    Download Citation