• Photonics Research
  • Vol. 12, Issue 1, 7 (2024)
Ze-Hao Wang1、2、†, Long-Kun Shan1、2、†, Tong-Tian Weng1、2, Tian-Long Chen3, Xiang-Dong Chen1、2、4, Zhang-Yang Wang3, Guang-Can Guo1、2、4, and Fang-Wen Sun1、2、4、*
Author Affiliations
  • 1CAS Key Laboratory of Quantum Information, University of Science and Technology of China, Hefei 230026, China
  • 2CAS Center for Excellence in Quantum Information and Quantum Physics, University of Science and Technology of China, Hefei 230026, China
  • 3University of Texas at Austin, Austin, Texas 78705, USA
  • 4Hefei National Laboratory, University of Science and Technology of China, Hefei 230088, China
  • show less
    DOI: 10.1364/PRJ.488310 Cite this Article Set citation alerts
    Ze-Hao Wang, Long-Kun Shan, Tong-Tian Weng, Tian-Long Chen, Xiang-Dong Chen, Zhang-Yang Wang, Guang-Can Guo, Fang-Wen Sun. Learning the imaging mechanism directly from optical microscopy observations[J]. Photonics Research, 2024, 12(1): 7 Copy Citation Text show less

    Abstract

    The optical microscopy image plays an important role in scientific research through the direct visualization of the nanoworld, where the imaging mechanism is described as the convolution of the point spread function (PSF) and emitters. Based on a priori knowledge of the PSF or equivalent PSF, it is possible to achieve more precise exploration of the nanoworld. However, it is an outstanding challenge to directly extract the PSF from microscopy images. Here, with the help of self-supervised learning, we propose a physics-informed masked autoencoder (PiMAE) that enables a learnable estimation of the PSF and emitters directly from the raw microscopy images. We demonstrate our method in synthetic data and real-world experiments with significant accuracy and noise robustness. PiMAE outperforms DeepSTORM and the Richardson–Lucy algorithm in synthetic data tasks with an average improvement of 19.6% and 50.7% (35 tasks), respectively, as measured by the normalized root mean square error (NRMSE) metric. This is achieved without prior knowledge of the PSF, in contrast to the supervised approach used by DeepSTORM and the known PSF assumption in the Richardson–Lucy algorithm. Our method, PiMAE, provides a feasible scheme for achieving the hidden imaging mechanism in optical microscopy and has the potential to learn hidden mechanisms in many more systems.
    rawimage=noise(emittersPSF)+background,(A1)

    View in Article

    Centerdistanceloss=|i,jIntensityij·Coordinateiji,jIntensityijCenterposition|.(B1)

    View in Article

    TV loss=i,j(Intensityi,j1Intensityi,j)2+(Intensityi+1,jIntensityi,j)2.(B2)

    View in Article

    Lossfunction=α1L1+α2MSSSIM+α3Centerdistance+α4TV,(B3)

    View in Article

    h(x,y,z;λ)=C00αcosθJ0(kρsinθ)eikzcosθsinθdθ,(D1)

    View in Article

    PSF(x,y,z)=|h(x,y,z;λem)|2.(D2)

    View in Article

    unmHG(x,y,z)=CnmHG(1/w)exp[ik(x2+y2/2R)]×exp[(x2+y2/w2)]exp[i(n+m+1)ψ]×Hn(x2/w)Hm(y2/w),(D3)

    View in Article

    unmLG(r,ϕ,z)=CnmLG(1/w)exp(ikr2/2R)exp(r2/w2)×exp[i(n+m+1)ψ]exp[i(nm)ϕ]×(1)min(n,m)(r2/w)|rm|×Lmin(n,m)|nm|(2r2/w2),(D4)

    View in Article

    NRMSE=i,j(ImagetrueImagetest)2Max(Imagetrue)Min(Imagetrue).(F1)

    View in Article

    MS-SSIM(x,y)=[lm(x,y)]αm·j=1M[cj(x,y)]βj[sj(x,y)]γj,(F2)

    View in Article

    l(x,y)=2μxμy+C1μx2+μy2+C1,(F3)

    View in Article

    c(x,y)=2σxσy+C2σx2+σy2+C2,(F4)

    View in Article

    s(x,y)=σxy+C3σxσy+C3,(F5)

    View in Article

    xnorm=xxminxmaxxmin,(F6)

    View in Article

    p(z)=1σ2exp(zσ2).(I1)

    View in Article

    Ze-Hao Wang, Long-Kun Shan, Tong-Tian Weng, Tian-Long Chen, Xiang-Dong Chen, Zhang-Yang Wang, Guang-Can Guo, Fang-Wen Sun. Learning the imaging mechanism directly from optical microscopy observations[J]. Photonics Research, 2024, 12(1): 7
    Download Citation