Deep learning-based optical aberration estimation enables offline digital adaptive optics and super-resolution imaging

Chang Qiao; Haoyu Chen; Run Wang; Tao Jiang; Yuwang Wang; Dong Li

doi:10.1364/PRJ.506778

[1] N. Ji. Adaptive optical fluorescence microscopy. Nat. Methods, 14, 374-380(2017).

[2] Q. Zhang, Q. Hu, C. Berlage. Adaptive optics for optical microscopy [Invited]. Biomed. Opt. Express, 14, 1732-1756(2023).

[3] K. M. Hampson, R. Turcotte, D. T. Miller. Adaptive optics for high-resolution imaging. Nat. Rev. Methods Primers, 1, 68(2021).

[4] D. Saha, U. Schmidt, Q. Zhang. Practical sensorless aberration estimation for 3D microscopy with deep learning. Opt. Express, 28, 29044-29053(2020).

[5] B. P. Cumming, M. Gu. Direct determination of aberration functions in microscopy by an artificial neural network. Opt. Express, 28, 14511-14521(2020).

[6] P. Zhang, S. Liu, A. Chaurasia. Analyzing complex single-molecule emission patterns with deep learning. Nat. Methods, 15, 913-916(2018).

[7] J. Wu, Y. Guo, C. Deng. An integrated imaging sensor for aberration-corrected 3D photography. Nature, 612, 62-71(2022).

[8] J. Wu, Z. Lu, D. Jiang. Iterative tomography with digital adaptive optics permits hour-long intravital observation of 3D subcellular dynamics at millisecond scale. Cell, 184, 3318-3332(2021).

[9] C. Qiao, D. Li, Y. Guo. Evaluation and development of deep neural networks for image super-resolution in optical microscopy. Nat. Methods, 18, 194-202(2021).

[10] Y. Zhang, K. P. Li, K. Li. Image super-resolution using very deep residual channel attention networks. Proceedings of the European Conference on Computer Vision (ECCV), 294-310(2018).

[11] O. Ronneberger, P. Fischer, T. Brox. U-net: convolutional networks for biomedical image segmentation. International Conference on Medical Image Computing and Computer-Assisted Intervention, 1-8(2015).

[12] J. Gu, H. Lu, W. Zuo. Blind super-resolution with iterative kernel correction. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 1604-1613(2019).

[13] D. Hendrycks, K. Gimpel. Gaussian error linear units (GELUs). arXiv(2016).

[14] J. Caballero, C. Ledig, A. Aitken. Real-time video super-resolution with spatio-temporal networks and motion compensation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 4778-4787(2017).

[15] S. Bell-Kligler, A. Shocher, M. Irani. Blind super-resolution kernel estimation using an internal-GaN. arXiv(2019).

[16] A. Liu, Y. Liu, J. Gu. Blind image super-resolution: a survey and beyond. arXiv(2021).

[17] J. Liang, G. Sun, K. Zhang. Mutual affine network for spatially variant kernel estimation in blind image super-resolution. Proceedings of the IEEE/CVF International Conference on Computer Vision, 4096-4105(2021).

[18] A. Shocher, S. Bagon, P. Isola. InGaN: capturing and remapping the ‘DNA’ of a natural image. arXiv(2018).

[19] D. Ren, K. Zhang, Q. Wang. Neural blind deconvolution using deep priors. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 3341-3350(2020).

[20] J. Liang, K. Zhang, S. Gu. Flow-based kernel prior with application to blind super-resolution. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 10601-10610(2021).

[21] L. B. Lucy. An iterative technique for the rectification of observed distributions. Astron. J., 79, 745(1974).

[22] W. H. Richardson. Bayesian-based iterative method of image restoration. J. Opt. Soc. Am., 62, 55-59(1972).

[23] Z. Liu, Y. Lin, Y. Cao. Swin transformer: hierarchical vision transformer using shifted windows. Proceedings of the IEEE/CVF International Conference on Computer Vision, 10012-10022(2021).

[24] https://github.com/HypnosRin/SFE-Net. https://github.com/HypnosRin/SFE-Net