Researching | Physics-constrained deep-inverse point spread function model: toward non-line-of-sight imaging reconstruction

Journals >Advanced Photonics Nexus >Volume 3 >Issue 2 >Page 026010 > Article

Advanced Photonics Nexus
Vol. 3, Issue 2, 026010 (2024)

Physics-constrained deep-inverse point spread function model: toward non-line-of-sight imaging reconstruction

Su Wu^1、†, Chan Huang², Jing Lin³, Tao Wang^1、4, Shanshan Zheng^1、4, Haisheng Feng^1、4, and Lei Yu^1、*

Author Affiliations

¹Chinese Academy of Sciences, Anhui Institute of Optics and Fine Mechanics, Hefei, China

²Hefei University of Technology, School of Physics, Department of Optical Engineering, Hefei, China

³Hefei Normal University, Department of Chemical and Chemical Engineering, Hefei, China

⁴University of Science and Technology of China, Science Island Branch of Graduate School, Hefei, China

show less

DOI: 10.1117/1.APN.3.2.026010 Cite this Article Set citation alerts

Su Wu, Chan Huang, Jing Lin, Tao Wang, Shanshan Zheng, Haisheng Feng, Lei Yu. Physics-constrained deep-inverse point spread function model: toward non-line-of-sight imaging reconstruction[J]. Advanced Photonics Nexus, 2024, 3(2): 026010 Copy Citation Text

show less

References

[1] X. Liu et al. Non-line-of-sight imaging using phasor-field virtual wave optics. Nature, 572, 620-623(2019).

[2] B. David et al. Wave-based non-line-of-sight imaging using fast f-k migration. ACM Trans. Graph., 38, 116(2019).

[3] T. Maeda et al. Thermal non-line-of-sight imaging, 1-11(2019).

[4] Y. Altmann et al. Quantum-inspired computational imaging. Science, 361, eaat2298(2018).

[5] W. Yang et al. None-line-of-sight imaging enhanced with spatial multiplexing. Opt. Express, 30, 5855-5867(2022).

[6] J. H. Nam et al. Low-latency time-of-flight non-line-of-sight imaging at 5 frames per second. Nat. Commun., 12, 6526(2021).

[7] F. Heide et al. Non-line-of-sight imaging with partial occluders and surface normals. ACM Trans. Graph., 38, 1-10(2019).

[8] C. Saunders, J. Murray-Bruce, V. K. Goyal. Computational periscopy with an ordinary digital camera. Nature, 565, 472(2019).

[9] J. Liu et al. Photon-efficient non-line-of-sight imaging. IEEE Trans. Comput. Imaging, 8, 639-650(2022).

[10] D. Faccio, A. Velten, G. Wetzstein. Non-line-of-sight imaging. Nat. Rev. Phys., 2, 318-327(2020).

[11] F. Mu et al. Physics to the rescue: deep non-line-of-sight reconstruction for high-speed imaging. IEEE Trans. Pattern Anal. Mach. Intell., 1-12(2022).

[12] T. Maeda et al. Recent advances in imaging around corners(2019).

[13] C. A. Metzler, D. B. Lindell, G. Wetzstein. Keyhole imaging: non-line-of-sight imaging and tracking of moving objects along a single optical path. IEEE Trans. Comput. Imaging, 7, 1-12(2020).

[14] D. Zhu, W. Cai. Fast non-line-of-sight imaging with two-step deep remapping. ACS Photonics, 9, 2046-2055(2022).

[15] A. Kirmani et al. Looking around the corner using transient imaging, 159-166(2009).

[16] A. Velten et al. Recovering three-dimensional shape around a corner using ultrafast time-of-flight imaging. Nat. Commun., 3, 745(2012).

[17] M. O’Toole, D. B. Lindell, G. Wetzstein. Real-time non-line-of-sight imaging, 1-2(2018).

[18] M. O’Toole, D. B. Lindell, G. Wetzstein. Confocal non-line-of-sight imaging based on the light-cone transform. Nature, 555, 338-341(2018).

[19] S. I. Young et al. Non-line-of-sight surface reconstruction using the directional light-cone transform, 1407-1416(2020).

[20] X. Liu, S. Bauer, A. Velten. Phasor field diffraction based reconstruction for fast non-line-of-sight imaging systems. Nat. Commun., 11, 1645(2020).

[21] W. Chen et al. Steady-state non-line-of-sight imaging, 6790-6799(2019).

[22] C. A. Metzler et al. Deep-inverse correlography: towards real-time high-resolution non-line-of-sight imaging. Optica, 7, 63-71(2020).

[23] S. Tong, A. M. Alessio, P. E. Kinahan. Noise and signal properties in PSF-based fully 3D PET image reconstruction: an experimental evaluation. Phys. Med. Biol., 55, 1453(2010).

[24] J. D. Rego, K. Kulkarni, S. Jayasuriya. Robust lensless image reconstruction via PSF estimation, 403-412(2021).

[25] G. Demoment. Image reconstruction and restoration: overview of common estimation structures and problems. IEEE Trans. Acoust. Speech Signal Process., 37, 2024-2036(1989).

[26] S. Stute, C. Comtat. Practical considerations for image-based PSF and blobs reconstruction in PET. Phys. Med. Biol., 58, 3849(2013).

[27] A. K. Trull et al. Point spread function based image reconstruction in optical projection tomography. Phys. Med. Biol., 62, 7784(2017).

[28] B. Zhu et al. Image reconstruction by domain-transform manifold learning. Nature, 555, 487-492(2018).

[29] L. Song et al. High-accuracy image formation model for coded aperture snapshot spectral imaging. IEEE Trans. Comput. Imaging, 8, 188-200(2022).

[30] K. C. Lee et al. Design and single-shot fabrication of lensless cameras with arbitrary point spread functions. Optica, 10, 72(2023).

[31] H. J. Zhu, B. C. Han, B. Qiu. Survey of astronomical image processing methods. Lect. Notes Comput. Sci., 9219, 420-429(2015).

[32] J. Smith et al. Image-to-image translation for wavefront and PSF estimation. Proc. SPIE, 12185, 121852L(2022).

[33] S. Ludwig et al. Image reconstruction and enhancement by deconvolution in scatter-plate microscopy. Opt. Express, 27, 23049-23058(2019).

[34] X. Shao et al. Applications of both time of flight and point spread function in brain PET image reconstruction. Nucl. Med. Commun., 37, 422-427(2016).

[35] D. Kidera et al. The edge artifact in the point-spread function-based PET reconstruction at different sphere-to-background ratios of radioactivity. Ann. Nucl. Med., 30, 97-103(2016).

[36] X. Xie et al. Extended depth-resolved imaging through a thin scattering medium with PSF manipulation. Sci. Rep., 8, 4585(2018).

[37] C. Pei et al. Dynamic non-line-of-sight imaging system based on the optimization of point spread functions. Opt. Express, 29, 32349-32364(2021).

[38] L. B. Wolff, S. K. Nayar, M. Oren. Improved diffuse reflection models for computer vision. Int. J. Comput. Vis., 30, 55-71(1998).

[39] C. Tian et al. Coarse-to-fine CNN for image super-resolution. IEEE Trans. Multimedia, 23, 1489-1502(2020).

[40] J. Yamanaka, S. Kuwashima, T. Kurita. Fast and accurate image super resolution by deep CNN with skip connection and network in network. Lect. Notes Comput. Sci., 10635, 217-225(2017).

[41] Z. Wang et al. Multi-memory convolutional neural network for video super-resolution. IEEE Trans. Image Process., 28, 2530-2544(2018).

[42] R. Kuschmierz et al. Ultra-thin 3D lensless fiber endoscopy using diffractive optical elements and deep neural networks. Light Adv. Manuf., 2, 415-424(2021).

[43] H. Zhu et al. DNF: diffractive neural field for lensless microscopic imaging. Opt. Express, 30, 18168-18178(2022).

[44] Q. Li et al. Imaging reconstruction through strongly scattering media by using convolutional neural networks. Opt. Commun., 477, 126341(2020).

[45] Y. Sun et al. Image reconstruction through dynamic scattering media based on deep learning. Opt. Express, 27, 16032-16046(2019).

[46] P. Netrapalli, P. Jain, S. Sanghavi. Phase retrieval using alternating minimization, 2796-2804(2013).

[47] E. J. Candes, X. Li, M. Soltanolkotabi. Phase retrieval via Wirtinger flow: theory and algorithms. IEEE Trans. Inf. Theory, 61, 1985-2007(2015).

Figures&Tables (11)

References (47)

Copy Citation Text

Su Wu, Chan Huang, Jing Lin, Tao Wang, Shanshan Zheng, Haisheng Feng, Lei Yu. Physics-constrained deep-inverse point spread function model: toward non-line-of-sight imaging reconstruction[J]. Advanced Photonics Nexus, 2024, 3(2): 026010

Download Citation

Set citation alerts for the article

Set citation alerts for the article

Save the article for my favorites

Paper Information

Recommended Topics

laser devices and laser physics

Lasers and Laser Optics

laser manufacturing

Instrumentation, Measurement and Metrology

Set citation alerts for the article

Please enter your email address