Dynamic Three-Dimensional Reconstruction of Soft Tissue in Neural Radiation Field for Robotic Surgery Simulators

Qi Chen; Zhibao Qin; Xiaoyu Cai; Shijie Li; Zijun Wang; Junsheng Shi; Yonghang Tai

doi:10.3788/AOS231537

[1] Widya A R, Monno Y, Okutomi M et al. Whole stomach 3D reconstruction and frame localization from monocular endoscope video[J]. IEEE Journal of Translational Engineering in Health and Medicine, 7, 3300310(2019).

[2] Agarwal S, Snavely N, Simon I et al. Building Rome in a day[C], 72-79(2009).

[3] Sun H Y, Wang S L, Bai J X et al. Confocal laser scanning and 3D reconstruction methods for the subsurface damage of polished optics[J]. Optics and Lasers in Engineering, 136, 106315(2021).

[4] Chen L, Tang W, John N W et al. SLAM-based dense surface reconstruction in monocular Minimally Invasive Surgery and its application to Augmented Reality[J]. Computer Methods and Programs in Biomedicine, 158, 135-146(2018).

[5] Cox I J, Hingorani S L, Rao S B et al. A maximum likelihood stereo algorithm[J]. Computer Vision and Image Understanding, 63, 542-567(1996).

[6] Mildenhall B, Srinivasan P P, Tancik M et al. NeRF: representing scenes as neural radiance fields for view synthesis[J]. Communications of the ACM, 65, 99-106(2021).

[7] Tong Z, Gao J, Zhang H T. Recognition, location, measurement, and 3D reconstruction of concealed cracks using convolutional neural networks[J]. Construction and Building Materials, 146, 775-787(2017).

[8] Hoang D C, Lilienthal A J, Stoyanov T. Object-RPE: dense 3D reconstruction and pose estimation with convolutional neural networks[J]. Robotics and Autonomous Systems, 133, 103632(2020).

[9] Sanz-Requena R, Moratal D, García-Sánchez D R et al. Automatic segmentation and 3D reconstruction of intravascular ultrasound images for a fast preliminar evaluation of vessel pathologies[J]. Computerized Medical Imaging and Graphics, 31, 71-80(2007).

[10] Shen Q H, Yang X Y, Wang X C. Anything-3D: towards single-view anything reconstruction in the wild[EB/OL]. http:∥arxiv.org/abs/2304.10261

[11] Li J N, Li D X, Xiong C M et al. BLIP: bootstrapping language-image pre-training for unified vision-language understanding and generation[EB/OL]. https:∥arxiv.org/abs/2201.12086

[12] Mazurowski M A, Dong H Y, Gu H X et al. Segment anything model for medical image analysis: an experimental study[J]. Medical Image Analysis, 89, 102918(2023).

[13] Wiesent K, Barth K, Navab N et al. Enhanced 3-D-reconstruction algorithm for C-arm systems suitable for interventional procedures[J]. IEEE Transactions on Medical Imaging, 19, 391-403(2000).

[14] González Izard S, Sánchez Torres R, Alonso Plaza Ó et al. Nextmed: automatic imaging segmentation, 3D reconstruction, and 3D model visualization platform using augmented and virtual reality[J]. Sensors, 20, 2962(2020).

[15] Zhang X Y, Zhu J. Research progress and applications of endoscopic optical coherence tomography[J]. Chinese Journal of Lasers, 50, 2107103(2023).

[16] Dong X W, Xu Y, Gao C et al. Calculation of light source corrected color temperature based on cone fundamental space[J]. Acta Optica Sinica, 42, 0233001(2022).

[17] Pertsch T, Xiao S M, Majumdar A et al. Optical metasurfaces: fundamentals and applications[J]. Photonics Research, 11, OMFA1-OMFA3(2023).

[18] Yin Y K, Yu K, Yu C Z et al. 3D imaging using geometric light field: a review[J]. Chinese Journal of Lasers, 48, 1209001(2021).

[19] Saracino A, Deguet A, Staderini F et al. Haptic feedback in the da Vinci Research Kit (dVRK): a user study based on grasping, palpation, and incision tasks[J]. The International Journal of Medical Robotics + Computer Assisted Surgery: MRCAS, 15, e1999(2019).

[20] Wang Y H, Long Y H, Fan S H et al. Neural rendering for stereo 3D reconstruction of deformable tissues in robotic surgery[M]. Medical image computing and computer-assisted intervention-MICCAI 2022, 13437, 431-441(2022).

[21] Müller T, Evans A, Schied C et al. Instant neural graphics primitives with a multiresolution hash encoding[J]. ACM Transactions on Graphics, 41, 1-15(2022).

[22] Chen A P, Xu Z X, Geiger A et al. TensoRF: tensorial radiance fields[M]. Computer vision-ECCV 2022, 13692, 333-350(2022).

[23] Pupezescu M C, Pupezescu V. Instant neural graphics primitives framework and its emerging applications in digitalization of dental care[C](2022).

[24] Zhao B Y, Mertz J. Resolution enhancement with deblurring by pixel reassignment[J]. Advanced Photonics, 5, 066004(2023).

[25] Jain A, Tancik M, Abbeel P. Putting NeRF on a Diet: semantically consistent few-shot view synthesis[C], 5865-5874(2021).

[26] Liu J H, Yang Y Q, Ma R et al. Research progress of organic NIR-Ⅱ fluorescent probes[J]. Chinese Journal of Lasers, 50, 2107101(2023).

[27] Li T N, Liu D Y, Zhang P R et al. Deep convolutional encoder-decoder neural network approach for functional near infrared spectroscopic imaging[J]. Chinese Journal of Lasers, 50, 2107107(2023).

[28] Tu R C, Mao X L, Ma Bet al. Deep cross-modal hashing with hashing functions and unified hash codes jointly learning[J]. IEEE Transactions on Knowledge and Data Engineering, 34, 560-572(2022).