• Laser & Optoelectronics Progress
  • Vol. 60, Issue 16, 1615009 (2023)
Guangzhao Zhu, Bo Wei*, Afeng Yang, and Xin Xu
Author Affiliations
  • School of Communication Engineering, Hangzhou Dianzi University, Hangzhou 310037, Zhejiang, China
  • show less
    DOI: 10.3788/LOP222692 Cite this Article Set citation alerts
    Guangzhao Zhu, Bo Wei, Afeng Yang, Xin Xu. Multi-View 3D Reconstruction Method Based on Self-Attention Mechanism[J]. Laser & Optoelectronics Progress, 2023, 60(16): 1615009 Copy Citation Text show less

    Abstract

    Multi-view stereo matching is a major hotspot in the field of computer vision. We propose a self-attention-based deep learning network (SA-PatchmatchNet) to address the issues of poor completeness of multi-view stereo reconstruction, inability to process high-resolution images, huge GPU memory consumption, and long running time. First, the feature extraction module extracted the image features and sent them to the learnable Patchmatch module to obtain the depth map, and then the depth map was optimized to generate the final depth map. Moreover, the self-attention mechanism was integrated into the feature extraction module to capture the important information in the deep reasoning task, thereby enhancing the network feature extraction ability. The experimental results show that the reconstruction completeness is improved by 5.8% and the entirety is improved by 2.3% compared with that of the PatchmatchNet when the SA-PatchmatchNet is tested on the Technical University of Denmark (DTU) dataset. The completeness and entirety of the proposed network are significantly improved compared with that of the other state-of-the-art (SOTA) methods.
    Guangzhao Zhu, Bo Wei, Afeng Yang, Xin Xu. Multi-View 3D Reconstruction Method Based on Self-Attention Mechanism[J]. Laser & Optoelectronics Progress, 2023, 60(16): 1615009
    Download Citation