• Laser & Optoelectronics Progress
  • Vol. 59, Issue 22, 2230001 (2022)
Yunle Ding1, Huiqin Wang1、*, Ke Wang1, Zhan Wang2, and Gang Zhen2
Author Affiliations
  • 1School of Information and Control Engineering, Xi'an University of Architecture and Technology, Xi'an 710055, Shaanxi, China
  • 2Shaanxi Provincial Institute of Cultural Relics Protection, Xi'an 710075, Shaanxi, China
  • show less
    DOI: 10.3788/LOP202259.2230001 Cite this Article Set citation alerts
    Yunle Ding, Huiqin Wang, Ke Wang, Zhan Wang, Gang Zhen. Three dimensional-CNN Classification Method of Mural Multispectral Image Pigments Based on Multiscale Feature Fusion[J]. Laser & Optoelectronics Progress, 2022, 59(22): 2230001 Copy Citation Text show less
    Schematic diagrams of 2D-CNN and 3D-CNN. (a) 2D-CNN; (b) 3D-CNN
    Fig. 1. Schematic diagrams of 2D-CNN and 3D-CNN. (a) 2D-CNN; (b) 3D-CNN
    Residual learning
    Fig. 2. Residual learning
    Schematic diagrams of atrous convolution kernel. (a) r=1; (b) r=2; (c) r=3
    Fig. 3. Schematic diagrams of atrous convolution kernel. (a) r=1; (b) r=2; (c) r=3
    Sserial hole convolution module
    Fig. 4. Sserial hole convolution module
    Multiscale feature fusion module
    Fig. 5. Multiscale feature fusion module
    Schematic of MFAC-Res3D-CNN
    Fig. 6. Schematic of MFAC-Res3D-CNN
    Multispectral image acquisition system of murals
    Fig. 7. Multispectral image acquisition system of murals
    Multispectral images of murals in each band
    Fig. 8. Multispectral images of murals in each band
    Simulated mural. (a) Simulated mural image; (b) pseudo color image; (c) truth image
    Fig. 9. Simulated mural. (a) Simulated mural image; (b) pseudo color image; (c) truth image
    Comparison of different network classification results. (a) Truth image; (b) SVM; (c) 2D-CNN; (d) Res-3D-CNN; (e) MFAC-Res3D-CNN
    Fig. 10. Comparison of different network classification results. (a) Truth image; (b) SVM; (c) 2D-CNN; (d) Res-3D-CNN; (e) MFAC-Res3D-CNN
    Comparison of different network classification details. (a) Truth image; (b) SVM;(c) 2D-CNN; (d) Res-3D-CNN; (e) MFAC-Res3D-CNN
    Fig. 11. Comparison of different network classification details. (a) Truth image; (b) SVM;(c) 2D-CNN; (d) Res-3D-CNN; (e) MFAC-Res3D-CNN
    NumberCategoryColorNumber of samples
    0Background871622
    1Mercuric sulfide57778
    2Mineral green49641
    3Chrome yellow294816
    4Graphite58765
    5Lazurite77061
    6Minium1417
    Table 1. Sample of multispectral image dataset of simulated murals
    CategoryBackgroundMercuric sulfideMineral greenChrome yellowGraphiteLazuriteMinium
    Background865044103616772548108221520
    Mercuric sulfide573571662217000
    Mineral green1677100478595000
    Chrome yellow2692911292031010
    Graphite2049071365657300
    Lazurite14520225218755170
    Minium135001001281
    Table 2. Classification confusion matrix of article model
    CategorySVM2D-CNNRes-3D-CNNMFAC-Res3D-CNN
    Background84.3697.4997.3499.24
    Mercuric sulfide92.3691.0595.4398.94
    Mineral green78.5392.3494.2596.41
    Chrome yellow88.5399.3297.5099.05
    Graphite80.3199.8894.6496.26
    Lazurite86.7284.9097.0197.99
    Minium63.7882.6389.8790.40
    OA84.7291.4597.5798.87
    AA82.0892.5195.1496.89
    Kappa78.6089.9895.4198.04
    Table 3. Comparison of dataset classification accuracy results
    TimeSVM2D-CNNRes-3D-CNNMFAC-Res3D-CNN
    Train678.51037.81265.61301.5
    Test7.9810.7515.9517.05
    Table 4. Comparison of training and testing time of different algorithms
    Yunle Ding, Huiqin Wang, Ke Wang, Zhan Wang, Gang Zhen. Three dimensional-CNN Classification Method of Mural Multispectral Image Pigments Based on Multiscale Feature Fusion[J]. Laser & Optoelectronics Progress, 2022, 59(22): 2230001
    Download Citation