• Optics and Precision Engineering
  • Vol. 32, Issue 7, 1087 (2024)
Haibin WU1, Shiyu DAI1, Aili WANG1,*, Iwahori YUJI2, and Xiaoyu YU3
Author Affiliations
  • 1Heilongjiang Province Key Laboratory of Laser Spectroscopy Technology and Application, College of Measurement and Control Technology and Communication Engineering, Harbin University of Science and Technology, Harbin50080, China
  • 2Department of Computer Science, Chubu University, Aichi487-8501, Japan
  • 3College of Electron and Information, University of Electronic Science and Technology of China,Zhongshan Institute, Zhongshan528400, China
  • show less
    DOI: 10.37188/OPE.20243207.1087 Cite this Article
    Haibin WU, Shiyu DAI, Aili WANG, Iwahori YUJI, Xiaoyu YU. Collaborative classification of hyperspectral and LiDAR data based on CNN-transformer[J]. Optics and Precision Engineering, 2024, 32(7): 1087 Copy Citation Text show less
    Model architecture of CLCT-Net
    Fig. 1. Model architecture of CLCT-Net
    Schematic diagram of shared feature extraction network
    Fig. 2. Schematic diagram of shared feature extraction network
    Schematic diagram of HSI encoder
    Fig. 3. Schematic diagram of HSI encoder
    Schematic diagram of LiDAR encoder
    Fig. 4. Schematic diagram of LiDAR encoder
    Pseudo color map and ground-truth map of Houston2013 dataset
    Fig. 5. Pseudo color map and ground-truth map of Houston2013 dataset
    Pseudo color map and ground-truth map of Trento dataset
    Fig. 6. Pseudo color map and ground-truth map of Trento dataset
    Feature visualizations of Houston2013 dataset
    Fig. 7. Feature visualizations of Houston2013 dataset
    Feature visualizations of Trento dataset
    Fig. 8. Feature visualizations of Trento dataset
    Classification results of different methods on Houston2013 dataset
    Fig. 9. Classification results of different methods on Houston2013 dataset
    Classification results of different methods on Trento dataset
    Fig. 10. Classification results of different methods on Trento dataset
    Class nameTrain numTest numColor
    Healthy grass1981 053
    Stressed grass1901 064
    Synthetic grass192505
    Trees1881 056
    Soil1861 056
    Water182143
    Residential1961 072
    Commercial1911 053
    Road1931 059
    Highway1911 036
    Railway1811 054
    Parking Lot11921 041
    Parking Lot2184285
    Tennis court181247
    Running track187473
    Total2 83212 197
    Table 1. Land class details in Houston2013 dataset
    Class nameTrain numTest numColor
    Apples1293 905
    Buildings152 778
    Ground105374
    Woods1548 969
    Wineyard18410 317
    Roads1223 052
    Total81929 395
    Table 2. Land class details in Trento dataset
    ClassTwo-BranchEndNetMDL-MiddleMAHiDFNetSpectrum-LiDARTB-HSICLCT-Net

    C1

    C2

    C3

    C4

    C5

    C6

    C7

    C8

    C9

    C10

    C11

    C12

    C13

    C14

    C15

    83.10

    84.10

    100.00

    93.09

    100.00

    99.30

    92.82

    82.34

    84.70

    65.44

    88.24

    89.53

    92.28

    96.76

    99.79

    81.58

    83.65

    100.00

    93.09

    99.91

    95.10

    82.65

    81.29

    88.29

    89.00

    83.78

    90.39

    82.46

    100.00

    98.10

    83.10

    85.06

    99.60

    91.57

    98.86

    100.00

    97.64

    88.13

    85.93

    74.42

    84.54

    95.39

    87.37

    95.14

    100.00

    98.53

    92.87

    91.11

    98.10

    98.38

    98.58

    99.15

    80.94

    98.04

    72.81

    72.71

    76.80

    95.80

    99.53

    100.53

    49.17

    35.93

    72.12

    65.08

    59.63

    24.54

    75.61

    75.23

    74.74

    62.37

    85.37

    50.13

    41.26

    28.46

    70.24

    100.0

    98.04

    30.64

    99.28

    99.62

    93.38

    85.66

    92.76

    94.46

    88.41

    96.16

    84.15

    96.30

    100.00

    90.27

    87.07

    98.05

    96.67

    94.78

    99.34

    77.14

    89.50

    83.31

    94.33

    92.84

    95.79

    86.26

    87.37

    100.00

    94.22

    OA87.9888.5289.5589.5860.8185.2292.01
    AA90.1189.9591.0591.3657.9989.9691.78
    K×10086.9887.5987.5988.7457.6784.0291.33
    Table 3. Comparison of classification accuracy of different methods on Houston2013 dataset
    ClassTwo-BranchEndNetMDL-MiddleMAHiDFNetSpectrum-LiDARTB-HSICLCT-Net

    C1

    C2

    C3

    C4

    C5

    C6

    99.78

    97.93

    99.93

    99.46

    98.96

    91.68

    88.19

    98.49

    95.19

    99.30

    91.96

    90.14

    99.50

    97.55

    99.10

    99.90

    99.71

    92.25

    99.91

    88.92

    97.53

    99.98

    99.90

    99.78

    74.00

    62.45

    26.00

    99.54

    98.45

    88.94

    99.19

    81.24

    63.46

    99.93

    97.35

    94.52

    99.30

    97.49

    96.45

    99.29

    99.70

    96.28

    OA98.3694.1798.7398.5984.9495.4298.90
    AA97.9693.8898.0097.5574.9089.2898.10
    K×10097.8392.2298.3298.1280.5693.8998.54
    Table 4. Comparison of classification accuracy of different methods on Trento dataset
    Method#param./MFLOPs/M
    Two-Branch1225
    EndNet0.070.49

    MDL-Middle

    MAHiDFNet

    0.25

    77

    4.7

    155

    CLCT-Net5.1384
    Table 5. FLOPs and parameters of different classification models
    Haibin WU, Shiyu DAI, Aili WANG, Iwahori YUJI, Xiaoyu YU. Collaborative classification of hyperspectral and LiDAR data based on CNN-transformer[J]. Optics and Precision Engineering, 2024, 32(7): 1087
    Download Citation