Author Affiliations
1Yunnan Water Resources and Hydropower Survey and Design Institute Co., Ltd., Kunming 650032, Yunnan , China2Yunnan Institute of Water & Hydropower Engineering Investigation, Design and Research, Kunming 650032, Yunnan , Chinashow less
Fig. 1. Network framework
Fig. 2. Structure of MCFF
Fig. 3. Structure of CGFI
Fig. 4. MUUFL dataset. (a) Hyperspectral image; (b) LiDAR image; (c) ground-truth map
Fig. 5. Houston 2018 dataset. (a) Hyperspectral image; (b) LiDAR image; (c) ground-truth map
Fig. 6. Trento dataset. (a) Hyperspectral image; (b) LiDAR image; (c) ground-truth map
Fig. 7. Classification results of different models on the MUUFL dataset. (a) Context CNN; (b) CRNN; (c) ViT; (d) SpectralFormer; (e) Two-Branch CNN; (f) Coupled CNN; (g) FusAtNet; (h) CFCGNet
Fig. 8. Classification results of different models on the Houston 2018 dataset. (a) Context CNN; (b) CRNN; (c) ViT; (d) SpectralFormer; (e) Two-Branch CNN; (f) Coupled CNN; (g) FusAtNet; (h) CFCGNet
Fig. 9. Classification results of different models on the Trento dataset. (a) Context CNN; (b) CRNN; (c) ViT; (d) SpectralFormer; (e) Two-Branch CNN; (f) Coupled CNN; (g) FusAtNet; (h) CFCGNet
| Class | Two-Branch CNN | Coupled CNN | Context CNN | CRNN | FusAtNet | ViT | SpectralFormer | CFCGNet |
|---|
| trees | 92.78 | 98.50 | 91.43 | 87.25 | 93.62 | 85.69 | 90.26 | 94.62 | | mostly grass | 59.74 | 78.66 | 63.76 | 85.36 | 84.64 | 81.56 | 75.46 | 88.54 | | mixed ground surface | 94.15 | 90.29 | 81.62 | 90.21 | 87.14 | 73.87 | 78.84 | 92.84 | | dirt and sand | 93.12 | 90.05 | 93.24 | 94.67 | 92.78 | 86.03 | 86.87 | 96.08 | | road | 92.46 | 96.83 | 89.06 | 84.51 | 86.13 | 86.31 | 89.13 | 92.33 | | water | 98.02 | 75.09 | 92.69 | 81.48 | 83.54 | 93.07 | 98.45 | 99.04 | | building shadow | 95.28 | 73.78 | 84.31 | 93.59 | 89.91 | 83.82 | 88.19 | 92.21 | | building | 94.42 | 96.81 | 81.72 | 90.22 | 95.28 | 83.74 | 77.82 | 92.47 | | sidewalk | 86.53 | 64.51 | 81.53 | 87.23 | 85.33 | 71.13 | 74.33 | 84.13 | | yellow curb | 99.37 | 19.75 | 99.03 | 93.28 | 91.80 | 100.00 | 94.08 | 100.00 | | cloth panels | 96.81 | 62.36 | 98.67 | 99.36 | 98.26 | 99.63 | 97.60 | 99.23 | | OA | 90.52 | 92.62 | 86.13 | 88.03 | 90.87 | 83.44 | 85.54 | 93.13 | | AA | 91.15 | 76.97 | 87.01 | 89.74 | 89.86 | 85.90 | 86.46 | 93.71 | | Kappa | 84.72 | 88.39 | 82.15 | 83.59 | 86.08 | 76.24 | 78.32 | 89.34 |
|
Table 1. Classification evaluation metrics of different models on the MUUFL dataset
| Class | Two-Branch CNN | Coupled CNN | Context CNN | CRNN | FusAtNet | ViT | SpectralFormer | CFCGNet |
|---|
| healthy grass | 93.41 | 91.05 | 94.72 | 79.02 | 77.98 | 75.54 | 85.64 | 76.04 | | stressed grass | 90.96 | 91.12 | 90.20 | 93.61 | 93.95 | 92.75 | 86.12 | 96.75 | | artificial turf | 99.15 | 99.68 | 99.23 | 98.49 | 99.54 | 76.02 | 99.31 | 98.86 | | evergreen trees | 94.53 | 95.41 | 93.38 | 96.54 | 93.85 | 97.13 | 96.89 | 97.37 | | deciduous trees | 95.47 | 95.97 | 85.96 | 86.59 | 88.21 | 81.76 | 85.83 | 94.02 | | bare earth | 99.54 | 99.08 | 99.23 | 99.63 | 100.00 | 97.88 | 89.57 | 99.91 | | water | 100.00 | 99.10 | 98.35 | 67.05 | 75.85 | 100.00 | 92.28 | 98.95 | | residential buildings | 83.86 | 88.44 | 85.81 | 93.96 | 94.81 | 88.79 | 88.73 | 98.31 | | non-residential buildings | 92.77 | 93.07 | 95.64 | 98.78 | 98.55 | 98.63 | 97.45 | 98.21 | | roads | 72.60 | 85.21 | 61.61 | 78.74 | 80.79 | 83.05 | 70.87 | 86.23 | | sidewalks | 63.39 | 58.48 | 61.40 | 73.04 | 77.38 | 74.84 | 73.20 | 80.95 | | crosswalks | 67.92 | 65.84 | 70.08 | 31.85 | 21.68 | 27.90 | 18.28 | 59.71 | | major thoroughfares | 70.19 | 65.36 | 69.39 | 88.30 | 86.84 | 83.65 | 82.99 | 97.50 | | highways | 91.85 | 93.26 | 85.11 | 90.02 | 89.31 | 84.74 | 82.27 | 89.27 | | railways | 96.47 | 99.58 | 96.48 | 100.00 | 99.15 | 99.31 | 99.03 | 98.60 | | paved parking lots | 93.17 | 89.45 | 88.30 | 95.96 | 94.99 | 93.46 | 88.54 | 97.50 | | unpaved parking lots | 98.83 | 99.91 | 99.08 | 55.63 | 46.52 | 23.60 | 64.09 | 72.27 | | cars | 94.52 | 95.53 | 93.52 | 81.95 | 85.29 | 86.83 | 85.82 | 93.97 | | trains | 96.58 | 97.76 | 96.41 | 95.15 | 95.17 | 95.74 | 95.16 | 98.83 | | stadium seats | 99.80 | 98.08 | 97.36 | 98.97 | 99.63 | 98.61 | 97.80 | 99.48 | | OA | 86.34 | 87.18 | 86.14 | 92.29 | 92.53 | 91.60 | 89.34 | 95.73 | | AA | 89.75 | 90.07 | 88.06 | 85.16 | 84.97 | 83.01 | 84.00 | 91.64 | | Kappa | 87.31 | 89.61 | 86.89 | 90.42 | 89.90 | 87.95 | 85.57 | 92.68 |
|
Table 2. Classification evaluation metrics of different models on the Houston 2018 dataset
| Class | Two-Branch CNN | Coupled CNN | Context CNN | CRNN | FusAtNet | ViT | SpectralFormer | CFCGNet |
|---|
| apple trees | 99.21 | 99.05 | 96.45 | 94.30 | 99.67 | 93.91 | 94.86 | 99.10 | | buildings | 97.12 | 97.77 | 94.63 | 93.34 | 96.83 | 94.81 | 93.44 | 96.77 | | ground | 83.54 | 82.84 | 80.29 | 79.53 | 79.01 | 82.43 | 80.03 | 88.27 | | wood | 98.99 | 99.27 | 96.93 | 95.73 | 99.74 | 97.35 | 96.05 | 99.80 | | vineyard | 93.96 | 93.00 | 93.61 | 93.81 | 98.92 | 99.00 | 95.86 | 99.77 | | roads | 88.52 | 88.62 | 85.71 | 87.87 | 89.75 | 76.41 | 80.74 | 90.65 | | OA | 95.74 | 95.54 | 94.05 | 93.58 | 97.81 | 94.74 | 93.70 | 98.26 | | AA | 93.56 | 93.43 | 91.27 | 90.76 | 93.99 | 90.65 | 90.16 | 95.73 | | Kappa | 93.35 | 93.22 | 93.54 | 92.45 | 95.57 | 92.20 | 89.46 | 97.41 |
|
Table 3. Classification evaluation metrics of different models on the Trento dataset
| Dataset | Complexity | Two-Branch CNN | Coupled CNN | Context CNN | CRNN | FusAtNet | ViT | SpectralFormer | CFCGNet |
|---|
| Houston | Training time | 727.39 | 184.43 | 2198.58 | 289.62 | 1329.44 | 309.49 | 399.01 | 1162.37 | | Testing time | 15.24 | 1.37 | 9.02 | 1.41 | 5.15 | 1.76 | 2.01 | 4.45 | | MUUFL | Training time | 338.32 | 88.67 | 972.82 | 135.97 | 773.52 | 155.44 | 175.74 | 589.76 | | Testing time | 7.04 | 0.76 | 3.97 | 0.70 | 3.60 | 4.94 | 6.31 | 2.94 | | Trento | Training time | 549.03 | 117.29 | 1947.73 | 217.76 | 394.22 | 76.2 | 95.81 | 312.40 | | Testing time | 17.21 | 1.65 | 8.70 | 1.06 | 12.44 | 4.85 | 3.74 | 9.47 |
|
Table 4. The computational cost of different models on various datasets
| Model | MUUFL | Honston 2018 | Trento |
|---|
| MCFF_base | MCFF | CGFI | OA | AA | Kappa | OA | AA | Kappa | OA | AA | Kappa |
|---|
| × | × | × | 86.51 | 87.29 | 83.01 | 88.65 | 86.73 | 87.53 | 95.74 | 91.18 | 93.04 | | × | √ | × | 90.44 | 91.18 | 86.85 | 92.57 | 88.09 | 89.97 | 96.93 | 93.71 | 95.20 | | × | × | √ | 89.15 | 90.68 | 86.21 | 91.84 | 88.81 | 90.16 | 96.42 | 94.86 | 96.07 | | √ | × | √ | 90.59 | 92.36 | 87.73 | 92.40 | 89.92 | 90.70 | 97.39 | 95.35 | 96.92 | | × | √ | √ | 92.13 | 93.77 | 89.04 | 94.77 | 91.14 | 91.68 | 98.26 | 95.73 | 97.41 |
|
Table 5. Experimental results of ablation of model structure
| Model | MUUFL | Honston 2018 | Trento |
|---|
| | OA | AA | Kappa | OA | AA | Kappa | OA | AA | Kappa | | √ | × | 89.04 | 90.18 | 86.74 | 91.53 | 87.73 | 88.21 | 96.44 | 94.06 | 95.10 | | × | √ | 92.13 | 93.77 | 89.04 | 94.77 | 91.14 | 91.68 | 98.26 | 95.73 | 97.41 |
|
Table 6. Experimental results of ablation of loss function
