Researching | Spectral-spatial Attention Residual Networks for Hyperspectral Image Classification

Journals >Acta Photonica Sinica >Volume 52 >Issue 12 >Page 1210002 > Article

Acta Photonica Sinica
Vol. 52, Issue 12, 1210002 (2023)

Spectral-spatial Attention Residual Networks for Hyperspectral Image Classification

Feifei WANG^1,3, Huijie ZHAO^1,2,3, Na LI^1,2,3,*, Siyuan LI⁴, and Yu CAI⁵

Author Affiliations

¹Key Laboratory of Precision Opto-Mechatronics Technology，Ministry of Education，School of Instrumentation and Optoelectronic Engineering，Beihang University，Beijing 100191，China

²Institute of Artificial Intelligence，Beihang University，Beijing 100191，China

³Aerospace Optical-Microwave Integrated Precision Intelligent Sensing，Key Laboratory of Ministry of Industry and Information Technology，Beihang University，Beijing 100191，China

⁴Key Laboratory of Spectral Imaging Technology，Xi'an Institute of Optics and Precision Mechanics，Chinese Academy of Sciences，Xi'an 710119，China

⁵China Academy of Launch Vehicle Technology，Beijing 100076，China

show less

DOI: 10.3788/gzxb20235212.1210002 Cite this Article

Feifei WANG, Huijie ZHAO, Na LI, Siyuan LI, Yu CAI. Spectral-spatial Attention Residual Networks for Hyperspectral Image Classification[J]. Acta Photonica Sinica, 2023, 52(12): 1210002 Copy Citation Text

show less

Flow chart of spectral-spatial attention residual network

Fig. 1. Flow chart of spectral-spatial attention residual network

Download full size | View in the Article

The structure of the central region spectral attention mechanism

Fig. 2. The structure of the central region spectral attention mechanism

Download full size | View in the Article

Fig. 3. The structure of the spatial attention mechanism

Download full size | View in the Article

Fig. 4. The structure of the residual network

Download full size | View in the Article

Fig. 5. The spectral residual network module

Download full size | View in the Article

Fig. 6. The spatial residual network module

Download full size | View in the Article

Fig. 7. The flow chart of SSARN with IP dataset as an example

Download full size | View in the Article

Fig. 8. IP dataset

Download full size | View in the Article

Fig. 9. SA dataset

Download full size | View in the Article

Fig. 10. PU dataset

Download full size | View in the Article

Fig. 11. Houston dataset

Download full size | View in the Article

Fig. 12. The visualization result of each algorithm on the IP dataset

Download full size | View in the Article

Fig. 13. The visualization result of each algorithm on the SA dataset

Download full size | View in the Article

Fig. 14. The visualization result of each algorithm on the PU dataset

Download full size | View in the Article

Fig. 15. The visualization result of each algorithm on the Houston dataset

Download full size | View in the Article

Sample category	Sample name	Training samples	Test samples
Total	——	2 045	10 249
0	Alfalfa	9	46
1	Corn-notill	285	1 428
2	Corn-mintill	166	830
3	Corn	47	237
4	Grass-pasture	96	483
5	Grass-trees	146	730
6	Grass-pasture-mowed	5	28
7	Hay-windrowed	95	478
8	Oats	4	20
9	Soybean-notill	194	972
10	Soybean-mintill	491	2 455
11	Soybean-clean	118	593
12	Wheat	41	205
13	Woods	253	1 265
14	Buildings-Grass-Trees-Drives	77	386
15	Stone-Steel-Towers	18	93

Table 1. The number of training and testing samples on IP dataset

View in the Article

Sample category	Sample name	Training samples	Test samples
Total	——	1 076	54 129
0	Brocoli_green_weeds_1	40	2 009
1	Brocoli_green_weeds_2	74	3 726
2	Fallow	39	1 976
3	Fallow_rough_plow	27	1 394
4	Fallow_smooth	53	2 678
5	Stubble	79	3 959
6	Celery	71	3 579
7	Grapes_untrained	225	11 271
8	Soil_vinyard_develop	124	6 203
9	Corn_senesced_green_weeds	65	3 278
10	Lettuce_romaine_4wk	21	1 068
11	Lettuce_romaine_5wk	38	1 927
12	Lettuce_romaine_6wk	18	916
13	Lettuce_romaine_7wk	21	1 070
14	Vinyard_untrained	145	7 268
15	Vinyard_vertical_trellis	36	1 807

Table 2. The number of training and testing samples on SA dataset

View in the Article

Sample category	Sample name	Training samples	Test samples
Total	——	423	42 776
0	Asphalt	66	6 631
1	Meadows	186	18 649
2	Gravel	20	2 099
3	Trees	30	3 064
4	Painted metal sheets	13	1 345
5	Bare Soil	50	5 029
6	Bitumen	13	1 330
7	Self-Blocking Bricks	36	3 682
8	Shadows	9	947

Table 3. The number of training and testing samples on PU dataset

View in the Article

Sample category	Sample name	Training samples	Test samples
Total	——	2 832	12 197
0	Healthy Grass	198	1 053
1	Stressed Grass	190	1 064
2	Synthetic Grass	192	505
3	Trees	188	1 056
4	Soil	186	1 056
5	Water	182	143
6	Residential	196	1 072
7	Commercial	191	1 053
8	Road	193	1 059
9	Highway	191	1 036
10	Railway	181	1 054
11	Parking Lot 1	192	1 041
12	Parking Lot 2	184	285
13	Tennis Court	181	247
14	Running Track	187	473

Table 4. The number of training and testing samples on Houston dataset

View in the Article

Size	IP	SA	PU	Houston
9×9	99.57	98.23	98.20	81.04
11×11	99.69	99.02	98.46	85.35
13×13	99.79	99.69	99.09	85.75
15×15	99.50	99.65	98.87	85.78
17×17	99.51	99.68	99.09	85.80
19×19	99.38	99.71	98.95	84.16

Table 5. The overall accuracy of the different size of the patch on the four datasets

View in the Article

Network	IP	SA	PU	Houston
Basic network	97.89	98.21	98.31	83.06
Spectral attention network	99.02	98.74	98.54	84.91
Spectral-spatial attention residual network	99.79	99.69	99.09	85.75

Table 6. The overall accuracy of the different network on the four datasets

View in the Article

IP dataset scale	5%	10%	15%	20%
2D CNN	65.49	71.18	80.17	82.85
3D CNN	73.61	84.20	90.79	93.23
HybirdSN	88.91	95.44	98.12	99.49
RIAN	87.65	93.87	96.75	97.82
SSFTT	94.98	98.19	99.30	99.45
SSARN	96.09	98.56	99.43	99.79

Table 7. The overall accuracy of the different network with different training ratios on the IP datasets

View in the Article

SA dataset scale	0.5%	1%	1.5%	2%
2D CNN	71.92	87.33	88.12	88.88
3D CNN	80.08	88.64	90.80	92.45
HybirdSN	93.27	95.55	98.17	98.44
RIAN	91.88	96.37	96.70	97.18
SSFTT	94.72	96.31	97.22	98.70
SSARN	95.02	97.89	98.71	99.69

Table 8. The overall accuracy of the different network with different training ratios on the IP datasets

View in the Article

PU dataset scale	0.3%	0.5%	0.7%	1%
2D CNN	76.13	82.92	84.86	89.22
3D CNN	76.94	82.21	85.10	86.24
HybirdSN	85.06	93.03	94.87	97.63
RIAN	76.82	89.98	91.74	94.03
SSFTT	86.99	94.78	96.65	97.24
SSARN	93.93	97.46	98.15	99.09

Table 9. The overall accuracy of the different network with different training ratios on the IP datasets

View in the Article

Sample category	2D CNN	3D CNN	HybridSN	RIAN	SSFTT	SSARN
0	43.48	84.78	100.00	95.62	95.65	100.00
1	75.28	91.81	99.72	96.29	99.93	99.86
2	74.22	88.67	99.76	96.63	99.40	99.28
3	59.92	91.14	94.94	94.94	97.89	100.00
4	93.17	94.62	98.14	95.03	99.38	100.00
5	98.08	99.45	100.00	99.45	99.45	100.00
6	57.14	60.71	96.43	92.86	100.00	100.00
7	93.31	100.00	100.00	99.79	100.00	100.00
8	50.00	95.00	100.00	95.00	100.00	100.00
9	76.95	90.43	98.66	97.48	99.69	99.38
10	85.17	93.36	99.71	98.45	99.02	99.76
11	62.56	82.97	99.33	96.46	98.82	99.83
12	99.02	100.00	100.00	98.54	99.51	100.00
13	95.02	97.15	100.00	99.84	100.00	100.00
14	80.83	96.37	100.00	97.41	100.00	100.00
15	78.49	93.55	100.00	98.92	98.92	98.92
AA	76.42	91.25	99.17	97.07	99.23	99.81
OA	82.85	93.23	99.49	97.82	99.45	99.79
Kappa	80.36	92.27	99.42	97.52	99.38	99.76

Table 10. The category accuracy，OA，AA and Kappa of the different algorithms on IP dataset

View in the Article

Sample category	2D CNN	3D CNN	HybridSN	RIAN	SSFTT	SSARN
0	98.41	96.67	100.00	99.85	100.00	100.00
1	97.26	99.60	100.00	99.76	100.00	100.00
2	94.33	99.44	100.00	99.54	99.80	100.00
3	98.57	98.42	99.50	98.28	98.14	98.64
4	97.87	97.80	98.02	98.47	97.42	99.96
5	98.36	100.00	100.00	99.92	100.00	100.00
6	97.26	97.09	100.00	99.89	100.00	99.97
7	79.3	86.86	96.73	92.91	97.32	99.35
8	99.19	97.11	100.00	98.94	99.90	100.00
9	79.44	92.04	98.87	99.18	96.71	98.93
10	92.13	91.67	100.00	96.44	98.60	99.81
11	99.95	99.48	99.90	99.95	99.90	100.00
12	95.63	99.78	99.56	98.91	100.00	100.00
13	95.70	97.01	98.88	97.66	99.63	99.63
14	70.20	77.45	95.18	94.30	97.84	99.52
15	93.97	93.69	99.45	97.12	99.28	100.00
AA	92.96	95.26	99.13	98.20	99.03	99.74
OA	88.88	92.45	98.44	97.18	98.70	99.69
Kappa	87.61	91.59	98.26	96.86	98.55	99.65

Table 11. The category accuracy，OA，AA and Kappa of the different algorithms on SA dataset

View in the Article

Sample category	2D CNN	3D CNN	HybridSN	RIAN	SSFTT	SSARN
0	92.34	83.40	97.62	93.83	96.00	100.00
1	97.42	99.36	99.83	98.05	99.84	99.98
2	59.27	48.26	83.99	67.37	92.85	98.71
3	73.60	90.57	97.55	95.72	97.49	94.13
4	98.44	84.54	100.00	99.85	100.00	99.85
5	76.91	66.81	97.95	92.56	98.11	100.00
6	79.85	70.23	97.89	75.79	81.88	94.81
7	87.78	74.71	93.70	96.44	92.18	97.58
8	93.77	91.13	94.72	85.64	96.30	97.99
AA	84.37	78.78	95.92	89.47	94.96	98.12
OA	89.22	86.24	97.63	94.03	97.24	99.09
Kappa	85.47	81.46	96.85	92.08	96.34	98.79

Table 12. Category accuracy，OA，AA and Kappa of the different algorithms on PU dataset

View in the Article

Sample category	2D CNN	3D CNN	HybridSN	RIAN	SSFTT	SSARN
0	82.72	82.53	72.93	81.29	82.53	82.62
1	84.21	82.05	81.96	58.55	84.77	85.15
2	97.82	92.28	85.15	88.32	85.94	100.00
3	91.29	91.57	72.25	80.21	92.99	91.67
4	98.20	99.24	98.49	83.62	99.72	100.00
5	94.41	92.31	77.62	60.84	94.41	95.80
6	75.75	75.19	66.33	66.51	83.86	86.38
7	66.95	56.51	73.31	45.11	65.43	88.60
8	73.47	66.95	50.05	58.83	74.41	81.87
9	44.79	50.77	100.00	20.17	51.74	47.88
10	78.18	73.91	86.34	35.48	74.38	81.50
11	77.91	72.33	80.31	51.30	78.48	93.47
12	84.21	81.75	65.61	45.61	89.83	85.26
13	98.79	96.76	100.00	76.52	99.19	100.00
14	100.00	82.45	100.00	86.68	94.93	100.00
AA	83.25	79.77	80.69	62.60	83.51	88.01
OA	79.92	76.95	79.41	60.66	80.66	85.75
Kappa	78.37	75.18	77.64	57.58	79.10	84.57

Table 13. The category accuracy，OA，AA and Kappa of the different algorithms on Houston dataset

View in the Article

Get PDF(in Chinese)

Figures&Tables (28)

References (25)

Copy Citation Text

Feifei WANG, Huijie ZHAO, Na LI, Siyuan LI, Yu CAI. Spectral-spatial Attention Residual Networks for Hyperspectral Image Classification[J]. Acta Photonica Sinica, 2023, 52(12): 1210002

Download Citation

Save the article for my favorites

Paper Information

Recommended Topics

laser devices and laser physics

Lasers and Laser Optics

laser manufacturing

Instrumentation, Measurement and Metrology

Set citation alerts for the article

Please enter your email address