Image Feature Fusion Based Remote Sensing Scene Zero-Shot Classification Algorithm

Chen Wu; Hongwei Wang; Yuwei Yuan; Zhiqiang Wang; Yu Liu; Hong Cheng; Jicheng Quan

doi:10.3788/AOS201939.0610002

Journals >Acta Optica Sinica >Volume 39 >Issue 6 >Page 0610002 > Article

Acta Optica Sinica
Vol. 39, Issue 6, 0610002 (2019)

Image Feature Fusion Based Remote Sensing Scene Zero-Shot Classification Algorithm

Chen Wu¹, Hongwei Wang², Yuwei Yuan³, Zhiqiang Wang², Yu Liu², Hong Cheng², and Jicheng Quan^2、*

Author Affiliations

¹ Naval Aviation University, Yantai, Shandong 264001, China

² Aviation University of Air Force, Changchun, Jilin 130022, China

³ The 91977 of PLA, Beijing 102200, China

show less

DOI: 10.3788/AOS201939.0610002 Cite this Article Set citation alerts

Chen Wu, Hongwei Wang, Yuwei Yuan, Zhiqiang Wang, Yu Liu, Hong Cheng, Jicheng Quan. Image Feature Fusion Based Remote Sensing Scene Zero-Shot Classification Algorithm[J]. Acta Optica Sinica, 2019, 39(6): 0610002 Copy Citation Text

show less

Fig. 1. Whole framework of proposed method

Download full size

Fig. 2. Flow chart of proposed algorithm

Download full size

Fig. 3. Images of several UCM classes. (a) Agricultural; (b) airplane; (c) baseball diamond; (d) dense residential; (e) freeway; (f) harbor; (g) storage tanks; (h) tennis court; (i) overpass; (j) golf course

Download full size

Fig. 4. Images of several AID classes. (a) Airport; (b) bareland; (c) beach; (d) bridge; (e) commercial; (f) playgound; (g) pond; (h) railway station; (i) stadium; (j) viaduct

Download full size

Fig. 5. Test accuracies of unseen classes of our algorithm's fusion on UCM dataset. (a) High-level feature fusion; (b) middle-level feature fusion; (c) low-level feature fusion; (d) different-level feature fusion

Download full size

Fig. 6. Test accuracies of unseen classes of our algorithm's fusion on AID dataset. (a) High-level feature fusion; (b) middle-level feature fusion; (c) low-level feature fusion; (d) different-level feature fusion

Download full size

Fig. 7. Overall loss and test accuracy of our algorithm's fusion on UCM dataset. (a) High-level feature fusion; (b) middle-level feature fusion; (c) low-level feature fusion; (d) different-level feature fusion

Download full size

Fig. 8. Overall loss and test accuracy of our algorithm's fusion on AID dataset. (a) High-level feature fusion; (b) middle-level feature fusion; (c) low-level feature fusion; (d) different-level feature fusion

Download full size

Features		OA
Features		LatEm	BiDiLEL	JLSE	SSE	DMaP	SAE	RKT	Ours
High-level features	CaffeNet	18.96	35.81	31.28	30.81	28.52	36.60	32.02	42.63
	VGGNet	20.06	32.83	35.04	26.23	28.61	45.60	34.24	46.82
	GoogLeNet	15.68	37.04	35.98	34.24	25.44	44.20	28.44	48.04
	ResNet	20.00	35.01	22.10	19.58	25.01	24.24	18.03	38.74
	Fusion_CAT	20.20	34.44	31.88	28.02	22.42	43.20	32.01	44.42
	Fusion_CCA	20.54	35.46	21.34	20.44	27.41	30.06	24.44	37.24
	Fusion_ADL	22.86	31.84	29.54	44.83	26.81	36.40	29.62	45.63
	Fusion_Ours	23.20	35.63	37.80	49.21	31.83	44.80	34.41	61.41
Middle-level features	BoVW	20.80	36.83	20.72	26.64	18.84	25.80	29.83	37.24
	IFK	20.74	47.04	27.34	19.24	26.76	39.20	26.04	49.22
	LDA	21.92	38.03	27.56	27.83	31.22	29.40	33.59	39.23
	LLC	20.82	45.37	26.66	18.21	33.44	28.20	27.18	47.42
	pLSA	19.64	39.19	29.78	22.64	31.63	29.20	26.21	40.81
	SPM	21.94	45.37	28.36	21.03	23.04	32.40	27.82	46.84
	VLAD	20.38	42.63	26.62	24.04	30.02	39.00	29.83	44.81
	Fusion_CAT	21.70	33.64	18.62	23.01	22.41	28.60	28.76	34.83
	Fusion_CCA	21.48	35.04	20.40	29.62	20.63	34.80	28.37	37.61
	Fusion_ADL	20.40	40.64	32.34	23.44	35.57	37.80	29.64	44.04
	Fusion_Ours	23.70	46.02	33.28	40.61	37.19	39.80	34.03	59.41
Low-level features	CH	19.76	25.84	21.24	21.64	15.60	20.60	16.21	26.21
	SIFT	20.80	43.24	21.22	21.38	41.82	28.40	20.02	44.63
	GIST	21.98	37.43	20.86	18.19	31.61	21.20	20.04	39.84
	LBP	20.28	44.41	31.50	26.41	39.24	37.00	26.24	45.82
	Fusion_CAT	20.20	47.04	22.64	21.43	41.03	39.40	20.03	47.81
	Fusion_CCA	20.22	44.83	25.80	28.84	38.81	34.80	32.84	45.84
	Fusion_ADL	23.06	47.04	30.96	27.81	41.61	37.60	30.03	47.43
	Fusion_Ours	23.20	47.24	32.40	34.24	43.24	41.20	38.19	54.47

Table 1. OA values of different ZSC algorithms on UCM dataset for fusion of the same level features%

Features		OA
Features		LatEm	BiDiLEL	JLSE	SSE	DMaP	SAE	RKT	Ours
High-level features	CaffeNet	19.48	51.30	40.63	49.05	41.54	36.30	45.55	52.23
	VGGNet	19.93	49.94	44.37	44.56	41.30	33.40	44.73	52.09
	GoogLeNet	20.12	51.59	44.80	48.76	45.21	43.30	51.18	53.27
	ResNet	19.98	37.69	17.95	29.47	36.39	32.20	17.87	41.65
	Fusion_CAT	19.99	52.99	39.74	29.76	35.80	38.90	49.64	53.41
	Fusion_CCA	18.86	51.03	20.70	45.86	38.70	45.10	35.50	51.96
	Fusion_ADL	20.42	52.77	43.83	55.15	46.21	43.90	52.60	55.52
	Fusion_Ours	21.02	53.09	47.22	55.38	45.62	54.90	50.12	68.34
Middle-level features	BoVW	20.04	36.04	40.33	51.83	29.70	44.00	35.56	52.76
	IFK	19.75	50.01	45.22	30.89	34.67	43.10	28.17	51.89
	LDA	20.01	36.12	42.35	47.10	34.44	38.90	36.21	48.67
	LLC	19.72	44.58	37.61	44.85	30.00	41.30	47.99	48.51
	pLSA	20.15	35.37	41.36	49.53	34.44	36.30	44.73	50.93
	SPM	20.04	43.36	38.78	37.46	35.15	37.20	33.43	46.80
	VLAD	20.13	36.39	35.47	41.60	29.17	35.60	33.37	46.56
	Fusion_CAT	20.15	35.18	40.67	41.78	37.34	43.80	34.50	46.37
	Fusion_CCA	21.76	36.35	19.12	15.74	23.61	27.40	31.66	46.04
	Fusion_ADL	20.11	44.13	37.34	38.76	34.73	33.80	33.79	44.18
	Fusion_Ours	20.91	45.17	38.28	42.49	40.53	36.90	32.43	66.05
Low-level features	CH	20.00	40.87	35.00	30.53	45.09	26.00	18.82	46.07
	SIFT	19.98	25.19	13.59	17.81	28.64	19.20	15.38	30.28
	GIST	19.81	27.34	29.17	17.51	39.70	26.70	15.50	40.68
	LBP	19.89	31.07	15.45	21.66	26.75	32.40	15.38	34.40
	Fusion_CAT	19.76	36.87	38.65	31.30	40.00	35.50	15.38	43.19
	Fusion_CCA	19.84	35.77	40.21	31.66	40.47	24.30	43.25	46.08
	Fusion_ADL	20.03	34.72	40.04	28.17	38.11	36.50	36.39	44.15
	Fusion_Ours	20.37	46.60	41.77	32.31	46.04	36.60	46.86	53.91

Table 2. OA values of different ZSC algorithms on AID dataset for fusion of the same level features%

Method	OA
Method	High-level feature	Middle-level feature	Low-level feature	Fusion_CAT	Fusion_CCA	Fusion_ADL	Fusion_Ours
LatEm	20.06VGGNet	21.94SPM	21.98GIST	20.96	20.87	21.6	27.46
BiDiLEL	37.04GoogLeNet	47.04IFK	44.41LBP	36.82	32.84	39.42	47.83
JLSE	35.98GoogLeNet	29.78pLSA	31.50LBP	34.24	35.12	37.20	38.52
SSE	34.24GoogLeNet	27.83LDA	26.41LBP	31.64	34.80	38.05	39.83
DMaP	28.61VGGNet	33.44LLC	41.82SIFT	30.62	39.21	38.83	42.44
SAE	45.60VGGNet	39.20IFK	37.00LBP	45.60	47.60	48.90	49.20
RKT	34.24VGGNet	33.59LDA	26.24LBP	35.64	34.81	33.43	38.54
Ours	48.04GoogLeNet	49.22IFK	45.82LBP	46.44	47.59	48.24	61.43

Table 3. OA values of different ZSC algorithms on UCM dataset for fusion of different level features%

Methods	OA
Methods	High-level feature	Middle-level feature	Low-level feature	Fusion_CAT	Fusion_CCA	Fusion_ADL	Fusion_Ours
LatEm	20.12GoogLeNet	20.15pLSA	20.00CH	20.08	20.13	20.33	21.16
BiDiLEL	51.59GoogLeNet	50.01IFK	40.87CH	53.50	47.51	52.58	54.55
JLSE	44.80GoogLeNet	45.22IFK	35.00CH	45.80	43.61	47.57	49.40
SSE	49.05CaffeNet	51.83BoVW	30.53CH	38.39	33.24	42.31	45.27
DMaP	45.21GoogLeNet	35.15SPM	45.09CH	40.47	45.03	45.86	47.37
SAE	43.30GoogLeNet	44.00BoVW	32.40LBP	28.20	37.4	39.5	42.20
RKT	51.18GoogLeNet	47.99LLC	18.82CH	51.49	47.75	50.71	52.43
Ours	52.23GoogLeNet	52.76BoVW	46.07CH	50.85	53.58	55.48	66.82