Facial Expression Recognition Based on Local Feature Fusion of Convolutional Neural Network

Lisha Yao; Guoming Xu; Feng Zhao

doi:10.3788/LOP57.041513

Journals >Laser & Optoelectronics Progress >Volume 57 >Issue 4 >Page 041513 > Article

Laser & Optoelectronics Progress
Vol. 57, Issue 4, 041513 (2020)

Facial Expression Recognition Based on Local Feature Fusion of Convolutional Neural Network

Lisha Yao, Guoming Xu^*, and Feng Zhao

Author Affiliations

Institute of Information and Software, Institute of Information Engineering, Anhui Xinhua University, Hefei, Anhui 230088, China

show less

DOI: 10.3788/LOP57.041513 Cite this Article Set citation alerts

Lisha Yao, Guoming Xu, Feng Zhao. Facial Expression Recognition Based on Local Feature Fusion of Convolutional Neural Network[J]. Laser & Optoelectronics Progress, 2020, 57(4): 041513 Copy Citation Text

show less

Fig. 1. Model structure of CNN

Download full size

Fig. 2. Diagram of mixing pool process

Download full size

Fig. 3. Flowchart of expression recognition framework

Download full size

Fig. 4. Average recognition rates of different algorithms for 7 kinds of facial expressions

Download full size

Fig. 5. Average recognition rates of different decision-level fusion methods for 7 kinds of facial expressions

Download full size

Layer	Structure design of CNN
Input	32×32
C1	5×5 conv, 32
S1	2×2,max-pooling & mean-pooling
C2	5×5 conv, 6
C3	5×5 conv, 128
S2	2×2,max-pooling & mean-pooling
Dropout	Dropout
FC	120
Output	Center loss & Softmax

Table 1. Parameter setting of CNN

Expression	Anger	Disgust	Fear	Happy	Neutral	Sad	Surprise
Anger	95.02	0.82	0.86	0.71	0.74	0.83	1.02
Disgust	1.56	94.14	0.95	0.52	1.17	0.82	0.84
Fear	0.81	0.83	94.77	0.61	1.02	0.82	1.14
Happy	0.72	0.75	0.65	95.66	1.01	0.63	0.58
Neutral	1.16	1.35	1.27	1.47	92.61	1.12	1.02
Sad	0.61	1.11	1.61	0.33	1.70	93.98	0.66
Surprise	0.69	0.72	0.81	0.53	0.92	0.62	95.71

Table 2. Average expression recognition rate on CK + database%

Expression	Anger	Disgust	Fear	Happy	Neutral	Sad	Surprise
Anger	100	0	0	0	0	0	0
Disgust	1.03	96.66	0.43	0.37	0.62	0.52	0.37
Fear	0.38	0.49	96.89	0.36	0.73	0.43	0.72
Happy	1.22	1.32	1.21	92.37	1.61	0.88	1.39
Neutral	0	0	0	0	100	0	0
Sad	0.35	0.66	0.81	0.36	0.68	96.88	0.26
Surprise	0.51	0.56	0.66	0.36	0.71	0.42	96.78

Table 3. Average expression recognition rate on JAFFE database %

Algorithm	Recognition rate /%	Recognition time /ms
Ref.[5]	92.10	1690
Ref.[12]	94.17	2773
Ref.[13]	89.01	2246
Ref.[14]	92.06	2655
Proposed method	94.56%	2685

Table 4. Comprehensive performance comparison of different algorithms

Lisha Yao, Guoming Xu, Feng Zhao. Facial Expression Recognition Based on Local Feature Fusion of Convolutional Neural Network[J]. Laser & Optoelectronics Progress, 2020, 57(4): 041513

Download Citation

Set citation alerts for the article

Tools

Set citation alerts for the article

Save the article for my favorites

Paper Information