Author Affiliations
1Inner Mongolia Key Laboratory of Pattern Recognition and Intelligent Image Progressing, School of Information Engineering, Inner Mongolia University of Science and Technology, Baotou, Inner Mongolia 0 14010, China2School of Information Engineering, Inner Mongolia University of Technology, Hohhot, Inner Mongolia 0 10051, China3School of Computer Engineering and Science, Shanghai University, Shanghai 200444, China4Information Science and Technology College, Dalian Maritime University, Dalian, Liaoning 116026, Chinashow less
Fig. 1. Model structure of VGGNet-19GP
Fig. 2. Model structure of EnsembleNet
Fig. 3. Example images from the CK+ dataset
Fig. 4. Example images from the FER2013 dataset
Fig. 5. Schematic of the training set data enhancement
Fig. 6. Schematic of the test set data enhancement
Fig. 7. Result graphs of simple average experiment. Comparison of the average accuracy of EnsembleNet, ResNet-18, and VGGNet-19GP under (a) PublicTest and (b) PrivateTest with the increase of epoch
Fig. 8. Result graphs of weighted average.Fluctuations of EnsembleNet with the change of ResNet-18 weights under (a) PublicTest and (b) PrivateTest, and the comparison with ResNet-18 and VGGNet-19GP
Fig. 9. VGGNet-19GP, ResNet-18, and EnsembleNet accuracy curves on PublicTest dataset
Fig. 10. VGGNet-19GP, ResNet-18, and EnsembleNet accuracy curves on PrivateTest dataset
Fig. 11. VGGNet-19GP, ResNet-18, and EnsembleNet accuracy curves on CK+ dataset
Fig. 12. Confusion matrix of EnsembleNet on the FER2013 dataset. (a) PublicTest Confusion Matrix; (b) PrivateTest Confusion Matrix
Fig. 13. Confusion matrix of EnsembleNet on the CK+ dataset
Fig. 14. Examples of correct classification and misclassification of PrivateTest
Model | FER2013 | CK+ |
---|
Public_Avg_Acc | Private_Avg_Acc | Avg_Acc |
---|
VGGNet-19GP | 70.616 | 71.848 | 91.107 | ResNet-18 | 71.327 | 72.271 | 92.845 | EnsembleNet | 71.697 | 73.854 | 97.611 |
|
Table 1. Average accuracy on the FER2013 and CK+ datasets%
Source | Method | Dataaugmented | Dropout | Accuracy /% |
---|
FER2013 | CK+ |
---|
Ref. [8] | Pre-processing+5_Layer_CNN | - | - | - | 97.75 | Ref. [9] | Landmark+5_Layer_CNN | - | - | - | 97.25 | Ref. [10] | CSACNN | √ | - | - | 97.45 | Ref. [11] | 7_CNN | - | - | - | 81.50 | Ref. [11] | 7_CNN | - | √ | | 82.90 | Ref. [11] | 7_CNN | √ | | | 84.42 | Ref. [11] | 7_CNN | √ | √ | | 84.55 | Ref. [12] | Cross-connect LeNet-5 | - | - | - | 83.74 | Ref. [13] | Siamese network with multiple channels | - | - | - | 92.06 | Ref. [14] | Multi-resolution feature fusion | - | - | - | 92.10 | Ref. [15] | Local feature fusion | - | - | - | 94.56 | Ref. [16] | Parallel CNN | √ | - | 65.6 | 94.03 | Ref. [17] | Ensemble CNNs+L2_Loss | - | - | 71.16 | - | Ref. [23] | CNN+FACS+AU | - | - | 72.1 | 98.62 | Ref. [24] | 11_Layer_CNN | - | √ | 65.3 | - | Ref. [25] | Fully-convolution neural network | - | - | 66 | - | Ref. [26] | 8_CNN(filters decreases with net depth) | - | - | 65 | - | Proposed | VGGNet-19GP | √ | √ | 71.848 | 91.107 | Proposed | ResNet-18 | √ | √ | 72.271 | 92.845 | Proposed | EnsembleNet | √ | √ | 73.854 | 97.611 |
|
Table 2. Comparison of proposed model with existing methods on the FER2013 and CK+datasets