• Laser & Optoelectronics Progress
  • Vol. 57, Issue 14, 141501 (2020)
Tao Zhou1, Xiaoqi Lü1、2、3、*, Guoyin Ren1, Yu Gu1、3, Ming Zhang1、4, and Jing Li1
Author Affiliations
  • 1Inner Mongolia Key Laboratory of Pattern Recognition and Intelligent Image Progressing, School of Information Engineering, Inner Mongolia University of Science and Technology, Baotou, Inner Mongolia 0 14010, China
  • 2School of Information Engineering, Inner Mongolia University of Technology, Hohhot, Inner Mongolia 0 10051, China
  • 3School of Computer Engineering and Science, Shanghai University, Shanghai 200444, China
  • 4Information Science and Technology College, Dalian Maritime University, Dalian, Liaoning 116026, China
  • show less
    DOI: 10.3788/LOP57.141501 Cite this Article Set citation alerts
    Tao Zhou, Xiaoqi Lü, Guoyin Ren, Yu Gu, Ming Zhang, Jing Li. Facial Expression Classification Based on Ensemble Convolutional Neural Network[J]. Laser & Optoelectronics Progress, 2020, 57(14): 141501 Copy Citation Text show less
    Model structure of VGGNet-19GP
    Fig. 1. Model structure of VGGNet-19GP
    Model structure of EnsembleNet
    Fig. 2. Model structure of EnsembleNet
    Example images from the CK+ dataset
    Fig. 3. Example images from the CK+ dataset
    Example images from the FER2013 dataset
    Fig. 4. Example images from the FER2013 dataset
    Schematic of the training set data enhancement
    Fig. 5. Schematic of the training set data enhancement
    Schematic of the test set data enhancement
    Fig. 6. Schematic of the test set data enhancement
    Result graphs of simple average experiment. Comparison of the average accuracy of EnsembleNet, ResNet-18, and VGGNet-19GP under (a) PublicTest and (b) PrivateTest with the increase of epoch
    Fig. 7. Result graphs of simple average experiment. Comparison of the average accuracy of EnsembleNet, ResNet-18, and VGGNet-19GP under (a) PublicTest and (b) PrivateTest with the increase of epoch
    Result graphs of weighted average.Fluctuations of EnsembleNet with the change of ResNet-18 weights under (a) PublicTest and (b) PrivateTest, and the comparison with ResNet-18 and VGGNet-19GP
    Fig. 8. Result graphs of weighted average.Fluctuations of EnsembleNet with the change of ResNet-18 weights under (a) PublicTest and (b) PrivateTest, and the comparison with ResNet-18 and VGGNet-19GP
    VGGNet-19GP, ResNet-18, and EnsembleNet accuracy curves on PublicTest dataset
    Fig. 9. VGGNet-19GP, ResNet-18, and EnsembleNet accuracy curves on PublicTest dataset
    VGGNet-19GP, ResNet-18, and EnsembleNet accuracy curves on PrivateTest dataset
    Fig. 10. VGGNet-19GP, ResNet-18, and EnsembleNet accuracy curves on PrivateTest dataset
    VGGNet-19GP, ResNet-18, and EnsembleNet accuracy curves on CK+ dataset
    Fig. 11. VGGNet-19GP, ResNet-18, and EnsembleNet accuracy curves on CK+ dataset
    Confusion matrix of EnsembleNet on the FER2013 dataset. (a) PublicTest Confusion Matrix; (b) PrivateTest Confusion Matrix
    Fig. 12. Confusion matrix of EnsembleNet on the FER2013 dataset. (a) PublicTest Confusion Matrix; (b) PrivateTest Confusion Matrix
    Confusion matrix of EnsembleNet on the CK+ dataset
    Fig. 13. Confusion matrix of EnsembleNet on the CK+ dataset
    Examples of correct classification and misclassification of PrivateTest
    Fig. 14. Examples of correct classification and misclassification of PrivateTest
    ModelFER2013CK+
    Public_Avg_AccPrivate_Avg_AccAvg_Acc
    VGGNet-19GP70.61671.84891.107
    ResNet-1871.32772.27192.845
    EnsembleNet71.69773.85497.611
    Table 1. Average accuracy on the FER2013 and CK+ datasets%
    SourceMethodDataaugmentedDropoutAccuracy /%
    FER2013CK+
    Ref. [8]Pre-processing+5_Layer_CNN---97.75
    Ref. [9]Landmark+5_Layer_CNN---97.25
    Ref. [10]CSACNN--97.45
    Ref. [11]7_CNN---81.50
    Ref. [11]7_CNN-82.90
    Ref. [11]7_CNN84.42
    Ref. [11]7_CNN84.55
    Ref. [12]Cross-connect LeNet-5---83.74
    Ref. [13]Siamese network with multiple channels---92.06
    Ref. [14]Multi-resolution feature fusion---92.10
    Ref. [15]Local feature fusion---94.56
    Ref. [16]Parallel CNN-65.694.03
    Ref. [17]Ensemble CNNs+L2_Loss--71.16-
    Ref. [23]CNN+FACS+AU--72.198.62
    Ref. [24]11_Layer_CNN-65.3-
    Ref. [25]Fully-convolution neural network--66-
    Ref. [26]8_CNN(filters decreases with net depth)--65-
    ProposedVGGNet-19GP71.84891.107
    ProposedResNet-1872.27192.845
    ProposedEnsembleNet73.85497.611
    Table 2. Comparison of proposed model with existing methods on the FER2013 and CK+datasets
    Tao Zhou, Xiaoqi Lü, Guoyin Ren, Yu Gu, Ming Zhang, Jing Li. Facial Expression Classification Based on Ensemble Convolutional Neural Network[J]. Laser & Optoelectronics Progress, 2020, 57(14): 141501
    Download Citation