An Attention Model-Based Facial Expression Recognition Algorithm

Jinghui Chu; Wenhao Tang; Shan Zhang; Wei Lü

doi:10.3788/LOP57.121015

[1] Li S, Deng W H[2019-10-29]. Deep facial expression recognition: a survey [2019-10-29].https:∥arxiv., org/abs/1804, 08348.

[2] Shan C F, Gong S G. McOwan P W. Facial expression recognition based on Local Binary Patterns: a comprehensive study[J]. Image and Vision Computing, 27, 803-816(2009).

[3] Ahonen T, Rahtu E, Ojansivu V et al. Recognition of blurred faces using Local Phase Quantization. [C]∥2008 19th International Conference on Pattern Recognition, December 8-11, 2008, Tampa, FL, USA. New York: IEEE, 1-4(2008).

[4] Dalal N, Triggs B. Histograms of oriented gradients for human detection. [C]∥2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05),June 20-25,2005,San Diego, CA, USA. New York: IEEE, 1-8(2005).

[5] Krizhevsky A, Sutskever I, Hinton G E. ImageNet classification with deep convolutional neural networks[J]. Communications of the ACM, 60, 84-90(2017).

[6] Simonyan K, Zisserman A[2019-10-29]. Very deep convolutional networks for large-scale image recognition [2019-10-29].https:∥arxiv., org/abs/1409, 1556.

[7] Szegedy C, Liu W, Jia Y Q et al. Going deeper with convolutions. [C]∥2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 7-12, 2015, Boston, MA, USA. New York: IEEE, 1-9(2015).

[8] He K M, Zhang X Y, Ren S Q et al. Deep residual learning for image recognition. [C]∥2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 27-30, 2016, Las Vegas, NV, USA. New York: IEEE, 770-778(2016).

[9] Huang G. Liu Z, van der Maaten L, et al. Densely connected convolutional networks. [C]∥2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 21-26, 2017, Honolulu, HI. New York: IEEE, 4700-4708(2017).

[10] Lucey P, Cohn J F, Kanade T et al. The Extended Cohn-Kanade Dataset (CK+): a complete dataset for action unit and emotion-specified expression. [C]∥2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition-Workshops, June 13-18, 2010, San Francisco, CA, USA. New York: IEEE, 94-101(2010).

[11] Pantic M, Valstar M, Rademaker R et al. Web-based database for facial expression analysis. [C]∥2005 IEEE International Conference on Multimedia and Expo, July 6, 2005, Amsterdam, The Netherlands. New York: IEEE, 1-5(2005).

[12] Valstar M F, Pantic M. Induced disgust, happiness and surprise: an addition to the mmi facial expression database[C]. Workshop on EMOTION (Satellite of LREC): Corpora for Research on Emotion and Affect, 65(2010).

[13] Viola P, Jones M J. Robust real-time face detection[J]. International Journal of Computer Vision, 57, 137-154(2004).

[14] Zhang K P, Zhang Z P, Li Z F et al. Joint face detection and alignment using multitask cascaded convolutional networks[J]. IEEE Signal Processing Letters, 23, 1499-1503(2016).

[15] King D E. Dlib-ml: A machine learning toolkit[J]. Journal of Machine Learning Research, 10, 1755-1758(2009).

[16] Bulat A, Tzimiropoulos G. How far are we from solving the 2D & 3D face alignment problem? (and a dataset of 230000 3D facial landmarks). [C]∥2017 IEEE International Conference on Computer Vision (ICCV), October 22-29, 2017, Venice. New York: IEEE, 1021-1030(2017).

[17] Liu P, Han S Z, Meng Z B et al. Facial expression recognition via a boosted deep belief network. [C]∥2014 IEEE Conference on Computer Vision and Pattern Recognition, June 23-28, 2014, Columbus, OH, USA. New York: IEEE, 1805-1812(2014).

[18] Jung H, Lee S, Yim J et al. Joint fine-tuning in deep neural networks for facial expression recognition. [C]∥2015 IEEE International Conference on Computer Vision (ICCV), December 7-13, 2015, Santiago, Chile. New York: IEEE, 2983-2991(2015).

[19] Kuo C M, Lai S H, Sarkis M. A compact deep learning model for robust facial expression recognition. [C]∥2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), June 18-22, 2018, Salt Lake City, UT, USA. New York: IEEE, 2121-2129(2018).

[20] He Z C, Zhao L Z, Chen C. Convolution neural network with multi-resolution feature fusion for facial expression recognition[J]. Laser & Optoelectronics Progress, 55, 071503(2018).

[21] Yao L S, Xu G M, Zhao F. Facial expression recognition based on local feature fusion of convolutional neural network[J]. Laser & Optoelectronics Progress, 57, 041513(2020).

[22] Meng Z B, Liu P, Cai J et al. Identity-aware convolutional neural network for facial expression recognition. [C]∥2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017), May 30-June 3, 2017, Washington, DC, USA. New York: IEEE, 558-565(2017).

[23] Ding H, Zhou S K, Chellappa R. FaceNet2ExpNet: regularizing a deep face recognition net for expression recognition. [C]∥2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017), May 30-June 3, 2017, Washington, DC, USA. New York: IEEE, 118-126(2017).

[24] Li S, Deng W H, Du J P. Reliable crowdsourcing and deep locality-preserving learning for expression recognition in the wild. [C]∥2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 21-26, 2017, Honolulu, HI. New York: IEEE, 2852-2861(2017).

[25] Cai J, Meng Z B, Khan A S et al. Island loss for learning discriminative features in facial expression recognition. [C]∥2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018), May 15-19, 2018, Xi'an. New York: IEEE, 302-309(2018).

[26] Wu H H, Su H S, Liu G H et al. Facial expression recognition algorithm based on cosine distance loss function[J]. Laser & Optoelectronics Progress, 56, 241502(2019).

[27] Vaswani A, Shazeer N, Parmar N et al[2019-10-29]. Attention is all you need [2019-10-29].https:∥arxiv., org/abs/1706, 03762.

[28] Jaderberg M, Simonyan K, Zisserman A et al[2019-10-29]. Spatial transformer networks [2019-10-29].https:∥arxiv., org/abs/1506, 02025.

[29] Hu J, Shen L, Sun G. Squeeze-and-excitation networks. [C]∥2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, June 18-23, 2018, Salt Lake City, UT. New York: IEEE, 7132-7141(2018).

[30] Park J, Woo S, Lee J et al[2019-10-29]. BAM: bottleneck attention module [2019-10-29].https:∥arxiv., org/abs/1807, 06514.

[31] Ekman P, W V. Facial Action Coding System (FACS): a technique for the measurement of facial action[J]. Rivista di Psichiatria, 47, 126-138(1978).

[32] Liu M Y, Li S X, Shan S G et al[M]. Deeply learning deformable facial action parts model for dynamic expression analysis, 143-157(2015).

[33] Zhao G Y, Pietikainen M. Dynamic texture recognition using local binary patterns with an application to facial expressions[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29, 915-928(2007).

[34] Klaser A, Marszałek M, Schmid C. A spatio-temporal descriptor based on 3D-gradients[J]. Proceedings of the British Machine Conference, 99, 1-10(2008).

[35] Liu M Y, Shan S G, Wang R P et al. Learning expressionlets on spatio-temporal manifold for dynamic facial expression recognition. [C]∥2014 IEEE Conference on Computer Vision and Pattern Recognition, June 23-28, 2014, Columbus, OH, USA. New York: IEEE, 1749-1756(2014).

[36] Yang H Y, Ciftci U, Yin L J. Facial expression recognition by de-expression residue learning. [C]∥2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, June 18-23, 2018, Salt Lake City, UT. New York: IEEE, 2168-2177(2018).

[37] Zhao X Y, Liang X D, Liu L Q et al[M]. Peak-piloted deep network for facial expression recognition, 425-442(2016).