Dual-Modal Emotion Recognition Based on Facial Expression and Body Posture in Video Sequences

Jiang Mingxing; Hu Min; Wang Xiaohua; Ren Fuji; Wang Haowen

doi:10.3788/lop55.071004

[1] Xia J, Pei D,Wang Q Z, et al. Face recognition based on local adaptive ternary derivative pattern coupled with Gabor feature［J］. Laser & Optoelectronics Progress, 2016, 53(11): 111004.

[2] Hong Y, Sun X X,Wang D, et al. Fast pose estimation method for unmanned aerial vehicle based on rectangular geometry feature［J］. Chinese Journal of Lasers, 2016, 43(5): 0508006.

[3] Huang Z W, Xue W T, Mao Q R, et al. Unsupervised domain adaptation for speech emotion recognition using PCANet［J］. Multimedia Tools & Applications, 2017, 76(5): 6785-6799.

[4] Tkalcˇicˇ M, Odic＇ A,Koir A. The impact of weak ground truth and facial expressiveness on affect detection accuracy from time continuous videos of facial expressions［J］. Information Sciences, 2013, 249(16): 13-23.

[5] Zhao G, Pietikinen M. Dynamic texture recognition using local binary patterns with an application to facial expressions［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2007, 29(6): 915-928.

[6] Ojala T, Pietikinen M, Menp T. Multiresolution gray-scale and rotation invariant texture classification with local binary patterns［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2002, 24(7): 971-987.

[7] Mu Y D,Yan S C, Liu Y, et al. Discriminative local binary patterns for human detection in personal album［C］∥Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition, 2008: 1-8.

[8] Wang X H, Hou D Y, Hu M, et al. Dual-modality emotion recognition based on composite spatio-temporal features［J］. Journal of Image and Graphics, 2017, 22(1): 39-48.

[9] Fu X F, Fu X J, Li J J, et al. Facial expression recognition using multiscale spatiotemporal local orientational pattern histogram projection in video sequences［J］. Journal of Computer-Aided Design & Computer Graphics, 2015, 27(6): 1060-1066.

[10] Gunes H, Piccardi M. A bimodal face and body gesture database for automatic analysis of human nonverbal affective behavior［C］∥Proceedings of IEEE Internatioual Conference on Pattern Recognition, 2006, 4: 1148-1153．

[11] Gunes H, Piccardi M. Bi-modal emotion recognition from expressive face and body gestures［J］. Journal of Network and Computer Applications, 2007, 30(4): 1334-1345．

[12] Gunes H, Piccardi M. Fusing face and body gesture for machine recognition of emotions［C］∥Proceedings of IEEE International Workshop on Robots and Human Interactive Communication, 2005: 306-311.

[13] Gunes H, Piccardi M. Affect recognition from face and body: early fusion vs. late fusion［C］∥Proceedings of IEEE International Conference on Systems, Man and Cybernetics, 2005, 4: 3437-3443.

[14] Chen S Z, Tian Y L,Liu Q S, et al. Recognizing expressions from face and body gesture by temporal normalized motion and appearance features［J］. Image and Vision Computing, 2013, 31(2): 175-185.

[15] Yan J J, Zheng W M,Xin M H, et al. Bimodal emotion recognition based on body gesture and facial expression［J］. Journal of Image and Graphics, 2013, 18(9): 1101-1106．

[16] Wang Y P,Hu Y H,Lei W H, et al. Aircraft target classification method based on texture feature of laser echo time-frequency image［J］. Acta Optica Sinica, 2017, 37(11): 1128004.

[17] Song Y J, Li D Y,Yang X Z, et al. Reliability evaluation of electronic products based on cloud models［J］. Acta Electronica Sinica, 2000, 28(12): 74-76.

[18] Zeng F X, Li L, Diao X P. Iterative closest point algorithm registration based on curvature features［J］. Laser & Optoelectronics Progress, 2017, 54(1): 011003.

[19] Liu H J,Liu Z,Jiang W L, et al. Approach based on cloud model and vector neural network for emitter identification［J］. Acta Electronica Sinica, 2010, 38(12): 2797-2804.

[20] Tan X Y,Triggs B. Enhanced local texture feature sets for face recognition under difficult lighting conditions［J］. IEEE Transactions on Image Processing, 2010, 19(6): 1635-1650.