Multi-Feature Fusion Human Behavior Recognition Algorithm Based on Convolutional Neural Network and Long Short Term Memory Neural Network

Youwen Huang; Chaolun Wan; Heng Feng

doi:10.3788/LOP56.071505

[1] Laptev I, Marszalek M, Schmid C et al. Learning realistic human actions from movies. [C]∥IEEE Conference on Computer Vision and Pattern Recognition, 1-8(2008).

[2] Xu H Y, Kong J, Jiang M et al. Action recognition based on histogram of spatio-temporal oriented principal components[J]. Laser & Optoelectronics Progress, 55, 061009(2018).

[3] Zhao X J, Zeng X Q. Action recognition method based on dense optical flow trajectory and sparse coding algorithm[J]. Journal of Computer Applications, 36, 181-187(2016).

[4] Xie F, Gong S R, Liu C P et al. Human action recognition by visual word based on local and global features[J]. Computer Science, 42, 293-298(2015).

[5] Luo H L, Wang C J, Lu F. Survey of video behavior recognition[J]. Journal on Communications, 39, 169-180(2018).

[6] Krizhevsky A, Sutskever I, Hinton G E. ImageNet classification with deep convolutional neural networks[J]. Communications of the ACM, 60, 84-90(2017). http://dl.acm.org/citation.cfm?id=2999257

[7] Chollet F. Xception: deep learning with depthwise separable convolutions. [C]∥IEEE Conference on Computer Vision and Pattern Recognition, 1800-1807(2017).

[8] Howard AG, ZhuM, ChenB, et al. MobileNets: efficient convolutional neural networks for mobile vision applications[J]. arXiv preprint arXiv:1704.04861, 2017.

[9] Hochreiter S, Schmidhuber J. Long short-term memory[J]. Neural Computation, 9, 1735-1780(1997).

[10] Cai M, Liu J. Maxout neurons for deep convolutional and LSTM neural networks in speech recognition[J]. Speech Communication, 77, 53-64(2016). http://www.sciencedirect.com/science/article/pii/S0167639315001375

[11] Donahue J, Hendricks L A, Guadarrama S et al. Long-term recurrent convolutional networks for visual recognition and description. [C]∥IEEE Conference on Computer Vision and Pattern Recognition, 2625-2634(2015).

[12] Wang L M, Xiong Y J, Wang Z et al[M]. Temporal segment networks: towards good practices for deep action recognition, 20-36(2016).

[13] Ji S W, Xu W, Yang M et al. 3D convolutional neural networks for human action recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35, 221-231(2013). http://doi.ieeecomputersociety.org/10.1109/TPAMI.2012.59

[14] Qin Y, Mo L F, Guo W K et al. Combination of 3D CNNs and LSTMs and its application in activity recognition[J]. Measurement & Control Technology, 36, 28-32(2017).

[15] Huang G, Liu Z. Maaten L V D, et al. Densely connected convolutional networks. [C]∥IEEE Conference on Computer Vision and Pattern Recognition, 2261-2269(2017).

[16] Brox T, Bruhn A, Papenberg N et al. High accuracy optical flow estimation based on a theory for warping. [C]∥European Conference on Computer Vision, 25-36(2004).

[17] Qu L, Wang K R, Chen L L et al. Fast road detection based on RGBD images and convolutional neural network[J]. Acta Optica Sinica, 37, 101003(2017).

[18] Yang Y, Saleemi I, Shah M. Discovering motion primitives for unsupervised grouping and one-shot learning of human actions, gestures, and expressions[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35, 1635-1648(2013). http://www.ncbi.nlm.nih.gov/pubmed/23209043

[19] Lu T R, Yu F Q, Yang H Z et al. Human action recognition based on dense trajectories with saliency detection[J]. Computer Engineering and Applications, 54, 163-167(2018).