• Infrared and Laser Engineering
  • Vol. 49, Issue 5, 20190552 (2020)
Xiaomin Pei1, Huijie Fan2, and Yandong Tang2
Author Affiliations
  • 1School of Information and Control Engineering, Liaoning Shihua University, Fushun 113001, China
  • 2State Key Laboratory of Robotics, Shenyang Institute of Automation, Chinese Academy of Sciences, Shenyang 110016, China
  • show less
    DOI: 10.3788/IRLA20190552 Cite this Article
    Xiaomin Pei, Huijie Fan, Yandong Tang. Two-person interaction recognition based on multi-stream spatio-temporal fusion network[J]. Infrared and Laser Engineering, 2020, 49(5): 20190552 Copy Citation Text show less
    Two person action skeletons
    Fig. 1. Two person action skeletons
    Spatial feature learning
    Fig. 2. Spatial feature learning
    Multi-stream spatio-temporal fusion network
    Fig. 3. Multi-stream spatio-temporal fusion network
    Multi-stream spatio-temporal fusion model classification accuracy for cross subject
    Fig. 4. Multi-stream spatio-temporal fusion model classification accuracy for cross subject
    Multi-stream spatio-temporal fusion model classification
    Fig. 5. Multi-stream spatio-temporal fusion model classification
    MethodCross-subjectCross view
    HBRNN[4]59.1%64.0%
    Part-aware LSTM[5]62.9%70.3%
    VA LSTM[6]79.4%87.6%
    Trust Gate ST-LSTM[7]69.2%77.7%
    AGC-LSTM[8]95.0%89.2%
    ST-GCN[9]81.5%88.3%
    Single stream (SST)85.61%92.42%
    Our method (MST)96.42%97.46%
    Table 1. Accuracy for human action recognition for NTU-RGBD dataset
    MethodAccuracy
    Co-occurrence RNN[10]90.4%
    STA-LSTM[5]91.5%
    Trust Gate ST-LSTM[7]93.3%
    VA-LSTM[6]97.6%
    Our method(weighted multi-stream)98.92%
    Table 2. Human action recognition accuracy for SBU dataset
    Net structuralParameterConvergenceAccuracy
    Multi-stream spatio-temporal modelN100epochs96.42%
    Single-stream spatio-temporal modelN100epochs85.61%
    Single-stream spatio-temporal model3N200epochs86.74%
    Table 3. Comparison of the network structure
    Xiaomin Pei, Huijie Fan, Yandong Tang. Two-person interaction recognition based on multi-stream spatio-temporal fusion network[J]. Infrared and Laser Engineering, 2020, 49(5): 20190552
    Download Citation