• Acta Optica Sinica
  • Vol. 38, Issue 6, 0615002 (2018)
Qinghui Li*, Aihua Li, Tao Wang, and Zhigao Cui
Author Affiliations
  • Academy of Operational Support, Rocket Force Engineering University, Xi’an, Shaanxi 710025, China
  • show less
    DOI: 10.3788/AOS201838.0615002 Cite this Article Set citation alerts
    Qinghui Li, Aihua Li, Tao Wang, Zhigao Cui. Double-Stream Convolutional Networks with Sequential Optical Flow Image for Action Recognition[J]. Acta Optica Sinica, 2018, 38(6): 0615002 Copy Citation Text show less
    (a) Original video frames; (b) optical flow images; (c) sequential optical flow images
    Fig. 1. (a) Original video frames; (b) optical flow images; (c) sequential optical flow images
    Double-stream network framework fusing appearance information and motion information
    Fig. 2. Double-stream network framework fusing appearance information and motion information
    Results of different subsequence lengths
    Fig. 3. Results of different subsequence lengths
    MethodSplit 1Split 2Split 3Average
    SI49.150.649.649.8
    SOF55.253.454.754.4
    DI50.752.553.152.1
    SOFI57.858.457.257.8
    SOFI+DI58.158.958.458.5
    SOFI+SI63.361.862.562.5
    Table 1. Recognition accuracy of HMDB51%
    MethodSplit 1Split 2Split 3Average
    SI81.480.581.981.3
    SOF79.381.579.580.1
    DI83.483.982.683.3
    SOFI85.886.185.285.7
    SOFI+DI87.786.987.387.3
    SOFI+SI89.690.990.390.3
    Table 2. Recognition accuracy of UCF101%
    NetworkHMDB51UCF101
    Spatial stream41.681.2
    Temporal stream54.375.6
    Original double-stream59.488.0
    Appearance stream43.482.3
    Motion stream55.479.1
    ST-ResNet65.692.7
    A&STM stream64.990.1
    LTM stream57.881.7
    Proposed double-stream72.694.8
    Table 3. Recognition accuracy of different convolutional networks%
    HMDB51UCF101
    Action categoryIncrementAction categoryIncrement
    Cartwheel35.6IceDancing24.6
    Climb_stairs32.3Hammering22.3
    Swing_baseball31.7FloorGymnastics17.8
    Hit30.0JumpRope17.2
    Handstand29.6Fencing16.4
    Smoke26.5BrushingTeeth13.9
    Drink24.3Skiing12.8
    Draw_sword19.8Nunchucks11.6
    Shoot_ball17.4CricketBowling11.2
    Wave16.9HighJump10.7
    Table 4. TOP10 categories of accuracy improvement%
    MethodHMDB51UCF101
    ShallowIDT+FV[15]57.284.8
    IDT+HSV[16]61.187.9
    MoFAP[17]61.788.3
    DeepCNN-hid6+IDT[18]-89.6
    TDD+IDT[19]65.991.5
    TSN[8]71.094.0
    I3D+Double-stream[20]66.493.4
    SOFI+Double-stream72.694.8
    Table 5. Comparison of recognition accuracy for different methods%
    Qinghui Li, Aihua Li, Tao Wang, Zhigao Cui. Double-Stream Convolutional Networks with Sequential Optical Flow Image for Action Recognition[J]. Acta Optica Sinica, 2018, 38(6): 0615002
    Download Citation