• Opto-Electronic Engineering
  • Vol. 49, Issue 7, 210429 (2022)
Lixia Xue, Kaijian Yin, Ronggui Wang, and Juan Yang*
Author Affiliations
  • School of Computer and Information, Hefei University of Technology, Hefei, Anhui 230031, China
  • show less
    DOI: 10.12086/oee.2022.210429 Cite this Article
    Lixia Xue, Kaijian Yin, Ronggui Wang, Juan Yang. Interactive instance proposal network for HOI detection[J]. Opto-Electronic Engineering, 2022, 49(7): 210429 Copy Citation Text show less
    Pipeline of human object interaction detection
    Fig. 1. Pipeline of human object interaction detection
    Overview of human object interaction detection based on interactive instance proposal network
    Fig. 2. Overview of human object interaction detection based on interactive instance proposal network
    Interactive instance proposal network
    Fig. 3. Interactive instance proposal network
    Structure of cross-modal information fusion module
    Fig. 4. Structure of cross-modal information fusion module
    Comparison of IIPN (bottom row) with Faster-RCNN (upper row)
    Fig. 5. Comparison of IIPN (bottom row) with Faster-RCNN (upper row)
    Visualization of detection results on HICO-DET
    Fig. 6. Visualization of detection results on HICO-DET
    Visualization of fusion attention
    Fig. 7. Visualization of fusion attention
    MethodDefaultKnown Object
    FullRareNon-RareFullRareNon-Rare
    Shen et al.[27]6.464.247.12---
    HO-RCNN[9]7.815.378.5410.418.9410.85
    iCAN[8]14.8410.4516.1516.2611.3317.73
    RPNN[11]17.3512.7818.71---
    PMFNet[31]17.4615.6518.0020.3417.4721.20
    DRG[7]19.2617.7419.7123.4021.7523.89
    Peyre et al.[32]19.4014.6020.90---
    VCL[10]19.4316.5520.2922.0019.0922.87
    BaseLine19.0514.7220.3521.2218.0622.88
    IIPN(Ours)19.9015.8421.1223.1319.0824.33
    Table 1. Experimental results on HICO-DET test set of different approaches
    MethodmAParole
    Gupta et al.[39]31.8
    GPNN[24]44.0
    iCAN[8]45.3
    RPNN[11]47.5
    VCL[10]48.3
    PMFNet w/o human pose[31]48.6
    PMFNet[31]52.0
    IIPN(Ours)50.3
    Table 2. Experimental results on V-COCO test set of different approaches
    MethodDefault(Full)Known Object(Full)
    BaseLine19.0521.22
    BaseLine + IIPN w/o attention19.3321.92
    BaseLine + IIPN with attention19.7222.33
    BaseLine + CIFM w/o attention19.2621.74
    BaseLine + CIFM with attention19.6021.89
    BaseLine + IIPN + CIFM (Ours)19.9023.13
    Table 3. Ablation studies of the proposed module on HICO-DET
    Default
    FullRareNon-Rare
    M=6, N=219.5315.1220.43
    M=6, N=319.7715.7420.99
    M=6, N=419.5615.7120.03
    M=8, N=219.6615.6720.86
    M=8, N=319.9015.8421.12
    M=8, N=419.6015.7020.76
    Table 4. Effects of different M and N on experimental results
    fhfoSPfufsemmAP
    19.03
    19.35
    19.63
    19.78
    19.90
    Table 5. Influence of different characteristics on experimental results
    Number of kResult on V-COCOResult on HICO-DET
    k=149.419.81
    k=250.319.90
    k=349.919.85
    Table 6. Influence of different kon experimental results
    Lixia Xue, Kaijian Yin, Ronggui Wang, Juan Yang. Interactive instance proposal network for HOI detection[J]. Opto-Electronic Engineering, 2022, 49(7): 210429
    Download Citation