• Journal of Infrared and Millimeter Waves
  • Vol. 43, Issue 6, 859 (2024)
Dan ZENG1, Jian-Ming WEI1, Jun-Jie ZHANG1, Liang CHANG2, and Wei HUANG1,*
Author Affiliations
  • 1School of Communication and Information Engineering,Shanghai University,Shanghai 200444,China
  • 2Innovation Academy for Microsatellites,Chinese Academy of Sciences,Shanghai 201203,China
  • show less
    DOI: 10.11972/j.issn.1001-9014.2024.06.017 Cite this Article
    Dan ZENG, Jian-Ming WEI, Jun-Jie ZHANG, Liang CHANG, Wei HUANG. Progressive spatio-temporal feature fusion network for infrared small-dim target detection[J]. Journal of Infrared and Millimeter Waves, 2024, 43(6): 859 Copy Citation Text show less
    Progressive spatio-temporal feature fusion network structure:(a)overall architecture of PSTFNet;(b)progressive temporal accumalation module;(c)multi-scale spatial feature fusion module
    Fig. 1. Progressive spatio-temporal feature fusion network structure:(a)overall architecture of PSTFNet;(b)progressive temporal accumalation module;(c)multi-scale spatial feature fusion module
    Progressive temporal accumulation module:(a)architecture of the P2DConv module;(b)architecture of the M3DConv module
    Fig. 2. Progressive temporal accumulation module:(a)architecture of the P2DConv module;(b)architecture of the M3DConv module
    SHU-MIRST dataset simulation flowchart:(a)background shooting;(b)target template production;(c)target 3D modeling;(d)image fusion algorithm for region resampling;(e)target template embedding
    Fig. 3. SHU-MIRST dataset simulation flowchart:(a)background shooting;(b)target template production;(c)target 3D modeling;(d)image fusion algorithm for region resampling;(e)target template embedding
    SHU-MIRST dataset statistical information: (a) distribution of target sizes;(b) distribution of mean SCR
    Fig. 4. SHU-MIRST dataset statistical information: (a) distribution of target sizes;(b) distribution of mean SCR
    Examples of target motion trajectory in the SHU-MIRST dataset
    Fig. 5. Examples of target motion trajectory in the SHU-MIRST dataset
    ROC curves of PSTFNet under different mSCR: (a) mSCR≤3;(b) mSCR>3;(c) all sequences
    Fig. 6. ROC curves of PSTFNet under different mSCR: (a) mSCR≤3;(b) mSCR>3;(c) all sequences
    Qualitative comparison results of PSTFNet and 6 benchmark algorithms on the SHU-MIRST Dataset
    Fig. 7. Qualitative comparison results of PSTFNet and 6 benchmark algorithms on the SHU-MIRST Dataset
    Visualization map of PSTFNet and the backbone network ResUNet at different stage of decoder
    Fig. 8. Visualization map of PSTFNet and the backbone network ResUNet at different stage of decoder
    方法SHU-MIRST(mSCR≤3)SHU-MIRST(mSCR>3)SHU-MIRST(all)
    IoU/(%)Pd/(%)Fa(10-6IoU/(%)Pd/(%)Fa(10-6IoU/(%)Pd/(%)Fa(10-6
    Top-Hat0.000.83856.812.6711.17185.810.934.45621.96
    IPI0.192.7580.232.7214.7557.341.086.9572.22
    PSTNN0.000.14122.942.4110.31129.360.843.70125.19
    WSLCM0.4545.804 623.485.6180.223 562.332.2657.854 252.08
    WSNM-STIPI9.6153.6135.9513.6766.0136.3511.0357.9536.09
    IMNN-LWEC0.000.0032.240.123.96139.760.041.3869.87
    ASTTV-NTLA0.000.3080.290.405.0234.670.141.9564.34
    RDIAN36.4052.0736.4667.3684.8415.4047.2363.5429.09
    DNANet38.7461.8239.7574.1985.5610.6051.1470.1329.55
    ISNet36.1749.0113.1565.3382.4613.2346.3860.7213.18
    UIUNet43.5455.9311.8874.2990.613.2854.3068.078.87
    SSTNet-64.0918.55-93.568.92-74.4015.17
    ResUNet-DTUM51.7868.5113.3275.5393.836.6060.0977.3710.97
    DNANet-DTUM51.9169.1921.6376.7193.982.6760.5977.8615.00
    Ours57.6875.8010.8076.2895.082.6964.1982.557.97
    Table 1. Quantitative comparison of different algorithms on the SHU-MIRST dataset
    方法IoU/(%)Pd/(%)Fa(10-6
    Top-Hat5.3924.66489.28
    IPI9.3836.5537.11
    PSTNN5.7917.5857.05
    WSLCM4.9237.441 389.62
    WSNM-STIPI17.7959.6638.92
    IMNN-LWEC3.107.99641.05
    ASTTV-NTLA0.271.82395.59
    RDIAN47.6986.043.95
    DNANet50.3482.575.15
    ISNet50.3582.383.86
    UIUNet48.7381.542.70
    SSTNet-85.114.83
    ResUNet-DTUM50.3186.192.87
    DNANet-DTUM50.9887.033.62
    Ours53.9391.252.26
    Table 2. Quantitative comparison of different algorithms on the IRDST-Real dataset
    方法IoU/(%)Pd/(%)Fa(10-6
    Backbone37.1750.4124.89
    Backbone + PTAM58.1076.2312.13
    Backbone + MSFM40.5856.7811.08
    PSTFNet64.1982.557.97
    Table 3. Results of the ablation experiment for the PSTFNet component modules
    方法IoU/(%)Pd/(%)Fa(10-6
    PSTFNet w/o PTAM40.5856.7811.08
    PSTFNet w/o PTAM L12343.2658.9513.14
    PSTFNet w/o PTAM L1249.0464.029.95
    PSTFNet w/o PTAM L155.3372.6010.29
    PSTFNet64.1982.557.97
    Table 4. Results of the PTAM layer ablation experiment
    方法IoU/(%)Pd/(%)Fa(10-6
    PSTFNet w/o PTAM40.5856.7811.08
    PSTFNet w/o M3DConv58.7474.997.17
    PSTFNet w/o P2Dconv53.8366.5113.23
    PSTFNet64.1982.557.97
    Table 5. Results of the PTAM composition ablation experiment
    方法IoU/(%)Pd/(%)Fa(10-6
    PSTFNet w/o MSFM58.1076.2312.13
    PSTFNet w/o MC60.5279.169.25
    PSTFNet w/o SA62.4781.3817.18
    PSTFNet64.1982.557.97
    Table 6. Results of the MSFM composition ablation experiment
    Dan ZENG, Jian-Ming WEI, Jun-Jie ZHANG, Liang CHANG, Wei HUANG. Progressive spatio-temporal feature fusion network for infrared small-dim target detection[J]. Journal of Infrared and Millimeter Waves, 2024, 43(6): 859
    Download Citation