• Optics and Precision Engineering
  • Vol. 32, Issue 3, 435 (2024)
Wei WANG, Feiya FU, Hao LEI, and Zili TANG*
Author Affiliations
  • The 63870 Unit of PLA, Weinan714299, China
  • show less
    DOI: 10.37188/OPE.20243203.0435 Cite this Article
    Wei WANG, Feiya FU, Hao LEI, Zili TANG. Attention interaction based RGB-T tracking method[J]. Optics and Precision Engineering, 2024, 32(3): 435 Copy Citation Text show less
    Basis framework of MDNet-based RGB-T tracking method
    Fig. 1. Basis framework of MDNet-based RGB-T tracking method
    Structure of transformer
    Fig. 2. Structure of transformer
    Framework of attention interaction based RGB-T tracking method
    Fig. 3. Framework of attention interaction based RGB-T tracking method
    Structure of FEI module
    Fig. 4. Structure of FEI module
    Structure of Self-feature Enhanced Encoder(SEE) and Cross-feature Interaction Decoder(CID)
    Fig. 5. Structure of Self-feature Enhanced Encoder(SEE) and Cross-feature Interaction Decoder(CID)
    Evaluation results of AIT and compared algorithms on GTOT dataset
    Fig. 6. Evaluation results of AIT and compared algorithms on GTOT dataset
    Evaluation results of AIT and compared algorithm on RGBT234 dataset
    Fig. 7. Evaluation results of AIT and compared algorithm on RGBT234 dataset
    Evaluation results of AIT and compared algorithms on LasHeR testing set
    Fig. 8. Evaluation results of AIT and compared algorithms on LasHeR testing set
    初始化:人工或检测算法给定第一帧的目标状态,离线训练的网络参数,随机初始化FC6层
    for t = 1 to T (T为视频序列总帧数)
    (第二帧开始由所提AIT确定目标状态)
    if t > 1 then
    步骤1根据上一帧得到的目标状态,采样候选图像块;
    步骤2利用卷积网络提取候选图像块的多层卷积特征;

    步骤3

    利用2.1节的EFI对各层卷积网络进行特征交互和增强,增强后的各层特征再经过一次EFI得到最终的融合特征;

    步骤4

    利用全连接层计算每个候选图像块为目标的概率,对概率最大的图像块做边框回归,并将其作为当前帧的跟踪结果;
    (在每一帧中初始化或者更新网络参数)

    步骤5

    根据当前帧的目标状态,采样正负训练样本,并添加至正负样本集,如果样本集内样本数量超过预设值,则剔除帧数最小的图像中采集的样本;
    步骤6每间隔10帧,利用训练样本集对全连接层参数进行微调训练。
    Table 1. [in Chinese]
    PRSR
    AIT0.8360.581
    AIT_ch0.8220.579
    AIT_spch0.8200.573
    AIT_conv30.8020.560
    AIT_nolastEFI0.8190.571
    Table 1. PR/SR scores of AIT and its variants on RGBT234 dataset