• Acta Photonica Sinica
  • Vol. 50, Issue 9, 0910004 (2021)
Lili TANG1、2, Gang LIU1、*, and Gang XIAO2
Author Affiliations
  • 1College of Automation Engineering, Shanghai Electric Power University, Shanghai200090, China
  • 2College of Aeronautics and Astronautics Aerospace, Shanghai Jiao Tong University, Shanghai0040, China
  • show less
    DOI: 10.3788/gzxb20215009.0910004 Cite this Article
    Lili TANG, Gang LIU, Gang XIAO. Infrared and Visible Image Fusion Method Based on Dual-path Cascade Adversarial Mechanism[J]. Acta Photonica Sinica, 2021, 50(9): 0910004 Copy Citation Text show less

    Abstract

    The thermal target information of infrared image and some detail information of visible images are usually ignored in image fusion method based on generative adversarial network. To address the problem, an infrared and visible image fusion method based on dual-path cascade adversarial mechanism is proposed. In the stage of the generator model, a dual-path cascade is used to extract features of infrared and visible images, respectively. To improve the quality of fusion, structural similarity is introduced into the loss function. In the stage of the discriminator model, a dual discriminator is used to distinguish the generated image from the true natural visual images. The proposed method is experimented on the public data, and compared with eight state-of-the-art image fusion methods. The experimental results show that the fusion image not only retains more the target information of infrared images, but also retains more detail information of visible images, which is superior to state-of-the-art methods in subjective evaluation and objective assessment.
    LG=Ladv+αLcon(1)

    View in Article

    Ladv=1Ni=12n=1NDiIfusedn-a2(2)

    View in Article

    Lcon=1HWIf-IrF2+ξIf-IvF2(3)

    View in Article

    Lcon=β1Lgrad+β2LSSIM(4)

    View in Article

    Lgrad=1HWλ1Ifused-IirF2+Ifused-IvisF2+λ2Ifused-IirF2+Ifused-IvisF2(5)

    View in Article

    LSSIM=1-SSIM(6)

    View in Article

    LD-IR=1Nn=1NDIRIir-b2+1Nn=1NDIRIfused-d2(7)

    View in Article

    LD-VIS=1Nn=1NDVISIvis-c2+1Nn=1NDVISIfused-d2(8)

    View in Article

    EN=-l=0L-1pllog2pl(9)

    View in Article

    QAB/F=i=1Mj=1N[QAF(i,j)ωA(i,j)+QBF(i,j)ωB(i,j)]i=1Mj=1N[ωA(i,j)+ωB(i,j)](10)

    View in Article

    QAF(i,j)=QaAF(i,j)QgAF(i,j)(11)

    View in Article

    QBF(i,j)=QaBF(i,j)QgBF(i,j)(12)

    View in Article

    SCD=r(D1,I1)+r(D2,I2)(13)

    View in Article

    r(DX,IX)=ij[DX(i,j)-D-X][IX(i,j)-IX-]ij[DX(i,j)-D-X]2[IX(i,j)-IX-]2(14)

    View in Article

    SSIMa(IF)=[SSIM(IF,I1)+SSIM(IF,I2)]×0.5(15)

    View in Article

    SF=RF2+CF2(16)

    View in Article

    RF=i=1Mj=1N[F(i,j)-F(i,j-1)]2(17)

    View in Article

    CF=i=1Mj=1N[F(i,j)-F(i,j-1)]2(18)

    View in Article

    Lili TANG, Gang LIU, Gang XIAO. Infrared and Visible Image Fusion Method Based on Dual-path Cascade Adversarial Mechanism[J]. Acta Photonica Sinica, 2021, 50(9): 0910004
    Download Citation