• Acta Photonica Sinica
  • Vol. 52, Issue 4, 0410003 (2023)
Decao MA1, Yong XIAN1、*, Juan SU2, Shaopeng LI1, and Bing LI1
Author Affiliations
  • 1College of War Support, Rocket Force University of Engineering, Xi'an 710025, China
  • 2College of Nuclear Engineering, Rocket Force University of Engineering, Xi'an 710025, China
  • show less
    DOI: 10.3788/gzxb20235204.0410003 Cite this Article
    Decao MA, Yong XIAN, Juan SU, Shaopeng LI, Bing LI. Visible-to-infrared Image Translation Based on an Improved Conditional Generative Adversarial Nets[J]. Acta Photonica Sinica, 2023, 52(4): 0410003 Copy Citation Text show less

    Abstract

    Using visible images to obtain corresponding infrared images is an effective approach to address the lack of infrared images in infrared guidance, infrared countermeasures, and infrared object recognition tasks. At present, the infrared radiation properties of the target can be efficiently simulated by current methods that use modeling of infrared properties to obtain infrared simulation images. However, the simulation process of this method requires tedious operations such as the classification and segmentation of target materials, and the infrared images obtained by the simulation lack texture information. The infrared image generation algorithm based on Generative Adversarial Networks (GAN) can effectively alleviate the problems of cumbersome and labor-intensive infrared image generation. However, some current infrared image generation algorithms based on GANs are prone to the problems of lack of image detail information and lack of structural information. This paper proposes a visible-to-infrared image translation algorithm based on an improved Conditional Generative Adversarial Nets (CGAN). Different from the current UNet network and its variants which focus on the utilization of the underlying features of the image, the generative network not only focuses on the utilization of the underlying features of the image, but also strengthens the utilization of the underlying features of the image. In addition, some network tricks of the ConvNext network are incorporated. Techniques such as using fewer normalization layers, layer normalization instead of batch normalization, etc. The adversarial network improves the quality of the generated images by calculating the first-order feature statistics (mean) and second-order feature statistics (standard deviation) of the generated images. The mean value contributes to the generation of grayscale information of infrared images, and the standard deviation contributes to the generation of structural information of infrared images. The research of the adversarial network focusing on the image receptive field features is transformed into the research of the image feature statistical information, which reduces the constraints on the generative network and releases the greater potential of the generative network. In the experiment, three datasets were used, namely the VEDAI dataset, the OSU dataset and the KAIST dataset, and six objective evaluation metrics were used to evaluate the quality of the generated images, including peak signal-to-noise ratio, structural similarity, multi-scale structural similarity, learning perceptual image patch similarity, Fréchet inception distance and normalized cross correlation. Compared with existing typical infrared image generation algorithms, the experimental results show that the proposed method can generate higher quality infrared images and achieve better performance in both subjective visual description and objective metric evaluation. In the matching application experiment, this paper adopts three traditional matching algorithms: SIFT algorithm, SURF algorithm and ORB algorithm, and three matching algorithms based on deep learning: the D2-Net algorithm, SuperGlue algorithm and LoFTR algorithm. The experimental results show that compared with the use of the visible image for matching, the image conversion algorithm is used to match the corresponding infrared image converted by the visible image, which can effectively reduce the matching endpoint error. The experimental results show that matching end point error is not strictly proportional to the six objective evaluation indexes, but there is a positive correlation between matching end point error and objective evaluation indexes. In general, the better the performance of objective evaluation indicators, the smaller the corresponding matching end point error. In summary, this paper proposes an improved conditional generative adversarial network for converting visible images to corresponding infrared images. The proposed method effectively alleviates the problems of the lack of texture detail information and the lack of structural information in the current infrared generation algorithm based on conditional generative adversarial network image generation in the process of image generation. The generated infrared image has good application value in the matching of the visible image and the infrared image. In addition, this paper provides new ideas for other image translation tasks.
    Decao MA, Yong XIAN, Juan SU, Shaopeng LI, Bing LI. Visible-to-infrared Image Translation Based on an Improved Conditional Generative Adversarial Nets[J]. Acta Photonica Sinica, 2023, 52(4): 0410003
    Download Citation