Aiming at the problem of difficult registration of feature points under the influence of modal and scale differences in the process of infrared image and visible image registration, an infrared image and visible image registration algorithm based on modal transformation and robust features is proposed. First, the generation adversarial network is used to generate the corresponding pseudo infrared image from the visible image; second, the position information of feature points in infrared image is extracted by accelerated robust feature (SURF) algorithm, and the feature description is realized by improved robust feature descriptor (PIIFD); then, based on the kernel method of Hilbert space reconstruction, a single Gaussian robust point matching model is established to estimate the mapping in the presence of outliers; finally, the weighted least square method is used to estimate the transformation type to realize image registration. The experimental results show that compared with other algorithms, the proposed algorithm can improve the registration accuracy in the case of large scale difference between infrared image and visible image, the effective registration rate is 96% and has strong robustness.