In order to solve the problems of large distortion of ultra-wide field of view (FOV) infrared image and obvious difference from human vision, in this paper, an ultra-wide FOV infrared image distortion correction algorithm is proposed based on accurate model and back-projection. The algorithm first describes the relationship between object point and image point in the ultra-wide FOV infrared camera imaging by using the accurate model. Then, aiming at the disadvantage of low sampling rate of infrared image pixels, the more accurate cubic convolution interpolation method is used to interpolate the image for completing the imaging information. Finally, according to the point coordinates of corrected image, combined with the correction model and the accurate model, the corresponding coordinates on the ultra-wide FOV infrared image point coordinates are calculated by backward projection. The pixel value of the nearest image point is used as the value of the corrected image point. The experimental results show that the proposed algorithm has clear boundary and no sawtooth effect, and the average restoration error of the straight line in the scene is less than 0.35 pixels, which indicates that the algorithm has excellent applicability for the distortion correction of the ultra-wide FOV infrared image.