In order to avoid these disadvantages like poor contrast, low signal-to-noise ratio, and blurry visual effect of infrared images reconstructed using the existing super-resolution image reconstruction methods, an improved reconstruction algorithm of infrared images is proposed based on human visual system. The algorithm introduces the contrast as prior information to reconstruct infrared images based on just noticeable difference, which projects objects interested to the ranges where human eyes are sensitive. Then create a measurement factor to distinguish project points and noise points to enhance the edge information of objects and filter noises. It is concluded from experimental results that the contrast of infrared images reconstructed has increased to double as the original value and the noise is eliminated. Visual effects of images obtained by the new method are improved dramatically.