This paper proposes an infrared and visible image fusion method based on gray energy difference for two purposes: one, to obtain the prominent target features in an infrared image for extracting the important details in the visible image, and two, to solve the problem that the target information in traditional algorithms is not sufficiently prominent and that the details and textures are often missing. In this method, first, the target feature in the infrared image is detected by a target extraction algorithm based on gray energy difference. Second, infrared and visible images are decomposed to high and low frequencies using a non-subsampled contourlet transform (NSCT). Third, the gray energy difference map is used as the fusion weight to fuse the low-frequency parts of the infrared image and the visible image. The high-frequency part is fused by the rule of weighted variance. Finally, the NSCT inverse transform is used to fuse the high-frequency and low-frequency coefficients to obtain the final fused image. In this study, three groups of classical infrared and visible images are selected for fusion experiments and compared with other methods through subjective vision and objective indicators. Experimental results show that the algorithm can effectively highlight target information, improving contrast and sharpness and retaining texture details.