Combining with the lαβ color space and the nonsubsampled contourlet transform (NSCT), a novel fusion algorithm for infrared and color visible images based on the color transfer and information entropy was proposed. Firstly, the R, G, B bands of the color visible image were averaged and then the intensity component image was attained. Secondly, the attained component image and the infrared image were merged based on the NSCT and then a grayscale fused image was obtained. Lastly, a color transfer method based on the lαβ color space was employed to transfer the color information of the color visible image into the fused grayscale image and then the final fused color image wss attained. When the grayscale visible image and the infrared image arefused, according to the physical characters of the infrared image, a weighted averaging scheme based on the local area mean and local area information entropy was proposed for the lowpass subband coefficients. As well, according to the distribution differences of the noise and the geometric characters of the image in the NSCT domain, a selection principle based on the local area direction entropy and the local area energy was proposed for the bandpass directional subband coefficients. Experimental results demonstrated that the proposed algorithm not only can extract the abundant background information of the visible image and the target localization information of the infrared image, but also can make the fused color image maintain the nature color information of the visible image as much as possible.