With the particular research on infrared and visible images, a fast color image fusion algorithm based on color transfer technique is presented. The method directly uses the grayscale fused image and the difference signals of input multiband images to construct the source YCBCR components, then uses the statistical color transfer technique in YCBCR space to form a color fused image whose color appearance is similar to that of the target image. Two different strategies which employ the pixel averaging fusion scheme and the multiresolution fusion scheme as the grayscale image fusion solution are proposed to fulfill high real-time need and high fusion quality need, respectively. Experimental results show that the proposed color image fusion algorithm can effectively produce a natural appearing daytime-like color fused image, and can also yield a satisfactory result even using the pixel averaging fusion scheme to implement the grayscale fusion.