There is high correlation among each channel in RGB color space. If the color inverse halftoning method with discrete channel is applied there are large worms and color differences which is sensitized by human visual will be left. Based on the model of vector error diffusion the halftoning noise characteristics for color image are developed, and a color inverse halftoning method based on Karhunen-Loeve transform and multi-scale pyramid transform is presented. Using the Karhunen-Loeve transform the correlation among color components are reduced, then the halftoning noise in high frequency detail is separated and de-noised by multi-scale median interpolating pyramid transform and Wiener filter. Finally the color inverse halftone is constructed from the inverse pyramid and Karhunen-Loeve transform. Experimental results demonstrate the proposed method can remove halftoning noise effectively and overcome chromatic aberration. Peak signal-to-noise ratio is increased for about 2～3 dB higher than the color inverse halftoning method with disrete channel, and the excellent visual effect is obtained.