A method of recognition of color image under incoherent optical system is proposed. A color image is decomposed into three monochromatic images to be processed, then a realizable phase-only optical transfer function (OTF) is designed. The passband of this OTF is optimized to increase the signal to noise ratio (SNR) and to yield a sharp correlation peak. This OTF can be synthesized by a couple of filters. The experimental results on color image recognition are presented.