An algorithm to segment color document images into four different classes--background, photograph, text and graph is developed. By use of a modified RGB-ellipse color model, several features from the histogram of high-frequency wavelet transformed sub-bands are got to classify the color document image into the according regions guided by the rough set theory. The simulation shows that its performance in segmenting color document image is perfect.