A Region of Interest (ROI) extraction algorithm of color image is proposed, which is based on the classical seeded region growing method guided by eye-tracking perception information. Based on the analysis of subjects’ eye-tracking data, the proposed algorithm can generate the interesting points automatically, whose neighborhood information is then used to obtain a series of initial ROIs by color homogeneity criterion in specific color space. The final ROI is picked up from the initial ROIs according to the viewer’s attentive degree. The proposed algorithm is tested in RGB, HSV and L*a*b color space. Experimental results show that the proposed algorithm using in RGB and HSV color space can accord human’s subjective perception, and have certain application value.