At present, many image tampering detection methods mainly measure the distance between image features to complete feature matching, ignoring the color information of the image, resulting in more false detection and missed detection in the detection results. In this paper, color information is introduced into the process of image feature matching, and a tamper detection algorithm based on color constraint model is designed. The Laplacian operator and Harris operator are utilized to extract the image features. The R, G and B primary color information of the pixels, as well as the image feature descriptor are adopted to form a color restriction model for measuring the color information between feature points. Then the image feature matching is accomplished by using the distance measurement between the measure value and the feature points to fully eliminate the mismatch phenomenon and effectively improve the matching accuracy. Additionally, the distance penalty model is constructed according to the distance variance between feature points to cluster the matched image features for identifying the tampered content accurately. The experimental results show that compared with other tamper detection algorithms, the proposed algorithm not only has higher detection accuracy for forgery content, but also has better adaptability for content operations such as blur and rotation.