In image segmentation, many fuzzy C-means algorithms considering neighborhood information can effectively reduce noise interference, but these algorithms need additional parameters, and the large cluster difference between nondestructive test images easily causes segmentation failures. To solve this problem, this paper presents a fuzzy C-means algorithm insensitive to cluster difference based on image patchs. First, the image patch is used to replace the pixel to enter the clustering process. The weight of the pixel in the image patch is adaptively determined by the spatial distance and gray scale of the pixels. Second, based on the concept of information quantity, the expression of cluster information quantity is given and introduced into the objective function to improve the sensitivity of common fuzzy C-means algorithms to the cluster difference. Third, the new expressions of membership degree and cluster center are obtained based on the new objective function, and the algorithm flow is given. Finally, the proposed algorithm is tested by the non-destructive test images with large cluster difference. The results show that the proposed algorithm has high segmentation accuracy and better visual effects.