Common fuzzy clustering algorithms can easily cause segmentation failure when an image exhibits unequal cluster sizes. Therefore, a fuzzy C-means clustering algorithm that is insensitive to cluster size is proposed. Firstly, the size of each cluster is integrated into the objective function of the fuzzy C-means algorithm with neighborhood information (FCM_S), which makes the cluster size play a role in the objective function. This improvement can balance the relative contribution from larger and smaller clusters to the objective function and weaken the sensitivity of the algorithm to unequal cluster sizes. Then, a new membership function and clustering center are deduced. Secondly, we design a new expression called “compactness” to represent the pixel distribution of each cluster, which is then introduced into the iterative clustering process. Finally, nondestructive testing images exhibiting unequal cluster sizes are used to verify the availability of the proposed algorithm. The segmentation results not only show improved visual segmentation effects but also show improved performances compared with those of other fuzzy clustering algorithms, as measured by two indices, i.e., segmentation accuracy and adjusted Rand index, thus demonstrating the anti-noise and size-insensitive capabilities of the proposed algorithm.