Aiming at the problem that periodic texture background affects the fabric defect detection, a fabric defect detection method based on coarseness measurement and color distance is proposed. Firstly, the detected image is transformed from RGB color space to HSV color space, and homomorphic filtering is carried out for three channels respectively to improve the contrast between defect and background. Fabric images are classified by coarseness measurement, the same categories of fabric images are divided into the same size and non-overlapping image blocks, and the color distances of each image block and its eight-neighbor image blocks are estimated respectively, so as the implementation of the rough localization of the defects can be done. Finally, the saliency and binary processing are performed on the rough location image blocks, which can effectively reduce the influence of the periodic texture background on the detection results. The experimental results show that compared with four methods proposed recently, the proposed method shows a better detection effect on the periodic texture fabric image, and the detection accuracy is higher.