Detection results can be affected by foreign matters such as dust and scurf since they lead to the contour of workpieces changed in vision detection system，even mini-parts have cleaned yet. Therefore, a study of location and elimination of foreign matters algorithm is presented based on priori knowledge of target contour extracted by region growing. The algorithm, no matter their positions and shapes, especially suits for foreign matters which have an obviously gray level compared to objects. Firstly, a work-piece image with foreign matters was obtained. A segmentation algorithm based on region was employed for priori knowledge on contour. Then, in order to eliminate an outer contour of a foreign matter, the corner of this contour were located from the curvature point of view. Finally, the disconnected outer contour was intelligently repaired according to the built priori knowledge. Experimental results indicate that the measurementaccuracy does not reduce and the error of measuring result is within 6 μm. No matter if there are foreign matters on the surface of parts, this method can still obtain the right judgments. Additionally, it also can guarantee the accuracy of measurement unchanged, so as to improve the reliability of the whole vision detecting system.