A method based on Support Vector Machine (SVM) classification algorithm to detect the defects of oil-seal surface was put forward, in which the defective area and non-defective area were treated as two different textures and were sampled respectively to be learned for classification by SVM. Testing areas were cut out of the preprocessed images of oil-sea different sections, and different feature parameters were selected according to the characteristics of various defects in oil-seal testing area on the base of image analysis. SVM recognition model was established by application of Radial Basis Function (RBF), and the parameters of RBF were optimized through cross validation experiments. The results showed that the proposed approach was characterized by low cost, high reliability, excellent generalization, and easy on-line implementation and so on, and could be applied for defect detection of various products.