Surface defect detection plays an important role in the selection and utilization of wood. A method is proposed for knot defect detection and localization based on the feature of gray and texture on the wood surface. First, the image is divided into blocks with equal sizes. The gray histogram of each subimage is calculated, and the gray maximum entropy is used as the criterion to achieve the preliminary recognition of the subimage. Second, the texture features of the preliminary result are extracted by local binary patterns algorithm. The support vector machine classification algorithm is utilized to precisely recognize the knot images. Finally, the subimages judged as knot images are joined together to obtain the final result. The experimental results show that the proposed method can obtain commendable recognition results. The knot recognition accuracy reaches 95% when confusion matrix is used as the evaluation criterion.