Wood is an indispensable renewable resource in people’s lives, and it also plays a vital role in architecture, craft, furniture, structural material and so on. The common wood species in the market are various, and the quality and price of different wood species also differ very much.Therefore, the use of intelligent technology to undertake correct wood classification can prevent illegal trader’s shoddy product and reduce the workload of wood classification personnel greatly. Though accurate wood classification results can be obtained through the genetic and anatomical information of the wood sample, the identification process of these two methods is relatively complex, not easy for non-professionals. With the help of image information or spectral information of wood surface, wood species can be classified and conveniently. However, due to the similarity among different wood species, the classification accuracy of these two methods is often not high or only suitable for some specific wood species. Therefore, we propose a multi-feature wood classification algorithm based on the image information and spectral information of wood cross-section. First, spectral reflectance curve and image information of wood cross-section are collected, respectively. Then, the Segnet image segmentation method is used to divide the wood samples into two groups: wood with and without pores. The characteristics of pores, spectral features and textural features are extracted from wood species with pores, and the textural features and spectral features are extracted from wood species without pores. Next, according to these characteristics, a support vector machine (SVM) is used to classify wood and record the classification results. Finally, the similarity criterion is used to judge the best classification results for the samples with inconsistent classification results. In order to verify the effectiveness of the method described in this paper, the mixed sample set of 20 common hardwood and softwood species is used and classified. Experimental results show that these three wood features can be used for classification, and the highest wood recognition rate is 93.00%, 89.33% and 69.23% for spectral, textural and pore features, respectively. By similarity measurement, the three wood features can complement each other so as to improve further the wood species classification accuracy with the highest recognition accuracy of 98%. To sum up, the method described in this paper can be used to classify a mixed wood sample set that includes hardwood and softwood. The spectral features, textural features and pore features of the wood cross-section can complement each other, thus improving classification accuracy. In addition, in this paper,we also compareour method with the state-of-the-art wood species identification methods and find that the classification rate of this algorithm is higher than other algorithms.