Tea polyphenols (TP) is one of the important ingredients of yellow tea, which has health and medicinal effects. Moreover, accurate estimation of tea polyphenol content is of great significance for tea quality identification and quantitative analysis. Previous scholars have used E-nose, E-tongue, hyperspectral and near-infrared techniques to conduct research on the estimation of tea polyphenols, and they have achieved good results. However, due to the lack of spatial characteristics, it is difficult to meet the accuracy requirements for the comprehensive judgment of the internal and external quality of tea. With the development of hyperspectral imaging system (HIS), although the estimation of tea texture based on the gray level co-occurrence matrix (GLCM) has made progress, there are still some obstacles in practical application. On the one hand, if the resolution is low, there will be no significant difference in the texture features of the image, and fewer features will not be able to fully interpret the image, resulting in lower model accuracy. On the other hand, if the resolution is high, the increase of features will make the model more complicated. Therefore, on the premise of retaining the original information of the hyperspectral image, it is necessary to explore further the potential features of hyperspectral images, especially the details of the texture. Consequently, a method of combining spectral and spatial features is proposed to improve the accuracy of tea polyphenol estimation. First, the wavelet coefficients are extracted using continuous wavelet transform based on the spectral information obtained from the hyperspectral image. Second, the wavelet coefficient features are extracted based on the wavelet coefficients, including 959 and 1 561 nm at the 4th scale, 1 321, 1 520 and 1 540 nm at the 5th scale, and 1 202 and 1 228 nm at the 6th scale. Furthermore, two characteristic wavelengths are preferred based on the sum of the energy of wavelet coefficients, which are 1 102 and 1 309 nm, respectively. Then, the gray level co-occurrence matrix and wavelet texture are extracted according to the hyperspectral image corresponding to the characteristic wavelength. Finally, the wavelet coefficient features, co-occurrence matrix, wavelet texture, and their combinations were used to construct an estimation model for the content of polyphenols in yellow tea. By comparing different regression methods based on different characteristics, including partial least squares regression (PLSR), support vector regression (SVR), and random forest (RF), five types of yellow tea were analyzed and verified. The experimental results show that SVR model based on the fusion of wavelet coefficient features, co-occurrence matrix texture, and wavelet texture achieves the best results with R2 of 0.933 0 for calibration set and 0.823 8 for the validation set. Therefore, the proposed model can effectively improve the prediction accuracy of tea polyphenol content, which also provide a technical basis for predicting other components of tea.