We employ terahertz time-domain spectroscopy (THz-TDS) combined with chemometrics to identify Calculus bovis and its confounding substances, and obtain the THz-TDS of Coptidis rhizome, Rhubarb, Cattail pollen, Calculus bovis, artificial Calculus bovis, and adulterate Calculus bovis. The random forest (RF) classification model and the support vector machine (SVM) model which adopts three kinds of parameter optimization are established, respectively. The classification and identification of the THz absorption spectra of six kinds of matter are conducted. In addition, the RF model based on the synthetic minority over-sampling technique (SMOTE) is proposed to solve the problem that the recognition rate of the RF model decreases due to the serious unbalanced sample dataset. The results show that both the RF model and the SVM model can achieve a recognition rate of about 95.00%. However, the RF model can run much faster, whose running time is only 2% of that of the optimal PSO-SVM model. The RF model based on the SMOTE technique can effectively solve the problem of low recognition rate caused by unbalanced data. The recognition rate increases from 84.17% to 94.17%, and the operation speed is basically constant. The research conclusion provides a new approach for the identification of rare Chinese medicine using terahertz spectroscopy.