The quality and safety of wheat is an important part of food safety. The traditional identification and detection method of moldy wheat seed requires complex processing steps, which is time-consuming and has poor feature extraction capability, and is prone to the loss of effective image information, resulting in poor wheat moldy seed identification detection. To solve the above problems, a terahertz spectral image recognition method for moldy wheat based on denoising convolutional neural network-broad learning system (D-BLS) is proposed in this paper. The method improves the traditional broad learning system (BLS) algorithm and constructs a D-BLS moldy wheat classification and recognition model by introducing a denoising convolutional neural network (DnCNN) denoising network to enhance image quality and improve the recognition accuracy of moldy wheat terahertz spectral images. The results show that D-BLS outperforms the traditional BLS algorithm in terms of recognition accuracy, with a recognition accuracy of 93.13%. Fruthermore, support vector machine (SVM), back propagation neural network (BPNN), convolutional neural network (CNN) are used for modeling to compare with D-BLS. The experimental results show that the classification accuracy of the D-BLS network is 13.83%, 7.79% and 3.96% higher than that of SVM, BPNN and CNN, respectively. Therefore, it is believed that the proposed D-BLS algorithm can provide a new effective method for early identification of wheat mold.