To improve the accuracy of shrimp freshness discrimination, we proposed a shrimp freshness detection method based on a broad learning system in this paper. In this study, firstly, multivariate scatter correction (MSC), standard normal variate (SNV), and direct orthogonal signal correction (DOSC) were used to preprocess the raw hyperspectral data of shrimp with different days of refrigeration. And secondly, t-distributed stochastic neighbor embedding (t-SNE) was used to visualize the data after preprocessing, and the visualization results showed that the DOSC-processed data had the best clustering effect. Then, the spectral data after DOSC preprocessing were used for feature extraction using random forest (RF), principal component analysis (PCA), and two-dimensional correction spectroscopy (2D-COS). Finally, the shrimp freshness was modeled based on the characteristic wavelength, and the broad learning system (BLS) was used in shrimp freshness modeling for the first time in this paper and compared with the classical discriminant models such as partial least squares discrimination analysis (PLS-DA) and extreme learning machine (ELM). The results indicated that the RF method minimized the redundant information in the spectra, while the BLS had high accuracy and shorter discrimination time than the linear modeling method PLS-DA and the nonlinear modeling method ELM, and thus the combined RF-BLS model obtained the best freshness discrimination performance. The experimental results indicated that the hyperspectral imaging technology combined with broad learning system to identify shrimp freshness is feasible. The method proposed in this paper can provide a theoretical basis for developing an online shrimp freshness detection system.