Moldy apple core is a kind of internal fruit disease that threatens consumers health. Rapid, nondestructive detection of moldy apple core is helpful to improve the quality of the apple and ensure the safety of consumers before entering the consumer market. In recent years, Vis/NIR spectroscopy has been widely used in the nondestructive detection of fruit quality by its advantages of rapid, nondestructive, simple operation, low cost and batch online detection. The selection of spectral detection mode according to the actual detection requirement is an important prerequisite for developing fruit spectral nondestructive detection. Three kinds of spectral data from 243 samples were obtained based on diffuse reflection, diffuse transmission and transmission spectrum acquisition systems built by the laboratory. Five spectral pretreatment methods, including S-G smoothing (S-G), multiplicative scatter correction (MSC), standard normal variation (SNV), first derivative (FD), and normalize (NOR), were used for spectral data preprocessing. Four manifold learning algorithms, including locally linear embedding (LLE), multidimensional scaling (MDS), distributed neighbor embedding (SNE) and t-distributed neighbor embedding (t-SNE), were systematically used for spectral data dimensionality reduction. These were compared with the traditional principal component analysis (PCA) dimensionality reduction method. Finally, the least squares-support vector machine (LS-SVM) classification model was established based on the dimensionality-reduced data. The results show that the transmission detection mode is better than the diffuse transmission detection mode, and the diffuse transmission detection mode is better than the diffuse reflection detection mode in three different detection modes. The distributed neighborhood embedding algorithm is better than other dimension reduction algorithms. The model constructed by transmission detection mode combined with the distributed neighborhood embedding dimension reduction algorithm performs best. The accuracy of the calibration set and test set is 99.52% and 97.14%, respectively. The research provide a reference for establishing a spectral nondestructive detection platform and developing detection equipment for moldy apple core.