Due to the advantages of high speed and high accuracy, infrared spectra play a vital role in classification and identification. For municipal solid wastes, the application of infrared spectra mainly focuses on recyclable garbage such as plastics, neglecting the deep separation of the non-recyclable wastes. Based on the current “Quartering Method” of municipal solid wastes, the residual wastes contain various high-value potential ingredients that can be sorted into cellulose, vinyl-polymers, and woods. These ingredients have different constituents and structures, so they have different infrared spectra. Therefore, the useful constituents can be further separated from the residual wastes by combing their infrared spectra and the machine learning classification models. This study collected cellulose, vinyl-polymers, woods, and low-value residual wastes, and 72 groups of infrared spectra data were obtained. The influence of data preprocessing, dimension reduction and algorithms on the sorting models' accuracy was investigated. Infrared spectra data were preprocessed by standard normal variate (SNV), multiplicative scatter correction (MSC), derivative correction (DC), and smooth. Principal component analysis (PCA) was used to reduce the dimension of the preprocessed data, and 72×8 and 72×5 datasets were obtained. Sorting models were built using probabilistic neural network (PNN), generalized regression neural network (GRNN), support vector machine (SVM), and random forest (RDF) algorithms. As a result, the classification accuracy of 5-Dimensional data was superior to that of 8-Dimensional data, with the average accuracy increasing 2.4%~4.4%. Based on 5-Dimensional data, DC/Smooth preprocessing achieved the highest average accuracy of 96.5% among the three preprocessing methods. The average accuracy of the PNN model was 4.2%~6.5% higher than the other three sorting models, up to 98.1%. As for the four types of residual wastes, the sorting accuracy for vinyl polymers was 93.8%, it was over 95% for cellulose and woods, and it could be up to 100% for the low-value wastes. This study examined the possibility and scientific potentiality of the combination of infrared spectroscopy and machine learning to achieve the deep sorting of residual wastes, providing a theoretical basis for the future development of fast and accurate deep separation equipment of municipal solid wastes.