Aiming at the problem of redundancy caused by the large amount of training sample data in the study of spectral reflectance, a sample classification method based on RGB information is proposed in this paper. Firstly, the color is clustered and the number of clusters is determined. Then, the BP neural network is used to reconstruct each spectral reflectance. The experimental results are evaluated by color difference, root mean square error and fitness coefficient, and compared with principal component analysis algorithm. From the experimental analysis, it can be concluded that the spectral reflectance reconstruction effect is the best when the number of clusters is 7. The average CIE2000 chromatic aberration is 0.836. The average root mean square error is 0.0149, and the average fitness coefficient is 99.82%. Finally, the color blocks with large reconstruction chromatic aberration are optimized. Experiments show that color clustering method can be well applied to spectral reflectance reconstruction.