The efficiency of single-junction solar cells has a limitation due to the low energy of the solar spectrum and weak photon response in the high-energy region. The spectral mismatch loss is the main factor that limits the efficiency of solar cells. An effective way of reducing the spectral mismatch loss is using spectral conversion materials to convert photons in low response regions into photons in high response regions. Spectral conversion materials include up-conversion (UC), down-shifting (DS), and quantum cutting (QC) luminescent materials. In the UC process, multiple photons with energies lower than the bandgap of the solar cell are converted into higher-energy photons. In the DS process, a high-energy photon is converted into a low-energy photon. In the QC process, a high-energy photon is converted into multiple low-energy photons. These spectrally converted photons can be efficiently used by solar cells, improving their energy conversion efficiency. Therefore, this study summarizes the spectral adaptation conditions of different types of solar cells, details the latest research progress of UC, DS, and QC luminescent materials and give a prospect of the future development direction of using spectral conversion materials to improve the efficiency of solar cells.