The current unmixing of rock thermal infrared spectrum mostly focuses on granule forms, which can’t meet the requirements of high accuracy when applied to natural massive rocks. This paper aims to study the influence of rock surface roughness on spectral unmixing, which will influence the spectral emissivity and unmixing accuracy. Three grades of roughness were defined according to the natural rock surface morphology, and according to their structural forms, three kinds of samples were simulated by the method of mineral block mosaicking. The effects of roughness level on rock spectrum and unmixing were investigated by unmixing experiments using the same and different roughness grades. The results showed that good unmixing could be achieved when the mineral endmember and rock shared the same roughness, while in terms of different roughness grades, 48.3% of the 60 unmixing results presented errors over 5%, and the maximum was 25.3%. Therefore, we suggest that the effects of roughness on spectral unmixing should be completely considered in the quantitative hyperspectral remote sensing of natural massive rock.