Analysis show that spectral information and three-dimensional shape structure data are both contained in Fourier transform imaging spectrometer’s interference signal. A new method of three-dimensional color shape measurement is proposed, which could measure spectral reflectance of each point on sample surface and three-dimensional shape data of sample, and then achieve the three-dimensional surface hyperspectral imaging. According to the texture mapping, three-dimensional data could be reconstructed in full-color about the microstructure colored-object. Experimental results demonstrate that the proposed method is effective.