For the influence of temperature drift of the spectral responsivity on the repeatability infrared spectral emissivity measurement system, a temperature drift correction method is proposed based on the polynomial fitting. By analyzing the function of detector output voltage depended on its temperature. After studying the functional relationship between the temperature and spectral responsivity of detector, the spectral response curve varies with temperature is fitted and get the fitting equation. Calculating the drift correction factor of spectral responsivity, the output voltage of infrared detector is corrected. The effect of spectral response drift on the output voltage of detector is eliminated. With the development of temperature drift correction device of spectral responsivity, the temperature drift curve of spectral response is measured. Compared to the exponential fitting, the fitting consistency of sixth-order polynomial curve is excellent. Because of the application of this method, the repeatability of spectral emissivity measurement system is improved.