The principle of reflecting rotating Fourier transform spectrometer was introduced in the present paper. The nonlinear problem of optical path difference (OPD) of rotating Fourier transform spectrometer universally exists, produced by the rotation of rotating mirror. The nonlinear OPD will lead to fictitious recovery spectrum, so it is necessary to compensate the nonlinear OPD. Three methods of correction for the nonlinear OPD were described and compared in this paper, namely NUFFT method, OPD replace method and interferograms fitting method. The result indicates that NUFFT was the best method for the compensation of nonlinear OPD, OPD replace method was better, its precision was almost the same as NUFFT method, and their relative error are superior to 0.13%, but the computation efficiency of OPD replace method is slower than NUFFT method, while the precision and computation efficiency of interferograms fitting method are not so satisfied, because the interferograms are rapid fluctuant especially around the zero optical path difference, so it is unsuitable for polynomial fitting, and because this method needs piecewise fitting, its computation efficiency is the slowest, thus the NUFFT method is the most suited method for the nonlinear OPD compensation of reflecting rotating Fourier transform spectrometer.