With high precision and sensitivity, frequency scanning interferometry (FSI) absolute distance measurement technique is getting more and more attention in the field of large-scale equipment manufacturing. Building an optical fiber auxiliary interferometer in an FSI system to realize the precise monitoring of optical frequency changes is necessary for achieving higher ranging accuracy. However, the stability of optical path of the auxiliary interferometer is limited by the environmental factors and dispersion effect, which would reduce the accuracy seriously. An optical spectrum calibration based FSI absolute distance measurement method is proposed in this paper for this problem. The optical path of the auxiliary interferometer in the FSI system is calibrated in situ with the help of the absorption spectrum of hydrogen cyanide (HCN) to provide stable and accurate optical frequency reference. What’s more, a fast compensation method for dispersion mismatch error is proposed to eliminate the influence of dispersion mismatch effect caused by the fiber optic assisted interferometer on measurement results. The proposed method is verified by comparison experiments with commercial interferometer at a range of 20 m. The result shows that the largest ranging deviation of the system is less than 50 μm, and the repeatability is better than ±4 μm.