For the low precision of in-plane displacement measurement using traditional digital speckle correlation methodin frequency domain, a new frequency domain correlation method based on fractal interpolation was proposed. Based on traditional digital speckle correlation in frequency domain, this method utilizes Hanning window function to filter the images in order to overcome the influence of edge effect on final displacement values. Considering theself-similarity of the sub-region and the whole imagein structure morphology and gray characteristics, fractal interpolation was adopted to improve the sub-pixel interpolation, which couldlocate relevant pointsand get more accurate sub-pixel displacement values. Experimental results show that this method could reduce the absolute error within 0.01 to 0.03 pixel and maintain the measurement speed as well. Moreover, the reliability of the algorithm is tested through the rigid translation experiments.