Disparity estimation is an important technique in the field of computational imaging. A calculation framework for reconstructing scene depth information from the Fourier disparity layer (FDL) is provided. A new method of deconstructing the disparity layer (DL) of the scene for achieving disparity reconstruction in the transform domain of the light field on the basis of FDL representation is realized. FDL is reconstructed with light field data, and the corresponding DL is obtained by inverse Fourier transform of FDL. Normalized cross-correlation (NCC) is applied to measure the correlation of the pixels in DL images with those in central view images to achieve the accurate reconstruction of scene disparity. Experimental results of both simulation data and real data show that the proposed method can effectively perform accurate scene disparity reconstruction.