Noise sources of white-light interference system and their influence on the interferograms are introduced. The noise in white-light interferometer is complicated because it contains both the temporal and spatial component, which will seriously affect the quality of interferograms and as a result, and reduce the measurement accuracy. Traditional denoising methods based on single component processing are unable to completely remove the noise and cause the distortion of interferometric data. Using the three-dimensional noise model, three-dimensional characteristic of noise in white-light interference system is analyzed. Experiments show that the three-dimensional denoising method can move the temporal and spatial component of noise without causing the distortion of original data. In order to solve the problem of serious time consumption from three-dimensional denoising and reconstruction, corresponding parallel algorithm based on Graphic Processing Unit (GPU) is designed, which can easily realize the real-time reconstruction.