A method based on Gabor filtering for speckle fringe pattern smoothing is presented. The fringe frequencies and fringe gradient orientation of the speckle fringe pattern are obtained by windowed Fourier operation. The Gabor filters with the particular frequencies and orientation are then determined and applied to smooth the speckle fringe pattern. Simulation and experimental results show that the speckle noise in the speckle fringe patterns is removed efficiently as well as the fringe structure is well-preserved. This method provides a good basis for further extracting fringe phase field from one speckle fringe pattern.