Speckle noise is one of the main limiting factors for the image quality of Fourier domain polarization-sensitive optical coherence tomography (FD-PS-OCT). It reduces image contrast and makes tissue boundaries blurred. In order to improve the image quality, a split-spectrum method for speckle noise reduction is proposed. The method splits FD-PS-OCT interference spectrum into several narrower spectrum. Then a filtering window function is performed on every individual narrower spectrum. After the traditional FD-PS-OCT data processing, the several individual spectrums can be averaged to suppress speckle noise. Simulations and experiments show that the proposed method can reduce speckle noise effectively and improve the image quality.