In the process of imaging, the division-of-focal-plane (DoFP) polarization detector is often disturbed by noise, and it affects the quality and accuracy of the polarization images. In this paper, first, based on the non-local self-similarity of the image and the correlation between images with different polarization directions, the image is divided into blocks by using the spatial distribution characteristics of the DoFP polarization image, and similar image blocks are selected to form a similar block matrix. Then, principal component analysis (PCA) is used to obtain the eigenvalue matrix and eigenvector matrix of the similar block matrix, based on the eigenvalue distribution characteristics of the noise and the similar block matrix, and use dimensionality reduction to denoise the image in the PCA domain. Finally, simulated and real DoFP polarization images are used to evaluate the denoising effect of the algorithm. Experimental results show that the algorithm can effectively suppress the noise in the image and preserve the texture and edge details of the image, which is at least 1 dB higher than the peak signal-to-noise ratio of existing algorithms.