In order to improve the spectral resolution of the static polarization spectrum imaging system and obtain better target recognition ability, the orthogonal combined Wollaston prism group structure was designed, and the static phase modulation technology was used to complete the polarization spectrum imaging of the target. This technology used a multi-stage prism combination method to expand the range of spatial optical path difference without expanding the size of the original static interference prism, thereby improving the static spectral resolution. The two-dimensional image and the spectrum were separated by the method of phase modulation and image periodic matching. The function relationship between the structure size and the modulation degree on the spectral resolution was analyzed by simulation. The feasibility of the separation of the two-dimensional image and the spectrum were verified by computational simulation. The signal contrast of the 30.0 cm aluminum plate target was tested under two different conditions: sunny and cloudy. The polarization spectrum imaging was used and data extraction were 53.4% and 49.3%, while the mean of test result based on intensity image was 24.4 % and 14.1%. This design can improve the target recognition effect and increase the spectral resolution accuracy.