The directional polarization camera is an ultra-wide-angle polarization imaging sensor, and stray light is one of the important factors that affect the radiation and polarization measurement precision. In order to avoid the influence of stray light on the inversion of high-precision quantitative parameters, special analysis, measurement and correction should be carried out in the laboratory. According to the characteristics of the optical system of the directional polarization camera, the stray light which affects the instrument is divided into local stray light and global stray light. The causes and performance characteristics of these two kinds of stray light are analyzed emphatically. On this basis, the stray light model of the instrument is constructed, and the method of obtaining the stray light coefficient matrix by partition of the illumination for correction the image is proposed. In the cases of unsaturation and supersaturation, by dividing the rectangular area and the two-dimensional rotation scanning imaging of the field of view, the relationship between the amount of radiation in the target area and the amount of scattered light in other non-target areas is established. Finally, the 11×11 regions of the stray light coefficient matrixes are obtained. The image is corrected according to the measured stray light coefficient matrixes. The results show that this correction method can eliminate at least 90% of stray light.