Traditional non-uniformity correction algorithm cannot correct the non-uniformity of polarization images, therefore a matrix correction algorithm was proposed. Firstly, the mechanism and imaging performance of non-uniformity in integrated microgrid polarimeters were presented. Then the difference of non-uniformity between the polarization image and traditional image was illustrated and the reasons for the failure of traditional methods were analyzed. By constructing a model of polarization imaging system in response to the incident polarized light source, the matrix correction algorithm was further proposed. After that, experiments gave the correction effects of polarization flatfiled images and scene images containing rich details. A quantitative analysis of experimental results proves that our matrix correction algorithm reduces the non-uniformity of flatfiled images to about 10%.