In order to solve the problem that infrared polarization image cannot be suitable for classifying targets, by analyzing the infrared polarization image at each polarization angle of redundancy and complementarity, a pattern recognition algorithm is proposed that is based on Fuzzy C-means clustering algorithm and edge extraction algorithm. Fuzzy C-means clustering algorithm and edge detection algorithm are respectively used for image processing, and then weighting method for image fusion is used to obtain the final result. The full accuracy, production accuracy and user accuracy of the obtained image are calculated and compared with the proposed algorithm and Fuzzy C-means clustering algorithm. The results show that the proposed algorithm is able to identify man-made targets clearly and classify them, which makes computers and observers recognize and classify the objects in the picture. Infrared polarization image pattern recognition makes infrared polarization imaging widely used in the area of detection, and it can detect man-made object under the background of mixed natural and obtain highly accurate results effectively and quickly.