Infrared thermography is an effective method for the detection of metal fatigue damage conditions. However, it does not consider the effect of surface microstructure on spontaneous emission in the fatigue damage process; thus, interpreting the infrared thermal image characteristics from a microcosmic perspective is difficult. Metal fatigue is a complex energy dissipation process, and spontaneous emission has polarization properties. By incorporating polarization detection into infrared thermography, besides temperature field information, surface texture information including emissivity change can also be obtained. Based on the foregoing, this paper obtained the thermal infrared polarization images of the metal material surface under cyclic load using Q235 low carbon steel as the research object by constructing a tensile fatigue test platform and a polarization thermal image acquisition platform. We investigated the evolution processes of surface morphology of metal materials during metal fatigue damage by using the gray level co-occurrence matrix (GLCM) to extract the thermal image texture information, such as polarization azimuth, polarization degree, and Stokes parameters. The results of the experiments show that the texture features of polarized thermal images of metal components change with fatigue damage cycles, and the co-occurrence matrix statistics exhibit some regularity with cycles.