Infrared polarization imaging technology has many advantages, such as improving target recognition rate and identifying camouflage targets, but the infrared polarization characteristics of targets are easy to be disturbed in complex environment, resulting in misjudgment and omission. To study the influence of ambient thermal radiation on the infrared polarization characteristics of the targets, the analytical model for the degree of infrared linear polarization of target surface under heat source irradiation is derived based on the polarization bidirectional reflection distribution function. Theoretical simulation and experimental verification show that the ambient thermal radiation will affect the infrared polarization characteristics of the targets. When the intensity of ambient thermal radiation is smaller than that of the target spontaneous thermal radiation, the degree of infrared linear polarization is negatively correlated with the intensity of ambient thermal radiation, which means the greater the intensity of ambient thermal radiation, the smaller the degree of infrared linear polarization. When the intensity of ambient thermal radiation is greater than that of the target spontaneous thermal radiation, the degree of infrared linear polarization is positively correlated with the intensity of ambient thermal radiation, which means the greater the intensity of ambient thermal radiation, the greater the degree of infrared linear polarization. The above results indicate that the degree of infrared linear polarization will change regularly when the targets are affected by external thermal radiation.