Restricted by the technology and material of infrared focal plane array (IRFPA), the infrared detector inevitably had some defective pixel such as blind pixel and flick pixel. The gray distribution and scale of blind flick pixel and infrared point target are consistent, which is easy to cause false alarm and missed detection of remote infrared detection system. Therefore, a dynamic real-time defect repair algorithm based on spatio-temporal statistical characteristics for infrared point target detection was proposed. Based on the indepth analysis of defect pixel, a spatial extreme value filter operator was constructed to extract the current frame feature mask, accumulate the historical mask value in the time domain, and make multi-dimensional judgment in combination with probability dynamic statistics. And the image pyramid was introduced to extract blind pixel, flick pixel and defect pixel clusters. Finally, the defect elements were eliminated by multi-scale median filter. The experiment takes DDR3 as off chip storage for FPGA hardware transplantation. The results show that the proposed algorithm cound be applied to all kinds of scenes requiring point target detection, dynamically update and repair new defective pixels, low computational complexity, strong real-time performance, the defect rate was reduced from 6 ‰ to 0.046 ‰, and the detection accuracy reached 98%.