The traditional infrared dim and small target detection algorithm is generally implemented by digital signal processing, which is complex and has poor real-time performance. In this study, a features from accelerated segment test (FAST) adaptive-threshold algorithm based on a field programmable gate array (FPGA) is proposed to detect infrared dim and small targets. Based on the characteristics of FPGA parallel processing, the hardware acceleration of the algorithm was realized by a pipeline design. The improved adaptive threshold method can generate appropriate thresholds according to different environments and avoid the loss or redundancy of dim and small infrared targets that is caused by improper threshold selection. Finally, two different groups of infrared images were used for an experiment. The results show that the algorithm can detect dim and small targets in an infrared image in real time and can achieve a high detection rate and low false alarm rate, thereby meeting the requirements of real-time performance and effectiveness.