In an actual scenario, as the detection distance decreases, the size of the infrared weak and small targets increases dynamically. Commonly used infrared weak and small target detection and tracking algorithms cannot continue to detect and track stably. To address these problems, we propose an adaptive infrared target size change detection and tracking method. The initial screening of weak and small targets is realized with the help of a low threshold signal-to-noise ratio and circumvents the missed detection and false detection of large targets via adaptive size segmentation. Subsequently, we built an alternative target library. Finally, the Kalman algorithm model was adopted to predict the motion trajectory, complete the small-scale wave-gate detection, and realize target tracking. Compared with the DBT conventional detection and tracking algorithm, our method considers the detection and tracking of weak and small targets and large-sized targets simultaneously. In the selected scene, where the target size dynamically increases, the detection and tracking rate of the algorithm in this study is improved by approximately 10%.