During the long-term tracking process of a single infrared target, many technical problems occur, such as strong background interference, occlusion, deformation, and target feature attenuation. An infrared target-tracking algorithm based on tracking-learning-detection (TLD) was proposed to solve these problems. Based on compressive tracking (CT), generalized Harr-like features were replaced by histograms of oriented gradient features. In our proposed method, a complementary random measurement matrix, which extracted texture and optimized grayscale feature-weights, was introduced. Moreover, a Kalman filter, used to record the space context location information, was adopted. Hence, the tracking failure and global retrieval problem of traditional CT and TLD algorithms can be solved when the target is occluded or deformed. The infrared image-tracking algorithm based on the combination of the TLD algorithm framework and improved CT algorithm effectively solves the problem of occlusion and strong interference and improves the tracking accuracy and long-term tracking stability of the algorithm. Experimental results show that the proposed algorithm can track well in real time and maintain good accuracy and robustness in an infrared ground environment.