In tracking process,Kernel Correlation Filter(KCF)algorithm is easy to lose the target due to the occlusion of the target or deformation and size change of the target itself.To solve the problem,this paper makes the following improvements based on the original algorithm.First,the method of detecting the response value of the model is added as the basis for determining whether the target is lost.Once the target is determined to be lost,the model will suspend updating,and the number of sampling windows will be increased to expand the target search range until the target is determined to be re-detected and located.The second is to add an adaptive multi-scale search strategy.Third,the Histogram of Oriented Gradient (HOG) feature adopted by the original algorithm is fused with the gray feature of the image as a new sample feature.In the infrared video sequence of experiment,the original algorithm and the improved algorithm were used to qualitatively compare and analyze the tracking effect.At the same time,the tracking accuracy tested in OTB-2013 was quantitatively evaluated.The experimental results show that the comprehensive performance of the improved algorithm and its robustness against occlusion,deformation and size change are improved.