Aiming at the problems of fast motion, scale variation, deformation and occlusion in visual target tracking algorithms, the visual target tracking algorithm based on image signature algorithm is proposed. The proposed algorithm is based on the correlation filtering algorithm. The target appearance model is constructed with various features, and the precision and robustness of the proposed algorithm are improved. In order to solve the target relocation problem under the condition of severe occlusion, the image signature algorithm is used to calculate image sparse salient regions and to obtain the position of candidate target. The candidate target is re-ranked by the classifier, and the target is relocated. Scale pool strategy and adaptive model updating strategy are used to solve the problems of scale variation and tracking drift in tracking. Standard data sets are used to test the performance of the proposed algorithm, and the results show that the proposed algorithm is superior to traditional correlation filtering algorithms in terms of tracking success rate and precision. The proposed algorithm can solve the target tracking problems under the conditions of fast motion, scale variation, deformation and occlusion.