In order to solve the problems that the similarity target tracking algorithm mainly considers the intraclass similarity of targets and ignores the interclass differences of different targets. A visual tracking algorithm based on classification-validation model is proposed, which adds attribute information to the similarity algorithm. The proposed algorithm constructs the loss function with similarity and class information, and learns intraclass similarity and interclass differences in high dimensional space. The classification and verification module is adopted to update network parameters when the target template and candidate target input into the network model. With the trained network, the deep embedding feature of target and candidate target is extracted, thus, the target tracking is achieved. Experiments are carried out on the OTB50 and UAV123 databases. Results show that the proposed algorithm can improve the tracking effect with increased target information, and has strong robustness to the similar targets.