In this paper, we propose a moving target tracking algorithm based on the adaptive fusion of depth futures. This algorithm is aimed at solving the problems of poor anti-occlusion ability and robustness of traditional tracking algorithms in complex scenes. Considering the strong robustness of deep features and the advantages of high precision of shallow features, the deep sparse features are constructed using the sparse autoencoder to extract target features. Then, the depth features are adjusted according to the correlation information between adjacent frames as well as tracking confidence adaptive fusion with texture information to improve the tracker performance. To improve the robustness of the tracking algorithm while suppressing tracking drift when the confidence is lower than the set threshold, we introduce an improved speeded up robust features algorithm to locate the target. Experimental results show that the proposed algorithm has higher tracking accuracy, better robustness in occlusion scenes, and can effectively suppress tracking drift compared with the mainstream tracking algorithms.