This study proposes a filter tracking algorithm based on the direction gradient histogram using time regularization and background-aware to overcome the problem of target background of the correlation filter (CF) having no optimal performance without time modeling. The training samples are firstly extracted from the real background, and classification ability of the filter is enhanced by adding the training samples. Subsequently, time regularization is introduced to construct the target relocation module under occlusion. In addition, the alternating direction multiplier method is used to optimize the solution target and reduce the computational complexity. Finally, a linear interpolation strategy is used to update the target location and scale. The proposed algorithm uses 100 video sequences and evaluation criteria in object tracking benchmark (OTB-2015) dataset for performance testing. Experimental results show that the accuracy score of filter tracking algorithm using time regularization and background-aware reaches 0.801 and success rate score is 0.762, which are 20% and 46.8% higher, respectively, compared to those of the kernelized correlation filter (KCF) algorithm. The proposed algorithm can solve the visual-tracking problem of off-plane rotation, occlusion, and background ambiguity, which has wide application prospects and use value.