Aiming at the problem of target tracking lost or failure when the traditional mean shift algorithm uses fixed kernels or symmetric kernel function to track targets, an adaptive bandwidth mean shift target tracking algorithm based on anisotropic kernel function is proposed to improve the accuracy and real-time of target tracking. The signed distance constraint function is introduced based on the signed distance kernel function, and anisotropic kernel function is constituted, which meets that the function value is zero in external area of target, and provides accurate tracking window for target tracking. According to the fact that the mean shift based on anisotropic kernel function must meet the weights sum of the sample points to center point vectors in the tracking window is zero when applying to target tracking, the mean shift window centers of anisotropic kernel function templates are calculated. Restrictions on the target template change before and after are carried out by using similarity threshold, and the adaptive updating of the anisotropic kernel function template and accurate real-time tracking of the target are realized. Experimental results show that the proposed algorithm has higher accuracy and better real-time performance.