Considering that it is difficult to locate the moving objects accurately from the clutter background by using the weighted kernel based color histogram to compute the feature of object template and candidate regions, we proposed a robust Mean-Shift object tracking algorithm based on improved weighted background to enhance histogram saliency.The saliencies of the object and of the background were calculated out from their bin of histogram, which was incorporated into the traditional similarity measurement for defining an improved weighted background coefficient.Therefore, the discriminabiltity of the object from the background was increased, and the effect of background information on object locating was reduced.Experiment result shows that the tracking effect is improved apparently.