To solve the problem of model drift caused by scale variation in visual vehicle tracking, this study proposes a scale search method for the vehicle target based on a kernelized correlation filtering algorithm. The change direction of the target scale is deduced by comparing the average peak related energy of correlation filtering responses obtained from object regions with three given scales, followed by an iterative search for the best scale of the current target in the change direction. To ensure that the correlation filtering template can adapt to the change of vehicle appearance in the process of motion, the template is upgraded with adaptive weight under the condition of the best scale estimation. The manner of adaptive weighting further improves the accuracy of the template. Numerous experiments show that the proposed method effectively solves the problem of model drift caused by scale change in vehicle tracking and provides better tracking performance than other correlation filtering algorithms.