Aiming at shortages of the mean shift tracking algorithm in dealing with the cases that the displacements of target between two successive frames are relatively large and the scales of target change quickly, and the poor adaptability of single feature to the changeable circumstance, an adaptive feature selection method is presented, to determine the most effective feature by analyzing the discriminative value of target and background. By representing the target model and the target candidate in terms of background weighted histogram and kernel weighted histogram respectively, and using the pyramid analysis technique, the pyramid mean shift tracking method is proposed to localize target via a coarse-to-fine way. Furthermore, a scale update mechanism is presented. Experimental results on various videos show that the proposed method can successfully cope with the cases such as high-speed moving target, scale variations, camera motion, partial occlusions, etc.