Particle degeneration distinctly jeopardizes the visual tracking performance in particle filter. In order to deal with this shortcoming, an improved particle filter with genetic evolution strategy is addressed. In the proposed algorithm, the degeneracy problem is solved by applying the selection operator of genetic algorithm to choose the suboptimal samples iteratively based on the predefined likelihood threshold, and then the crossover and mutation operation are implemented to the samples which are not selected, so the diversity of the particles is maintained. Furthermore, considering the object appearance changes, the multi-template is employed to adaptively update the appearance of tracked object for keeping the accuracy in the scenario of tracking. The experiment results show that the proposed visual tracking algorithm can effectively track the moving object in the real-time indoor video sequences and is robust to the illumination and pose variations.