For detecting and tracking the moving objects of interest by changing illumination, a new method is proposed. The proposed method combines improved Vibe algorithm with the particle filter algorithm by introducing illumination feeding model to Vibe algorithm, and reconstructing the particle filter to solve the changing illumination problem. In the phase of moving target tracking, Linear Regression Classifier(LRC) is introduced into the recognition algorithm, as well as the background information. The improved Local Ternary Pattern(LTP) is adopted to carry out feature extraction and recognition. Developing the program and carrying out some experiments, the experimental results demonstrate that the new method can assist detecting and tracking moving target with changing illumination study.