Aiming at the problems of ghost, background turbulence in high frequency, camera jitter and error of background update caused by spatial propagation technique in classic visual background extractor（ViBe） algorithm, an improved ViBe algorithm is proposed. Combined with visual saliency, the new method determines whether the ghost target exists in the background model or not, and adaptively changes the time subsampling factor through the level of ghost for each pixel in the background model, which can improve the rate of ghost elimination. Self-adaptive threshold is adopted in the process of model matching by establishing a blinking degree matrix to judge the high-frequency disturbance level of background, so that the background model is better suitable for the dynamic background. Small object discard and hole filling strategies are added to the new method. It can judge if a foreground pixel is a noise point caused by camera jitter or an error of background update by counting pixel numbers in 24-connected neighboring region of foreground pixels. Therefore, it can improve the robustness of the algorithm. Experiments demonstrate that the improved algorithm is a good way to make up for the deficiency of the original ViBe algorithm. The accuracy and recognition rate are improved greatly.