In the case of detecting motion object in the first frame by visual background extractor (ViBe) algorithm, motion objects frequently stay in the initial position for a long time, leading to a false foreground and lowering the detection accuracy. In this study, we focus on solving the problems. The initial background model is established by selecting pixels with similar color and spatial position as the sample set. Furthermore, the weight of color and spatial position in the similarity function is determined by the entropy approach. In addition, the adaptive threshold is determined by the iterative approach in classification to enhance the segmentation accuracy under various conditions. Finally, the updated probability of the background model is determined using a binary exponential distribution model with the result of the frame difference approaches. The experimental results show that the algorithm can guarantee the accuracy of the results in the presence of noise, illumination, and dynamic background. Compared with ViBe algorithm, the algorithm's precision in this study is increased by 21.56%, which effectively eliminates the effect of ghosting.