To solve the problems about the ghost, high frequency noises from dynamic background and background model update error, an improved visual background extraction algorithm is proposed. The original image is accurately segmented into several regions by employing the superpixel model. The superpixels of true moving object from visual background extraction results are reclassified. And the ghost region is accurately identified, which can immediately detect and feedback ghost information to refresh its background model. Thus, the key problem about ghost region detection in global scale is resolved. According to the superpixel segmentation results, the small noise objects are discarded and the holes filling strategies are added to enhance robustness of the proposed algorithm. Experimental results show that the precision and recognition rate are remarkably improved by employing standard datasets.