To solve the problem that jitter video moving object detection is not accurate, we propose a moving object detection method based on block gray projection, background difference and continuous inter-frame difference. By dividing the image frames into block processing, the algorithm combines the discrete decision mechanism to remove the target regions with low gray gradient and the local motion, so as to improve the accuracy of global motion vector estimation. A cross correlation calculation is done for the gray projection of the block area and the image correction is completed. For the corrected sequence frames, we propose a fusion strategy based on background difference and continuous three frame difference method, which can deal with and enhance the moving target area. The selfadaptive thresholding segmentation for the differential image smoothing fusion processing and Otsu method is used to detect foreground moving targets. In jitter video sequence experiment, compared with different algorithms, the proposed algorithm can effectively detect moving targets in the jitter scene, and ensure better detection results and faster detecting speed.