An improved Fourier-Mellin transform algorithm for video stabilization is presented to enhance the real-time performance, and to solve the problem that motion estimation miscalculates when an image is partially occluded. The computation blocking of estimating scale, rotary and translation in jitter image sequence are analyzed. Then, improve the real Fourier transform by using conjugate symmetry of Fourier transform and access the wi by looking up tables. The computational performance is enhanced effectively. At last, the statistics histogram is used to remove pseudo global motion vector by judging concentration degree. The results show that the real-time performance of video stabilization exceeds the existing one 42% when processing 512 pixels×512 pixels gray image sequence, meet real-time requirements.It can efficiently remove estimation miscalculations because of image occluded partially, and further enhance the robustness.