Occlusion of the tracked object occurs frequently under complex background. At this time, a single track method can’t solve the occlusion problem. Under stationary background, object motion track can be used to calculate the future position when occlusion occurs. But there are motions of both the scene background and tracked foreground object on the moving platform, which results in complex object motion track. Aiming at the problem presented above, an object motion prediction method with background motion estimation is presented. The Kanade-Lucas-Tomasi (KLT) feature tracker is used to match corner features between adjacent pairs of video frames to obtain a sparse estimate of the optic flow field. A global, six-parameter affine motion model is fit to the observed displacement vectors to approximate the flow field induced by camera motion. Then, we use a Random Sample Consensus (RANSAC) procedure to robustly estimate affine parameters from the observed displacement vectors to compensate for the motion of the scene background. Finally, it is turned into object motion estimation under stationary background. Experimental results show that the new method effectively allows tracking through temporary, total occlusions.