Aimed at target tracking in the video image sequences, the tracking algorithm which combines the Mean shift algorithm and Kalman filtering has been proposed, but the offset in the image plane caused by the motion of the robot can not be ignored when implementing the real-time moving object following with a mobile robot. Based on the description of the relationship between the offset and the motion, the target dynamics with the motion of robot as the external control is depicted, and the algorithm that combines the Mean shift and Kalman filter in a novel way is proposed. With the state estimation of Kalman filter as the starting position of the Mean shift and with the converge location of the Mean shift as tracking results of current frame, the state estimation is replaced with the converge location of the Mean shift. Two algorithms work alternately and interact with each other. Experiment results indicate that the proposed algorithm is real-time and robust on dynamic target tracking under complex outdoor environment.