A novel system to independently detect small moving object in IR image sequences taken from a moving platform is proposed. Egomotion effects are removed through a waveletbased robust multiscale affine image registration process. The low frequency images are segmented with the improved Fisher algorithm and wavelet transformation, then the object is separated from background. The difference of the segmented image after registration is computed, and the kinematic track of the object is obtained. The experimental result shows that the system is effective.