Traditional matching will be degraded in scene－matching and correlative tracking system because of the presence of the difference of the gray value and the distortion between the object image and the reference image. And the risk of template drift will increased by the method of simply updating the template. Eigen tracking approach for tracking rigid objects is presented which builds on and extends work on SVD method. It is a promising candidate for an appearance－based object representation. Firstly the test object images are decomposed to get the eigne space of the object. And then they are applied as the characters of the object image in the matching process. The objects over long image sequences are tracked by using the techniques. This method could bear affine image motion and image distortion to some extent and it is very promising.