To meet the demand of TV theodolite tracking system, a new algorithm is proposed in this paper. According to the characteristics of moving targets and tracking system, fast image difference computing method based on angle changes of TV theodolite is found, with calculation much reduced. Then after median filtering and image segmentation of difference image,precision location for targets accomplished. Experiment results prove it a fast, stable, effective algorithm and a real-time processing mode in tracking system.