A tracking algorithm of infrared target is proposed by combining non-linear edge detection and Mean Shift method. The non-linear edge detection algorithm employs dual-window arithmetic operators that have the advantage of few calculation amount, high speed and good image quality and so on. The result of edge detection is binary images. Based on this information, the Mean Shift method is improved to implement target tracking. The tracking algorithm of improved Mean Shift combines the information of the local standard deviation calculation of the target area, describes the target based on the probability density function about gray value and the local standard deviation and selects cascade kernel function to calculate the target density that make up the shortage only using gray to describe the target features. Experimental results show that the edge of infrared target under complex background is detected clearly and infrared target is auto-tracked accurately.