A real-time correlation tracking algorithm, based on edge amplitude distribution, is presented to solve the aerial object tracking problem. We use the edge amplitude distribution histogram to express the target and track the target by computing the best correlation between the object model and the current object. The traditional mean shift algorithm requires a symmetric kernel, such as a circle or a rectangle, and assumes that the target’s scale is symmetric during the course of tracking. The pixels are used on the edge as the samples, about 5% to 10% of the total pixels in the kernel. The object model is updated by computing the Bhattacharyya coefficient. The experimental results show that the Bhattacharyya coefficient during the course of tracking keeps between 0.95 and 1.0, and algorithm runs well at 50 fps.