In order to extract the small moving target accurately in real time in the aerial video, we propose a fusion detection method for an improved ORB feature matching and differential multiplication algorithm. First of all, as the original ORB appears to be a large number of false matching problems, we describe feature points based on K nearest neighbor. After the description of the feature points in two consecutive frames by two-way matching, we further refine consistency by sequential sampling algorithm. Then, the purified matching points are used to calculate the background motion model, compensating background activity. Finally, the four consecutive frames difference multiplication and morphology processing are used to accurately segment the moving small target in the aerial video. Experimental results show that the match after purification method of accuracy up to 99.9%, the average time of which is 0.46 s, and processing speed is about 5 times of SURF feature matching algorithm, 25 times of the SIFT feature matching algorithm, so it can meet the requirements of aerial video real-time processing and has stronger ability to resist noise.