A fast image matching method based on the improved accelerate-KAZE (AKAZE) algorithm is proposed to address the issues of low matching rate and weak robustness in UAV image matching. The proposed method first constructs the nonlinear scale space during the feature extraction stage using the AKAZE algorithm, and then efficiently describes the feature points using the fast retina keypoint (FREAK) descriptor. Later, the obtained feature points are prematched using the grid-based motion statistic (GMS) method to distinguish them with high robustness. The matching outcomes are then further screened using the basis of random sample consensus (RANSAC) algorithm. Experiments are conducted on an Oxford standard image dataset and an RSSCN7 remote sensing image dataset to verify the effectiveness of the proposed method. The proposed method is compared with the improved AKAZE, ORB, KAZE, and SIFT+FREAK algorithms. Continuous testing can guarantee that the proposed method can achieve fast image registration while maintaining high accuracy. It can maintain a high robustness under image illumination change, fuzzy transformation, and compression transformation and can meet the needs of UAV image real-time matching.