To meet the requirements of real-time and robust image stitching of unmanned aerial vehicle (UAV) aerial photography, this paper proposes an aerial image stitching algorithm for UAVs based on an improved fast feature-point extraction and description (ORB) algorithm combined with a progressive sample consensu (PROSAC) algorithm. First, the feature points are detected by the speeded up robust feature (SURF) algorithm and described by the rotation-aware binary robust independent elementary features (rBRIEF) algorithm with rotation characteristics. Next, the bidirectional matching algorithm and nearest-neighbor distance ratio algorithm are used to implement feature point coarse matching; subsequently, the PROSAC algorithm is used to eliminate mismatches. Then, the global homography transformation model is used for image registration. Finally, the gradual-in and gradual-out image blending method is used to seamlessly blend the images. The experimental results indicate that the algorithm achieves excellent balance between accuracy and speed, and realizes fast and good image stitching.