Aiming at the defect of low feature matching accuracy of the ORB algorithm, combined with the optical flow characteristics of the pyramid, this paper proposes a method to optimize the ORB feature matching. First, the region matching method is used to process the matching images, the best trusted matching sub-blocks are selected, and the invalid matching area is narrowed. Then the ORB keywords are extracted from the sub-blocks and the matching descriptors are calculated to obtain the coarse matching point pairs. Pyramid optical flow method is used to track the ORB feature points, and the motion displacement vectors of the feature points are calculated to remove the incorrect matching pairs in the rough matching part. Finally, the random sample consensus algorithm is used to further remove redundant matching points to obtain a more accurate match. Experimental results show that the optimized ORB algorithm can well possess the real-time performance and accuracy. The average time for feature matching is about 87% of the original ORB algorithm, and the average matching rate is over 98%.