Some fundamental problems such as weak stability of feature points, uneven distribution, and poor matching quality arise in the matching process of heterogeneous images owing to the difference in the field of view of the image to be matched and the nonlinear difference in pixel gray. To mitigate these issues, an image feature point matching algorithm based on scale-invariant feature transform (SIFT) algorithm is proposed herein. First, in the feature point detection, the weight coefficient was set in the scale space and the grid was set for each layer of images. Combined with the phase response intensity map of the image, the evenly distributed and stable feature points were selected using the quadtree method. Second, the descriptor was reconstructed and the normalized Euclidean distance was used to measure the feature descriptor instead of Euclidean distance. Furthermore, a two-way matching strategy was used for rough matching. Finally, the random sample consensus (RANSAC) algorithm was used for purification. Experimental results show that the proposed algorithm can extract reliable and stable features between heterogeneous images and improve the accuracy of feature point matching.