Mismatch elimination is an important means of improving the accuracy of feature matching. Due to the large amount of data, texture duplication, light intensity changes, and other characteristics of satellite-borne optical remote sensing images, the performance of existing mismatch elimination methods is degraded. To solve this problem, a method based on local and guided global geometric constraints is proposed to eliminate mismatches. Based on the initial matching set, local consistency of features is used to filter out mismatches. Then, according to the transformation relationship between images, a feature topological structure is constructed, and its geometric attributes are extracted to describe structural similarity. Based on this, a feature global structure consistency constraint model is established, and residual mismatches are eliminated by deriving the optimal solution of the model. A guided matching strategy is adopted for global constraint, and matching points with high local consistency are selected to form a high internal point rate matching set, which is applied as the feature global neighborhood to improve the robustness and efficiency of the algorithm. The experimental results show that, in comparison with existing methods, the proposed method has better matching performance for satellite-borne optical remote sensing images, with an average accuracy and recall of 0.9 and 0.89, respectively. It is robust on the initial matching set with different internal point rates, and the average F score is 0.86.