The SIFT descriptor has poor real-time performance and conventional binary descriptors are poorly robust to scale, rotation, and viewpoint changes. They can be improved by optimizing the sampling pattern and adding gray-value differential invariant comparisons. This study proposed a binary descriptor with a higher robustness. First, a sampling pattern with scale association and number marking was presented. Then, each sampling point of the sampling pattern was rotated to a specific position to ensure that the descriptor was invariant to scale and rotation. Subsequently, the influence of sampling pairs on the descriptor was analyzed, and 128 sampling pairs after machine learning were chosen. Finally, intensity comparison and gradient absolute value comparison were selected to build the descriptor. The image keypoint detection was based on the difference of Gaussians method. Experiment results show that the proposed descriptor is 84% and 67% faster than the SIFT descriptor in descriptor construction and descriptor matching, respectively. Its accuracy is 3% to 5% higher than that of the conventional binary descriptor in image matching with view change, and the recall rate is more than 30%. The descriptor presented in this study is suitable for time-critical image matching.