An improved shape context algorithm is proposed to overcome the shortcoming of traditional shape context algorithm in lacking of rotation invariance ability. The algorithm looks for the direction where the most sampling points are included to change the image angle. Then it compares the corresponding regions in the image and calculates the match cost to add the rotation invariance ability into the algorithm. To improve the calculation speed, the pruning is induced in the algorithm to address the problem of traversing sample points in calculating the histogram distance. The experiment in the case of the same recall rate shows that the Precision Racall Line(PR) curve of proposed algorithm is closed to that of the Zheng, but the calculating speed is double that of above mentioned. Moreover, the PR curve of this algorithm is obviously better than that of traditional shape context algorithm, and the retrieval speed is increased greatly. Therefore, the overall performance of the algorithm is improved, and it is more suitable for the binary image retrieval.