In the existing face recognition methods, recognition time usually increases linearly with the scale of face databases, which will certainly make the recognition algorithms less effective in practice. A novel face recognition scheme is presented. Firstly, a group of typical faces is selected, called reference faces, as comparison foundation through clustering in the eigenspace. Then, a similarity sorting space is built up based on the similarities with reference faces. For recognition, only need to compute the similarity of the input face with the reference faces to pick up the most similar faces in the sorting space quickly, then to compute the similarity with these picked up faces to determine the final recognition result. Since a lot of calculation is done offline, the recognition speed is improved greatly. Through selecting the number of reference faces and the mostly expected similar face set, the recognition time for different scales of databases can be controlled approximately. Experiments show that the recognition capability of out scheme decreases a little than ordinary methods,but the recognition speed is increased greatly.