We propose a method for face affine registration based on a single template. Firstly, in order to overcome the local deformation caused by face affine transformation, the color feature is introduced to balance the color similarity and shape mismatch rate between the template face image and the target face image, and then the face rough search algorithm is proposed based on the color feature. Secondly, the affine transformation obtained by face rough search algorithm is used as an initial constraint to establish the face shape fine registration algorithm based on the previous affine constraints. In each iteration step of the algorithm, the affine transformation obtained in the previous iteration is used to establish the correspondence of the nearest point, and the new affine transformation is solved by use of the objective function based on the previous affine constraint. The proposed algorithm successfully solves the problem that a face shape is difficult to register under the conditions of presence of rotation, scale transformation, and noise interference. Compared with the traditional face affine registration algorithm, the proposed algorithm effectively improves the robustness and accuracy of face affine registration.