Since the Pyramid Histogram of Oriented Gradients (PHOG) has a poor performance in describing the shape of faces with noise or abrupt intensity changes, a face recognition algorithm based on improved PHOG is proposed. Firstly, we improve the PHOG feature used in the clear outline face recognition to describe the further refinement of the local structure of the face. Then the noise is restrained through the improved normalized method. Finally, KPCA is used to project improved PHOG feature into the more expressive kernel space to further select the feature with strong descriptive ability and the nearest method is adopted for classification. The experimental results show that the characteristics combined improved PHOG with KPCA has obvious advantages in face recognition, and the experimental results on ORL, FERET and YALE face database can achieve high face recognition rate up to 98%，95% and 98.67%. It is shown that theproposed method has better effect on noise suppression and improving the recognition rate.