Gaussian radial basis function is usually applied as the kernel function of the kernel fisher discriminant analysis (KFD) in face recognition application. However, the parameter σ of the kernel function has a great impact on the classification. At present, the parameter is usually selected based on experience, and the process of KFD costs too much time for dealing with a large number of samples. To solve these problems, a method of face recognition is presented based on wavelet transform and improved KFD. It employs wavelet transform to compress the data of face image. And it applies PSO algorithm to automatically obtain the parameter to enhance the ability of classification when KFD is employed to complete feature extraction. Finally, support vector machine is used for classification. Numerical experimental results show that the method has a better operational efficiency and more accurate recognition rate than the traditional method of KFD.