According to the requirements of real-time and accuracy of people detection, we propose the support vector machine based on optimized kernel function in people detection, which uses histogram of oriented gradients algorithm to extract the features of people and the support vector machine algorithm as the classifier. On the basis of the traditional algorithm, we propose the combined kernel function as the kernel function of the classifier. After setting the slack variable and introducing the penalty factor, we combine genetic algorithm and K-fold cross validation optimization to select and optimize the combination coefficients and parameters, and build the final classifier for people detection based on the optimize parameters. Results show that the proposed algorithm achieves better result, and can satisfy the requirement of real-time and accuracy in people detection.