The accuracy of head pose estimation is easy to be affected by illumination, occlusion and other disturbances in practical applications and a large number of calculations are difficult to meet timeliness of practical applications. In order to solve these problems, a method based on multi-scale convolutional neural network is proposed. The feature extraction of the input head pose image is performed by using different scale convolution kernels, which enriches the image features while preserving the image information, and enhances the robustness of the algorithm to the interference factors. At the same time, the 1×1 convolution is introduced to reduce the network structure parameters, reduce the computational complexity of the system, and improve the timeliness of the algorithm. The result of experiment shows that the recognition rates of the proposed algorithm on Pointing'04 and CAS-PEAL-R1 databases are 96.5% and 98.9%, respectively. The method shows good robustness to illumination, expression, occlusion and other disturbances, and has better operation and speed.