A model pruning method based on gray correlation analysis is proposed to solve the problem that the convolutional neural network cannot be deployed on embedded devices due to the huge computation and memory space. For the weight model file after data training, the importance of each convolution kernel is quantized by using the pruning method based on gray correlation analysis. In each pruning, the convolution kernel with the minimum quantization result is deleted from the model so as to reduce the computation and accelerate the inferential speed. Iteration training is used to compensate for the performance loss of the new model. The experimental results show that compared with APoZ method and L1 method, the accuracy of the proposed method increases by 5.3% and 10.4% at the same inferential speed, the acceleration effect of VGG-16 model is 2.7 times that of the original model, and the memory space is reduced to 1/13.5.