Hyperspectral image classification is one of the research hotspots in the field of remote sensing. It is an important means of earth observation and has important applications in areas such as fine identification of ground objects. The use of convolutional neural networks (CNN) can effectively extract advanced features from the original image with high classification accuracy. However, CNN has a huge amount of calculations and requires high-performance hardware. In order to improve the computational efficiency of the model, the CNN model can be trained on the GPU. Existing parallel algorithms such as GCN (GPU based Cube-CNN) cannot make full use of the parallel capabilities of the GPU, and the algorithm acceleration effect is not ideal. In order to further improve the efficiency of the algorithm, the GGCN (GPU based Cube-CNN improved by GEMM) parallel acceleration algorithm based on the general matrix multiply (GEMM) algorithm is proposed. G-PNPE(GEMM based Parallel Neighbor Pixels Extraction) reorganizes and arranges the input data and convolution kernel to achieve parallel calculation of convolution, which effectively improves the utilization of GPU and increases the training efficiency of the algorithm. By analyzing the experimental results on the three datasets, the classification accuracy of the improved algorithm is consistent with the original algorithm, and the training time of the CNN network is shortened by about 30%, which proves the effectiveness and superiority of the algorithm.