ing at the problem that the shallow machine learning algorithm commonly used in remote sensing image classification application cannot satisfy the classification accuracy in the current mass remote sensing image data environment, we propose a method to apply the fully convolution neural network to the remote sensing image classification. To reduce the loss of image feature map in the pooling process, we add the fusion of the pool layer and the deconvolution layer. To improve the reliability of fusion, we add the scale layer. To obtain finer edge classification results, considering the spatial correlation between pixels mean-shift clustering is used to obtain the spatial relationship of pixels. Classes of regional objects are determined by the maximum sum and the minimum variance of the regional pixel probabilities. Images of typical regions are chosen to carry out the classification experiments, and the classification method proposed in this paper is compared with those of the fully convolution neural network, support vector machine, and artificial neural network. The results show that the accuracy of the classification method proposed in this paper is obviously higher than that of the traditional machine learning methods.