In order to solve the problem that the deep learning model PointNet only uses independent point convolution for feature extraction, which leads to the lack of local information, a fusion graph convolution deep learning model based on spatial domain features and spectral domain features is proposed in this paper. In this model, the graph structure is constructed by spatial and spectral methods to extract different neighborhood features, the deep abstract features are obtained by fusing neighborhood features and independent point features, the spatial pyramid pooling method is used to deepen the fine-grained description in the pooling layer. Experimental results on airborne LiDAR scanning point clouds and multispectral aerial images provided by the International Photogrammetry and Remote Sensing Association show that compared with other comparison methods, the classification effect of the method is better, the classification accuracy is 84.3%, and urban scenes can be realized effective classification of point cloud data.