In view of the high resolution remote sensing image with multi-scale, complex spatial distribution and the characteristics of a wide range of features, the method of high resolution remote sensing image classification is proposed based on multi-scale and multi-feature fusion, which is starting with the scale effect of feature extraction from remote sensing image and various conspicuousness of different objects. The optimal segmentation scale function is constructed using the method. The optimal scales of different objects are obtained, and texture, color and shape features are extracted respectively. The multi-scale and multi-feature weighted fusion is realized by using significant differences of different objects in characteristics. The weighted fusion method breaks through the limitation of the conventional optimal scale segmentation algorithm, which fails to fully consider the diversity of all kinds of features of different objects. By analyzing the significance of all kinds of features, a model is established based on the weight of each feature. Experimental results show that the accuracy of this method is increased by about 7% compared with that of the traditional unsupervised classification algorithms, and the operation efficiency is high.