A novel approach based on the Edge Improved Local Binary Pattern (EILBP) visual feature and PLSA model for scene classification was presented. Moreover, the EILBP features could not only capture the distribution information of the local edge gradient and direction, but also obtain the local structures information for describing image. At first, EILBP features were extracted from edge regions as visual words, and then these visual words were formed by clustering method. After that, the Bag-Of-Words (BOF) model was used to represent the image contents. At last, the potential semantics was excavated by PLSA model and the confusion matrix was obtained by KNN classifier. Experiment results show that this method achieves higher accuracies, especially performs well in the images with much edge contours and also it needn’t require experts to annotate the scene content in advance.