In order to solve the problem that the traditional methods of object shape classification spend too much training time and the shape is represented inaccurately, an image classification method is proposed based on the improved Bayesian program learning. Firstly, the preprocessed object contours are segmented into fixed-length fragments and the feature information is represented with the shape descriptors. Then, the contour fragments in the same object class are trained into a contour fragment library using the Gaussian mixture model. Finally, the Bayesian classifier is used to calculate the similarity between the ten fragments of the test object and each contour fragment library, and the classification result is the category with the highest similarity value. The experimental results on standard Animal database show that the proposed method has a good classification accuracy, meanwhile, it greatly shortens the training time.