In this paper, two improved network models, SVM (Linear)-based VGG19 and XGBoost-based VGG19, are constructed by combining the VGG19 convolutional neural network with two machine learning algorithms. Moreover, the VGG19 model and the two improved models are employed to classify bacterial pneumonia and viral pneumonia images. Additionally, the performances of the three models are evaluated and compared, the results show that the average accuracies of the three models are all above 85.9%. The improved VGG19 models show superior stability in accuracy over conventional VGG19 model, and the comprehensive performance of XGBoost-based VGG19 model is best, which verifies the effectiveness of deep learning models combined with machine learning models.