To address the issue of the sparsity of image text data and the limitation of traditional image features, this study proposes an image annotation algorithm that combines a convolutional neural network (CNN) and a topic model. Herein, a Dirichlet topic model is used to model text data on image training sets and generate text topic distribution and text topic label distribution, which reduces the dimension and sparsity of image text data. Considering the sparse distribution of image text topic, the CNN is used to extract high-level visual image features, and the loss function is improved to reconstruct the CNN. Multiple classifiers are constructed based on the high-level visual image features and corresponding multi-text topics to perform multi-label classification learning on image text topics and obtain the text-topic distribution of image. Finally, the text-topic distribution and text-topic label distribution are combined to calculate the probability of the image label. Based on the contrast experiment on Corel5K and IAPR TC-12 image annotation datasets, the proposed algorithm effectively improves the performance of image annotation.